-- Logs begin at Sun 2022-08-07 10:14:18 UTC, end at Sun 2022-08-07 11:21:10 UTC. -- Aug 07 10:40:28.325539 dv-cinder-850320-3-iscsi systemd[1]: Starting Devstack devstack@keystone.service... Aug 07 10:40:28.333864 dv-cinder-850320-3-iscsi devstack@keystone.service[80133]: [uWSGI] getting INI configuration from /etc/keystone/keystone-uwsgi-public.ini Aug 07 10:40:28.335919 dv-cinder-850320-3-iscsi devstack@keystone.service[80133]: *** Starting uWSGI 2.0.18-debian (64bit) on [Sun Aug 7 10:40:28 2022] *** Aug 07 10:40:28.335919 dv-cinder-850320-3-iscsi devstack@keystone.service[80133]: compiled with version: 10.0.1 20200405 (experimental) [master revision 0be9efad938:fcb98e4978a:705510a708d3642c9c962beb663c476167e4e8a4] on 11 April 2020 11:15:55 Aug 07 10:40:28.335919 dv-cinder-850320-3-iscsi devstack@keystone.service[80133]: os: Linux-5.4.0-72-generic #80-Ubuntu SMP Mon Apr 12 17:35:00 UTC 2021 Aug 07 10:40:28.335919 dv-cinder-850320-3-iscsi devstack@keystone.service[80133]: nodename: dv-cinder-850320-3-iscsi Aug 07 10:40:28.335919 dv-cinder-850320-3-iscsi devstack@keystone.service[80133]: machine: x86_64 Aug 07 10:40:28.335919 dv-cinder-850320-3-iscsi devstack@keystone.service[80133]: clock source: unix Aug 07 10:40:28.335919 dv-cinder-850320-3-iscsi devstack@keystone.service[80133]: pcre jit disabled Aug 07 10:40:28.335919 dv-cinder-850320-3-iscsi devstack@keystone.service[80133]: detected number of CPU cores: 4 Aug 07 10:40:28.335919 dv-cinder-850320-3-iscsi devstack@keystone.service[80133]: current working directory: / Aug 07 10:40:28.335919 dv-cinder-850320-3-iscsi devstack@keystone.service[80133]: detected binary path: /usr/bin/uwsgi-core Aug 07 10:40:28.335919 dv-cinder-850320-3-iscsi devstack@keystone.service[80133]: your processes number limit is 47680 Aug 07 10:40:28.335919 dv-cinder-850320-3-iscsi devstack@keystone.service[80133]: your memory page size is 4096 bytes Aug 07 10:40:28.335919 dv-cinder-850320-3-iscsi devstack@keystone.service[80133]: detected max file descriptor number: 2048 Aug 07 10:40:28.335919 dv-cinder-850320-3-iscsi devstack@keystone.service[80133]: lock engine: pthread robust mutexes Aug 07 10:40:28.336638 dv-cinder-850320-3-iscsi devstack@keystone.service[80133]: thunder lock: enabled Aug 07 10:40:28.336638 dv-cinder-850320-3-iscsi devstack@keystone.service[80133]: uwsgi socket 0 bound to UNIX address /var/run/uwsgi/keystone-wsgi-public.socket fd 4 Aug 07 10:40:28.336638 dv-cinder-850320-3-iscsi devstack@keystone.service[80133]: Python version: 3.8.10 (default, Jun 22 2022, 20:18:18) [GCC 9.4.0] Aug 07 10:40:28.382471 dv-cinder-850320-3-iscsi devstack@keystone.service[80133]: Python main interpreter initialized at 0x55a6111579c0 Aug 07 10:40:28.382471 dv-cinder-850320-3-iscsi devstack@keystone.service[80133]: python threads support enabled Aug 07 10:40:28.382471 dv-cinder-850320-3-iscsi devstack@keystone.service[80133]: your server socket listen backlog is limited to 100 connections Aug 07 10:40:28.382471 dv-cinder-850320-3-iscsi devstack@keystone.service[80133]: your mercy for graceful operations on workers is 90 seconds Aug 07 10:40:28.382837 dv-cinder-850320-3-iscsi devstack@keystone.service[80133]: mapped 403077 bytes (393 KB) for 2 cores Aug 07 10:40:28.382913 dv-cinder-850320-3-iscsi devstack@keystone.service[80133]: *** Operational MODE: preforking *** Aug 07 10:40:28.383173 dv-cinder-850320-3-iscsi devstack@keystone.service[80133]: *** uWSGI is running in multiple interpreter mode *** Aug 07 10:40:28.383173 dv-cinder-850320-3-iscsi devstack@keystone.service[80133]: spawned uWSGI master process (pid: 80133) Aug 07 10:40:28.383365 dv-cinder-850320-3-iscsi systemd[1]: Started Devstack devstack@keystone.service. Aug 07 10:40:28.384257 dv-cinder-850320-3-iscsi devstack@keystone.service[80133]: spawned uWSGI worker 1 (pid: 80143, cores: 1) Aug 07 10:40:28.384257 dv-cinder-850320-3-iscsi devstack@keystone.service[80133]: spawned uWSGI worker 2 (pid: 80144, cores: 1) Aug 07 10:40:28.384440 dv-cinder-850320-3-iscsi devstack@keystone.service[80133]: running "unix_signal:15 gracefully_kill_them_all" (master-start)... Aug 07 10:40:33.880291 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: /usr/lib/python3/dist-packages/secretstorage/dhcrypto.py:15: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Aug 07 10:40:33.880291 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: from cryptography.utils import int_from_bytes Aug 07 10:40:33.881080 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: /usr/lib/python3/dist-packages/secretstorage/util.py:19: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Aug 07 10:40:33.881080 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: from cryptography.utils import int_from_bytes Aug 07 10:40:33.889536 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: /usr/lib/python3/dist-packages/secretstorage/dhcrypto.py:15: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Aug 07 10:40:33.889536 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: from cryptography.utils import int_from_bytes Aug 07 10:40:33.890005 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: /usr/lib/python3/dist-packages/secretstorage/util.py:19: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Aug 07 10:40:33.890005 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: from cryptography.utils import int_from_bytes Aug 07 10:40:33.929458 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=80144) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2589}} Aug 07 10:40:33.929645 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG uwsgi [-] Configuration options gathered from: {{(pid=80144) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2590}} Aug 07 10:40:33.929836 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG uwsgi [-] command line args: [] {{(pid=80144) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2591}} Aug 07 10:40:33.930103 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG uwsgi [-] config files: ['/etc/keystone/keystone.conf'] {{(pid=80144) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2592}} Aug 07 10:40:33.930280 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG uwsgi [-] ================================================================================ {{(pid=80144) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2594}} Aug 07 10:40:33.930598 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG uwsgi [-] admin_token = **** {{(pid=80144) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 07 10:40:33.930907 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG uwsgi [-] config_dir = [] {{(pid=80144) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 07 10:40:33.931163 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG uwsgi [-] config_file = ['/etc/keystone/keystone.conf'] {{(pid=80144) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 07 10:40:33.931366 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG uwsgi [-] config_source = [] {{(pid=80144) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 07 10:40:33.931580 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG uwsgi [-] debug = True {{(pid=80144) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 07 10:40:33.931798 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG uwsgi [-] default_log_levels = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'dogpile=INFO', 'routes=INFO'] {{(pid=80144) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 07 10:40:33.933028 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG uwsgi [-] default_publisher_id = None {{(pid=80144) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 07 10:40:33.933028 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG uwsgi [-] insecure_debug = False {{(pid=80144) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 07 10:40:33.933028 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG uwsgi [-] instance_format = [instance: %(uuid)s] {{(pid=80144) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 07 10:40:33.933028 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG uwsgi [-] instance_uuid_format = [instance: %(uuid)s] {{(pid=80144) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 07 10:40:33.933283 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG uwsgi [-] list_limit = None {{(pid=80144) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 07 10:40:33.933496 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG uwsgi [-] log_config_append = None {{(pid=80144) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 07 10:40:33.933714 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG uwsgi [-] log_date_format = %Y-%m-%d %H:%M:%S {{(pid=80144) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 07 10:40:33.933914 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG uwsgi [-] log_dir = None {{(pid=80144) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 07 10:40:33.934683 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG uwsgi [-] log_file = None {{(pid=80144) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 07 10:40:33.935113 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG uwsgi [-] log_rotate_interval = 1 {{(pid=80144) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 07 10:40:33.935475 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG uwsgi [-] log_rotate_interval_type = days {{(pid=80144) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 07 10:40:33.935849 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG uwsgi [-] log_rotation_type = none {{(pid=80144) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 07 10:40:33.936148 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG uwsgi [-] logging_context_format_string = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=80144) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 07 10:40:33.936330 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG uwsgi [-] logging_debug_format_suffix = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=80144) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 07 10:40:33.936510 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG uwsgi [-] logging_default_format_string = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=80144) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 07 10:40:33.936792 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG uwsgi [-] logging_exception_prefix = ERROR %(name)s %(instance)s {{(pid=80144) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 07 10:40:33.937075 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG uwsgi [-] logging_user_identity_format = %(user)s %(project)s %(domain)s %(system_scope)s %(user_domain)s %(project_domain)s {{(pid=80144) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 07 10:40:33.937355 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG uwsgi [-] max_logfile_count = 30 {{(pid=80144) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 07 10:40:33.937648 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG uwsgi [-] max_logfile_size_mb = 200 {{(pid=80144) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 07 10:40:33.937940 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG uwsgi [-] max_param_size = 64 {{(pid=80144) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 07 10:40:33.938236 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG uwsgi [-] max_project_tree_depth = 5 {{(pid=80144) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 07 10:40:33.938314 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=80143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2589}} Aug 07 10:40:33.938429 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG uwsgi [-] Configuration options gathered from: {{(pid=80143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2590}} Aug 07 10:40:33.938542 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG uwsgi [-] max_token_size = 16384 {{(pid=80144) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 07 10:40:33.938633 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG uwsgi [-] command line args: [] {{(pid=80143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2591}} Aug 07 10:40:33.938847 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG uwsgi [-] notification_format = cadf {{(pid=80144) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 07 10:40:33.938949 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG uwsgi [-] config files: ['/etc/keystone/keystone.conf'] {{(pid=80143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2592}} Aug 07 10:40:33.939083 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG uwsgi [-] ================================================================================ {{(pid=80143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2594}} Aug 07 10:40:33.939256 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG uwsgi [-] notification_opt_out = ['identity.authenticate.success', 'identity.authenticate.pending', 'identity.authenticate.failed'] {{(pid=80144) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 07 10:40:33.939348 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG uwsgi [-] admin_token = **** {{(pid=80143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 07 10:40:33.939573 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG uwsgi [-] config_dir = [] {{(pid=80143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 07 10:40:33.939763 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG uwsgi [-] config_file = ['/etc/keystone/keystone.conf'] {{(pid=80143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 07 10:40:33.939841 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG uwsgi [-] public_endpoint = http://10.222.0.25/identity {{(pid=80144) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 07 10:40:33.939947 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG uwsgi [-] config_source = [] {{(pid=80143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 07 10:40:33.940084 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG uwsgi [-] publish_errors = False {{(pid=80144) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 07 10:40:33.940164 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG uwsgi [-] debug = True {{(pid=80143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 07 10:40:33.940339 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG uwsgi [-] pydev_debug_host = None {{(pid=80144) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 07 10:40:33.940426 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG uwsgi [-] default_log_levels = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'dogpile=INFO', 'routes=INFO'] {{(pid=80143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 07 10:40:33.940554 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG uwsgi [-] default_publisher_id = None {{(pid=80143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 07 10:40:33.940636 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG uwsgi [-] pydev_debug_port = None {{(pid=80144) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 07 10:40:33.940735 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG uwsgi [-] insecure_debug = False {{(pid=80143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 07 10:40:33.940826 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG uwsgi [-] rate_limit_burst = 0 {{(pid=80144) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 07 10:40:33.940986 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG uwsgi [-] instance_format = [instance: %(uuid)s] {{(pid=80143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 07 10:40:33.941234 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG uwsgi [-] instance_uuid_format = [instance: %(uuid)s] {{(pid=80143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 07 10:40:33.941317 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG uwsgi [-] rate_limit_except_level = CRITICAL {{(pid=80144) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 07 10:40:33.941464 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG uwsgi [-] list_limit = None {{(pid=80143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 07 10:40:33.941645 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG uwsgi [-] log_config_append = None {{(pid=80143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 07 10:40:33.941729 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG uwsgi [-] rate_limit_interval = 0 {{(pid=80144) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 07 10:40:33.941808 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG uwsgi [-] log_date_format = %Y-%m-%d %H:%M:%S {{(pid=80143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 07 10:40:33.941987 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG uwsgi [-] log_dir = None {{(pid=80143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 07 10:40:33.942166 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG uwsgi [-] log_file = None {{(pid=80143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 07 10:40:33.942250 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG uwsgi [-] standard_threads = False {{(pid=80144) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 07 10:40:33.942399 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG uwsgi [-] log_rotate_interval = 1 {{(pid=80143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 07 10:40:33.942645 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG uwsgi [-] log_rotate_interval_type = days {{(pid=80143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 07 10:40:33.942739 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG uwsgi [-] strict_password_check = False {{(pid=80144) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 07 10:40:33.942891 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG uwsgi [-] log_rotation_type = none {{(pid=80143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 07 10:40:33.943160 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG uwsgi [-] logging_context_format_string = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=80143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 07 10:40:33.943242 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG uwsgi [-] syslog_log_facility = LOG_USER {{(pid=80144) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 07 10:40:33.943340 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG uwsgi [-] logging_debug_format_suffix = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=80143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 07 10:40:33.943517 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG uwsgi [-] logging_default_format_string = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=80143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 07 10:40:33.943602 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG uwsgi [-] use_eventlog = False {{(pid=80144) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 07 10:40:33.943762 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG uwsgi [-] logging_exception_prefix = ERROR %(name)s %(instance)s {{(pid=80143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 07 10:40:33.943933 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG uwsgi [-] use_journal = False {{(pid=80144) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 07 10:40:33.944019 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG uwsgi [-] logging_user_identity_format = %(user)s %(project)s %(domain)s %(system_scope)s %(user_domain)s %(project_domain)s {{(pid=80143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 07 10:40:33.944227 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG uwsgi [-] max_logfile_count = 30 {{(pid=80143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 07 10:40:33.944305 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG uwsgi [-] use_json = False {{(pid=80144) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 07 10:40:33.944459 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG uwsgi [-] max_logfile_size_mb = 200 {{(pid=80143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 07 10:40:33.944669 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG uwsgi [-] use_stderr = False {{(pid=80144) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 07 10:40:33.944755 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG uwsgi [-] max_param_size = 64 {{(pid=80143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 07 10:40:33.944904 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG uwsgi [-] max_project_tree_depth = 5 {{(pid=80143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 07 10:40:33.945041 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG uwsgi [-] use_syslog = False {{(pid=80144) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 07 10:40:33.945149 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG uwsgi [-] max_token_size = 16384 {{(pid=80143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 07 10:40:33.945393 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG uwsgi [-] notification_format = cadf {{(pid=80143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 07 10:40:33.945542 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG uwsgi [-] watch_log_file = False {{(pid=80144) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 07 10:40:33.945677 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG uwsgi [-] notification_opt_out = ['identity.authenticate.success', 'identity.authenticate.pending', 'identity.authenticate.failed'] {{(pid=80143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 07 10:40:33.946112 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG uwsgi [-] oslo_concurrency.disable_process_locking = False {{(pid=80144) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 10:40:33.946197 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG uwsgi [-] public_endpoint = http://10.222.0.25/identity {{(pid=80143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 07 10:40:33.946337 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG uwsgi [-] publish_errors = False {{(pid=80143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 07 10:40:33.946572 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG uwsgi [-] pydev_debug_host = None {{(pid=80143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 07 10:40:33.946651 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG uwsgi [-] oslo_concurrency.lock_path = None {{(pid=80144) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 10:40:33.946806 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG uwsgi [-] pydev_debug_port = None {{(pid=80143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 07 10:40:33.947016 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG uwsgi [-] rate_limit_burst = 0 {{(pid=80143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 07 10:40:33.947268 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG uwsgi [-] rate_limit_except_level = CRITICAL {{(pid=80143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 07 10:40:33.947349 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG uwsgi [-] profiler.connection_string = messaging:// {{(pid=80144) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 10:40:33.947501 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG uwsgi [-] rate_limit_interval = 0 {{(pid=80143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 07 10:40:33.947735 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG uwsgi [-] standard_threads = False {{(pid=80143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 07 10:40:33.947819 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG uwsgi [-] profiler.enabled = False {{(pid=80144) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 10:40:33.947965 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG uwsgi [-] strict_password_check = False {{(pid=80143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 07 10:40:33.948208 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG uwsgi [-] syslog_log_facility = LOG_USER {{(pid=80143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 07 10:40:33.948386 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG uwsgi [-] use_eventlog = False {{(pid=80143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 07 10:40:33.948471 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG uwsgi [-] profiler.es_doc_type = notification {{(pid=80144) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 10:40:33.948565 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG uwsgi [-] use_journal = False {{(pid=80143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 07 10:40:33.948741 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG uwsgi [-] use_json = False {{(pid=80143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 07 10:40:33.948915 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG uwsgi [-] use_stderr = False {{(pid=80143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 07 10:40:33.949005 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG uwsgi [-] profiler.es_scroll_size = 10000 {{(pid=80144) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 10:40:33.949079 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG uwsgi [-] use_syslog = False {{(pid=80143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 07 10:40:33.949307 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG uwsgi [-] watch_log_file = False {{(pid=80143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 07 10:40:33.949561 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG uwsgi [-] oslo_concurrency.disable_process_locking = False {{(pid=80143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 10:40:33.949645 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG uwsgi [-] profiler.es_scroll_time = 2m {{(pid=80144) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 10:40:33.950081 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG uwsgi [-] oslo_concurrency.lock_path = None {{(pid=80143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 10:40:33.950249 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG uwsgi [-] profiler.filter_error_trace = False {{(pid=80144) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 10:40:33.950422 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG uwsgi [-] profiler.connection_string = messaging:// {{(pid=80143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 10:40:33.950671 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG uwsgi [-] profiler.enabled = False {{(pid=80143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 10:40:33.950758 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG uwsgi [-] profiler.hmac_keys = SECRET_KEY {{(pid=80144) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 10:40:33.950946 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG uwsgi [-] profiler.es_doc_type = notification {{(pid=80143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 10:40:33.951190 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG uwsgi [-] profiler.es_scroll_size = 10000 {{(pid=80143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 10:40:33.951286 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG uwsgi [-] profiler.sentinel_service_name = mymaster {{(pid=80144) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 10:40:33.951444 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG uwsgi [-] profiler.es_scroll_time = 2m {{(pid=80143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 10:40:33.951682 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG uwsgi [-] profiler.filter_error_trace = False {{(pid=80143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 10:40:33.951802 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG uwsgi [-] profiler.socket_timeout = 0.1 {{(pid=80144) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 10:40:33.951925 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG uwsgi [-] profiler.hmac_keys = SECRET_KEY {{(pid=80143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 10:40:33.952091 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG uwsgi [-] profiler.trace_sqlalchemy = False {{(pid=80144) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 10:40:33.952222 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG uwsgi [-] profiler.sentinel_service_name = mymaster {{(pid=80143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 10:40:33.952339 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=80144) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 10:40:33.952461 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG uwsgi [-] profiler.socket_timeout = 0.1 {{(pid=80143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 10:40:33.952584 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_enabled = False {{(pid=80144) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 10:40:33.952701 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG uwsgi [-] profiler.trace_sqlalchemy = False {{(pid=80143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 10:40:33.952815 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_process_name = {{(pid=80144) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 10:40:33.952943 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=80143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 10:40:33.953067 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=80144) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 10:40:33.953184 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_enabled = False {{(pid=80143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 10:40:33.953304 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=80144) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 10:40:33.953421 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_process_name = {{(pid=80143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 10:40:33.953535 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG uwsgi [-] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=80144) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 10:40:33.953730 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=80143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 10:40:33.953856 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=80143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 10:40:33.953976 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG uwsgi [-] audit.namespace = openstack {{(pid=80144) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 10:40:33.954092 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG uwsgi [-] oslo_policy.enforce_new_defaults = False {{(pid=80144) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 10:40:33.954205 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG uwsgi [-] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=80143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 10:40:33.954329 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG uwsgi [-] oslo_policy.enforce_scope = False {{(pid=80144) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 10:40:33.954445 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG uwsgi [-] audit.namespace = openstack {{(pid=80143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 10:40:33.954559 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG uwsgi [-] oslo_policy.policy_default_rule = default {{(pid=80144) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 10:40:33.954681 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG uwsgi [-] oslo_policy.enforce_new_defaults = False {{(pid=80143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 10:40:33.954681 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG uwsgi [-] oslo_policy.enforce_scope = False {{(pid=80143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 10:40:33.954905 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG uwsgi [-] oslo_policy.policy_dirs = ['policy.d'] {{(pid=80144) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 10:40:33.955063 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG uwsgi [-] oslo_policy.policy_default_rule = default {{(pid=80143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 10:40:33.955191 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG uwsgi [-] oslo_policy.policy_file = policy.yaml {{(pid=80144) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 10:40:33.955306 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG uwsgi [-] oslo_policy.policy_dirs = ['policy.d'] {{(pid=80143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 10:40:33.955426 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG uwsgi [-] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=80144) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 10:40:33.955426 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG uwsgi [-] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=80144) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 10:40:33.955649 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG uwsgi [-] oslo_policy.policy_file = policy.yaml {{(pid=80143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 10:40:33.955777 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_crt_file = None {{(pid=80144) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 10:40:33.955898 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG uwsgi [-] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=80143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 10:40:33.956014 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_key_file = None {{(pid=80144) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 10:40:33.956129 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG uwsgi [-] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=80143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 10:40:33.956249 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG uwsgi [-] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=80144) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 10:40:33.956364 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_crt_file = None {{(pid=80143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 10:40:33.956487 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG uwsgi [-] keystone_authtoken.admin_password = **** {{(pid=80144) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 10:40:33.956603 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_key_file = None {{(pid=80143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 10:40:33.956603 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG uwsgi [-] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=80143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 10:40:33.956818 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG uwsgi [-] keystone_authtoken.admin_tenant_name = admin {{(pid=80144) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 10:40:33.956818 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG uwsgi [-] keystone_authtoken.admin_token = **** {{(pid=80144) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 10:40:33.957032 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG uwsgi [-] keystone_authtoken.admin_password = **** {{(pid=80143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 10:40:33.957155 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG uwsgi [-] keystone_authtoken.admin_user = None {{(pid=80144) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 10:40:33.957270 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG uwsgi [-] keystone_authtoken.admin_tenant_name = admin {{(pid=80143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 10:40:33.957384 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG uwsgi [-] keystone_authtoken.auth_admin_prefix = {{(pid=80144) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 10:40:33.957497 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG uwsgi [-] keystone_authtoken.admin_token = **** {{(pid=80143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 10:40:33.957497 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG uwsgi [-] keystone_authtoken.admin_user = None {{(pid=80143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 10:40:33.957711 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG uwsgi [-] keystone_authtoken.auth_host = 127.0.0.1 {{(pid=80144) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 10:40:33.957711 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG uwsgi [-] keystone_authtoken.auth_port = 35357 {{(pid=80144) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 10:40:33.957939 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG uwsgi [-] keystone_authtoken.auth_admin_prefix = {{(pid=80143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 10:40:33.958073 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG uwsgi [-] keystone_authtoken.auth_protocol = https {{(pid=80144) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 10:40:33.958194 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG uwsgi [-] keystone_authtoken.auth_host = 127.0.0.1 {{(pid=80143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 10:40:33.958315 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG uwsgi [-] keystone_authtoken.auth_section = None {{(pid=80144) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 10:40:33.958431 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG uwsgi [-] keystone_authtoken.auth_port = 35357 {{(pid=80143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 10:40:33.958554 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG uwsgi [-] keystone_authtoken.auth_type = None {{(pid=80144) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 10:40:33.958676 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG uwsgi [-] keystone_authtoken.auth_protocol = https {{(pid=80143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 10:40:33.958797 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG uwsgi [-] keystone_authtoken.auth_uri = None {{(pid=80144) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 10:40:33.958950 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG uwsgi [-] keystone_authtoken.auth_section = None {{(pid=80143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 10:40:33.959068 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG uwsgi [-] keystone_authtoken.auth_version = None {{(pid=80144) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 10:40:33.959183 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG uwsgi [-] keystone_authtoken.auth_type = None {{(pid=80143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 10:40:33.959376 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG uwsgi [-] keystone_authtoken.cache = None {{(pid=80144) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 10:40:33.959501 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG uwsgi [-] keystone_authtoken.auth_uri = None {{(pid=80143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 10:40:33.959618 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG uwsgi [-] keystone_authtoken.cafile = None {{(pid=80144) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 10:40:33.959732 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG uwsgi [-] keystone_authtoken.auth_version = None {{(pid=80143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 10:40:33.959845 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG uwsgi [-] keystone_authtoken.certfile = None {{(pid=80144) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 10:40:33.959961 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG uwsgi [-] keystone_authtoken.cache = None {{(pid=80143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 10:40:33.960084 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG uwsgi [-] keystone_authtoken.delay_auth_decision = False {{(pid=80144) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 10:40:33.960200 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG uwsgi [-] keystone_authtoken.cafile = None {{(pid=80143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 10:40:33.960320 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG uwsgi [-] keystone_authtoken.enforce_token_bind = permissive {{(pid=80144) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 10:40:33.960436 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG uwsgi [-] keystone_authtoken.certfile = None {{(pid=80143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 10:40:33.960540 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG uwsgi [-] keystone_authtoken.http_connect_timeout = None {{(pid=80144) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 10:40:33.960637 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG uwsgi [-] keystone_authtoken.delay_auth_decision = False {{(pid=80143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 10:40:33.960759 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG uwsgi [-] keystone_authtoken.http_request_max_retries = 3 {{(pid=80144) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 10:40:33.960881 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG uwsgi [-] keystone_authtoken.enforce_token_bind = permissive {{(pid=80143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 10:40:33.960998 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG uwsgi [-] keystone_authtoken.identity_uri = None {{(pid=80144) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 10:40:33.961112 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG uwsgi [-] keystone_authtoken.http_connect_timeout = None {{(pid=80143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 10:40:33.961226 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG uwsgi [-] keystone_authtoken.include_service_catalog = True {{(pid=80144) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 10:40:33.961340 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG uwsgi [-] keystone_authtoken.http_request_max_retries = 3 {{(pid=80143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 10:40:33.961462 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG uwsgi [-] keystone_authtoken.insecure = False {{(pid=80144) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 10:40:33.961578 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG uwsgi [-] keystone_authtoken.identity_uri = None {{(pid=80143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 10:40:33.961693 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG uwsgi [-] keystone_authtoken.interface = internal {{(pid=80144) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 10:40:33.961791 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG uwsgi [-] keystone_authtoken.include_service_catalog = True {{(pid=80143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 10:40:33.961893 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG uwsgi [-] keystone_authtoken.keyfile = None {{(pid=80144) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 10:40:33.961991 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG uwsgi [-] keystone_authtoken.insecure = False {{(pid=80143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 10:40:33.962096 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_conn_get_timeout = 10 {{(pid=80144) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 10:40:33.962203 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG uwsgi [-] keystone_authtoken.interface = internal {{(pid=80143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 10:40:33.962299 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_dead_retry = 300 {{(pid=80144) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 10:40:33.962395 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG uwsgi [-] keystone_authtoken.keyfile = None {{(pid=80143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 10:40:33.962491 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_maxsize = 10 {{(pid=80144) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 10:40:33.962626 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_conn_get_timeout = 10 {{(pid=80143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 10:40:33.962743 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_socket_timeout = 3 {{(pid=80144) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 10:40:33.962855 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_dead_retry = 300 {{(pid=80143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 10:40:33.963023 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_unused_timeout = 60 {{(pid=80144) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 10:40:33.963152 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_maxsize = 10 {{(pid=80143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 10:40:33.963270 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG uwsgi [-] keystone_authtoken.memcache_secret_key = **** {{(pid=80144) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 10:40:33.963394 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_socket_timeout = 3 {{(pid=80143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 10:40:33.963511 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG uwsgi [-] keystone_authtoken.memcache_security_strategy = None {{(pid=80144) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 10:40:33.963632 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_unused_timeout = 60 {{(pid=80143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 10:40:33.963748 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG uwsgi [-] keystone_authtoken.memcache_use_advanced_pool = True {{(pid=80144) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 10:40:33.963859 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG uwsgi [-] keystone_authtoken.memcache_secret_key = **** {{(pid=80143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 10:40:33.963981 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG uwsgi [-] keystone_authtoken.memcached_servers = None {{(pid=80144) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 10:40:33.964106 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG uwsgi [-] keystone_authtoken.memcache_security_strategy = None {{(pid=80143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 10:40:33.964223 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG uwsgi [-] keystone_authtoken.region_name = None {{(pid=80144) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 10:40:33.964351 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG uwsgi [-] keystone_authtoken.memcache_use_advanced_pool = True {{(pid=80143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 10:40:33.964468 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles = ['service'] {{(pid=80144) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 10:40:33.964591 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG uwsgi [-] keystone_authtoken.memcached_servers = None {{(pid=80143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 10:40:33.964698 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles_required = False {{(pid=80144) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 10:40:33.964819 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG uwsgi [-] keystone_authtoken.region_name = None {{(pid=80143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 10:40:33.964944 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG uwsgi [-] keystone_authtoken.service_type = None {{(pid=80144) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 10:40:33.965069 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles = ['service'] {{(pid=80143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 10:40:33.965187 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG uwsgi [-] keystone_authtoken.token_cache_time = 300 {{(pid=80144) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 10:40:33.965292 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles_required = False {{(pid=80143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 10:40:33.965467 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG uwsgi [-] keystone_authtoken.www_authenticate_uri = None {{(pid=80144) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 10:40:33.965635 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG uwsgi [-] keystone_authtoken.service_type = None {{(pid=80143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 10:40:33.965796 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG uwsgi [-] cache.backend = dogpile.cache.memcached {{(pid=80144) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 10:40:33.965928 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG uwsgi [-] keystone_authtoken.token_cache_time = 300 {{(pid=80143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 10:40:33.966052 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG uwsgi [-] cache.backend_argument = **** {{(pid=80144) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 10:40:33.966170 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG uwsgi [-] keystone_authtoken.www_authenticate_uri = None {{(pid=80143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 10:40:33.966286 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG uwsgi [-] cache.config_prefix = cache.oslo {{(pid=80144) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 10:40:33.966391 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG uwsgi [-] cache.backend = dogpile.cache.memcached {{(pid=80143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 10:40:33.966505 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG uwsgi [-] cache.dead_timeout = 60.0 {{(pid=80144) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 10:40:33.966505 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG uwsgi [-] cache.debug_cache_backend = False {{(pid=80144) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 10:40:33.966727 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG uwsgi [-] cache.backend_argument = **** {{(pid=80143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 10:40:33.966848 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG uwsgi [-] cache.enable_retry_client = False {{(pid=80144) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 10:40:33.966980 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG uwsgi [-] cache.config_prefix = cache.oslo {{(pid=80143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 10:40:33.967097 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG uwsgi [-] cache.enable_socket_keepalive = False {{(pid=80144) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 10:40:33.967097 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG uwsgi [-] cache.enabled = True {{(pid=80144) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 10:40:33.967326 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG uwsgi [-] cache.dead_timeout = 60.0 {{(pid=80143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 10:40:33.967446 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG uwsgi [-] cache.expiration_time = 600 {{(pid=80144) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 10:40:33.967562 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG uwsgi [-] cache.debug_cache_backend = False {{(pid=80143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 10:40:33.967684 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG uwsgi [-] cache.hashclient_retry_attempts = 2 {{(pid=80144) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 10:40:33.967800 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG uwsgi [-] cache.enable_retry_client = False {{(pid=80143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 10:40:33.967916 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG uwsgi [-] cache.hashclient_retry_delay = 1.0 {{(pid=80144) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 10:40:33.968042 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG uwsgi [-] cache.enable_socket_keepalive = False {{(pid=80143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 10:40:33.968159 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG uwsgi [-] cache.memcache_dead_retry = 300 {{(pid=80144) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 10:40:33.968274 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG uwsgi [-] cache.enabled = True {{(pid=80143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 10:40:33.968389 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG uwsgi [-] cache.memcache_pool_connection_get_timeout = 10 {{(pid=80144) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 10:40:33.968504 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG uwsgi [-] cache.expiration_time = 600 {{(pid=80143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 10:40:33.968619 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG uwsgi [-] cache.memcache_pool_flush_on_reconnect = False {{(pid=80144) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 10:40:33.968743 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG uwsgi [-] cache.hashclient_retry_attempts = 2 {{(pid=80143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 10:40:33.968860 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG uwsgi [-] cache.memcache_pool_maxsize = 10 {{(pid=80144) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 10:40:33.968976 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG uwsgi [-] cache.hashclient_retry_delay = 1.0 {{(pid=80143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 10:40:33.969091 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG uwsgi [-] cache.memcache_pool_unused_timeout = 60 {{(pid=80144) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 10:40:33.969206 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG uwsgi [-] cache.memcache_dead_retry = 300 {{(pid=80143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 10:40:33.969322 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG uwsgi [-] cache.memcache_servers = ['localhost:11211'] {{(pid=80144) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 10:40:33.969446 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG uwsgi [-] cache.memcache_pool_connection_get_timeout = 10 {{(pid=80143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 10:40:33.969563 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG uwsgi [-] cache.memcache_socket_timeout = 1.0 {{(pid=80144) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 10:40:33.969678 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG uwsgi [-] cache.memcache_pool_flush_on_reconnect = False {{(pid=80143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 10:40:33.969800 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG uwsgi [-] cache.proxies = [] {{(pid=80144) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 10:40:33.969925 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG uwsgi [-] cache.memcache_pool_maxsize = 10 {{(pid=80143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 10:40:33.970042 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG uwsgi [-] cache.retry_attempts = 2 {{(pid=80144) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 10:40:33.970166 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG uwsgi [-] cache.memcache_pool_unused_timeout = 60 {{(pid=80143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 10:40:33.970283 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG uwsgi [-] cache.retry_delay = 0.0 {{(pid=80144) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 10:40:33.970407 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG uwsgi [-] cache.memcache_servers = ['localhost:11211'] {{(pid=80143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 10:40:33.970530 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG uwsgi [-] cache.socket_keepalive_count = 1 {{(pid=80144) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 10:40:33.970646 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG uwsgi [-] cache.memcache_socket_timeout = 1.0 {{(pid=80143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 10:40:33.970762 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG uwsgi [-] cache.socket_keepalive_idle = 1 {{(pid=80144) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 10:40:33.970987 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG uwsgi [-] cache.proxies = [] {{(pid=80143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 10:40:33.971108 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG uwsgi [-] cache.socket_keepalive_interval = 1 {{(pid=80144) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 10:40:33.971225 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG uwsgi [-] cache.retry_attempts = 2 {{(pid=80143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 10:40:33.971340 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG uwsgi [-] cache.tls_allowed_ciphers = None {{(pid=80144) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 10:40:33.971456 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG uwsgi [-] cache.retry_delay = 0.0 {{(pid=80143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 10:40:33.971571 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG uwsgi [-] cache.tls_cafile = None {{(pid=80144) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 10:40:33.971701 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG uwsgi [-] cache.socket_keepalive_count = 1 {{(pid=80143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 10:40:33.971827 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG uwsgi [-] cache.tls_certfile = None {{(pid=80144) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 10:40:33.971953 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG uwsgi [-] cache.socket_keepalive_idle = 1 {{(pid=80143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 10:40:33.972070 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG uwsgi [-] cache.tls_enabled = False {{(pid=80144) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 10:40:33.972186 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG uwsgi [-] cache.socket_keepalive_interval = 1 {{(pid=80143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 10:40:33.972310 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG uwsgi [-] cache.tls_keyfile = None {{(pid=80144) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 10:40:33.972427 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG uwsgi [-] cache.tls_allowed_ciphers = None {{(pid=80143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 10:40:33.972553 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG uwsgi [-] application_credential.cache_time = None {{(pid=80144) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 10:40:33.972670 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG uwsgi [-] cache.tls_cafile = None {{(pid=80143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 10:40:33.972786 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG uwsgi [-] application_credential.caching = True {{(pid=80144) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 10:40:33.972902 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG uwsgi [-] cache.tls_certfile = None {{(pid=80143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 10:40:33.973021 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG uwsgi [-] application_credential.driver = sql {{(pid=80144) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 10:40:33.973137 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG uwsgi [-] cache.tls_enabled = False {{(pid=80143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 10:40:33.973264 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG uwsgi [-] application_credential.user_limit = -1 {{(pid=80144) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 10:40:33.973382 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG uwsgi [-] cache.tls_keyfile = None {{(pid=80143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 10:40:33.973498 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG uwsgi [-] assignment.driver = sql {{(pid=80144) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 10:40:33.973614 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG uwsgi [-] application_credential.cache_time = None {{(pid=80143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 10:40:33.973736 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG uwsgi [-] assignment.prohibited_implied_role = ['admin'] {{(pid=80144) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 10:40:33.973853 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG uwsgi [-] application_credential.caching = True {{(pid=80143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 10:40:33.973976 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG uwsgi [-] auth.application_credential = None {{(pid=80144) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 10:40:33.974094 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG uwsgi [-] application_credential.driver = sql {{(pid=80143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 10:40:33.974210 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG uwsgi [-] auth.external = None {{(pid=80144) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 10:40:33.974325 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG uwsgi [-] application_credential.user_limit = -1 {{(pid=80143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 10:40:33.974440 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG uwsgi [-] auth.mapped = None {{(pid=80144) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 10:40:33.974555 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG uwsgi [-] assignment.driver = sql {{(pid=80143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 10:40:33.974679 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG uwsgi [-] auth.methods = ['external', 'password', 'token', 'oauth1', 'mapped', 'application_credential'] {{(pid=80144) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 10:40:33.974797 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG uwsgi [-] assignment.prohibited_implied_role = ['admin'] {{(pid=80143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 10:40:33.974913 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG uwsgi [-] auth.oauth1 = None {{(pid=80144) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 10:40:33.975089 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG uwsgi [-] auth.application_credential = None {{(pid=80143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 10:40:33.975205 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG uwsgi [-] auth.password = None {{(pid=80144) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 10:40:33.975352 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG uwsgi [-] auth.external = None {{(pid=80143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 10:40:33.975480 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG uwsgi [-] auth.token = None {{(pid=80144) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 10:40:33.975599 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG uwsgi [-] auth.mapped = None {{(pid=80143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 10:40:33.975724 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG uwsgi [-] catalog.cache_time = None {{(pid=80144) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 10:40:33.975842 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG uwsgi [-] auth.methods = ['external', 'password', 'token', 'oauth1', 'mapped', 'application_credential'] {{(pid=80143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 10:40:33.975960 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG uwsgi [-] catalog.caching = True {{(pid=80144) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 10:40:33.976076 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG uwsgi [-] auth.oauth1 = None {{(pid=80143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 10:40:33.976192 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG uwsgi [-] catalog.driver = sql {{(pid=80144) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 10:40:33.976309 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG uwsgi [-] auth.password = None {{(pid=80143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 10:40:33.976433 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG uwsgi [-] catalog.list_limit = None {{(pid=80144) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 10:40:33.976551 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG uwsgi [-] auth.token = None {{(pid=80143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 10:40:33.976669 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG uwsgi [-] catalog.template_file = default_catalog.templates {{(pid=80144) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 10:40:33.976787 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG uwsgi [-] catalog.cache_time = None {{(pid=80143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 10:40:33.976903 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG uwsgi [-] credential.auth_ttl = 15 {{(pid=80144) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 10:40:33.977019 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG uwsgi [-] catalog.caching = True {{(pid=80143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 10:40:33.977143 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG uwsgi [-] credential.cache_time = None {{(pid=80144) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 10:40:33.977262 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG uwsgi [-] catalog.driver = sql {{(pid=80143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 10:40:33.977378 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG uwsgi [-] credential.caching = True {{(pid=80144) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 10:40:33.977494 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG uwsgi [-] catalog.list_limit = None {{(pid=80143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 10:40:33.977610 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG uwsgi [-] credential.driver = sql {{(pid=80144) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 10:40:33.977717 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG uwsgi [-] catalog.template_file = default_catalog.templates {{(pid=80143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 10:40:33.977841 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG uwsgi [-] credential.key_repository = /etc/keystone/credential-keys/ {{(pid=80144) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 10:40:33.977959 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG uwsgi [-] credential.auth_ttl = 15 {{(pid=80143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 10:40:33.978084 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG uwsgi [-] credential.provider = fernet {{(pid=80144) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 10:40:33.978202 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG uwsgi [-] credential.cache_time = None {{(pid=80143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 10:40:33.978318 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG uwsgi [-] credential.user_limit = -1 {{(pid=80144) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 10:40:33.978434 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG uwsgi [-] credential.caching = True {{(pid=80143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 10:40:33.978559 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG uwsgi [-] domain_config.cache_time = 300 {{(pid=80144) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 10:40:33.978677 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG uwsgi [-] credential.driver = sql {{(pid=80143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 10:40:33.978795 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG uwsgi [-] domain_config.caching = True {{(pid=80144) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 10:40:33.978912 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG uwsgi [-] credential.key_repository = /etc/keystone/credential-keys/ {{(pid=80143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 10:40:33.979056 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG uwsgi [-] domain_config.driver = sql {{(pid=80144) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 10:40:33.979172 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG uwsgi [-] credential.provider = fernet {{(pid=80143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 10:40:33.979301 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG uwsgi [-] endpoint_filter.driver = sql {{(pid=80144) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 10:40:33.979427 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG uwsgi [-] credential.user_limit = -1 {{(pid=80143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 10:40:33.979545 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG uwsgi [-] endpoint_filter.return_all_endpoints_if_no_filter = True {{(pid=80144) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 10:40:33.979662 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG uwsgi [-] domain_config.cache_time = 300 {{(pid=80143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 10:40:33.979784 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG uwsgi [-] endpoint_policy.driver = sql {{(pid=80144) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 10:40:33.979903 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG uwsgi [-] domain_config.caching = True {{(pid=80143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 10:40:33.980028 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG uwsgi [-] eventlet_server.admin_bind_host = 0.0.0.0 {{(pid=80144) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 10:40:33.980145 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG uwsgi [-] domain_config.driver = sql {{(pid=80143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 10:40:33.980262 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG uwsgi [-] eventlet_server.admin_port = 35357 {{(pid=80144) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 10:40:33.980387 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG uwsgi [-] endpoint_filter.driver = sql {{(pid=80143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 10:40:33.980504 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG uwsgi [-] eventlet_server.public_bind_host = 0.0.0.0 {{(pid=80144) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 10:40:33.980620 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG uwsgi [-] endpoint_filter.return_all_endpoints_if_no_filter = True {{(pid=80143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 10:40:33.980745 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG uwsgi [-] eventlet_server.public_port = 5000 {{(pid=80144) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 10:40:33.980864 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG uwsgi [-] endpoint_policy.driver = sql {{(pid=80143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 10:40:33.981466 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG uwsgi [-] federation.assertion_prefix = {{(pid=80144) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 10:40:33.981569 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG uwsgi [-] eventlet_server.admin_bind_host = 0.0.0.0 {{(pid=80143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 10:40:33.981686 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG uwsgi [-] federation.caching = True {{(pid=80144) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 10:40:33.981803 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG uwsgi [-] eventlet_server.admin_port = 35357 {{(pid=80143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 10:40:33.981930 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG uwsgi [-] federation.default_authorization_ttl = 0 {{(pid=80144) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 10:40:33.982048 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG uwsgi [-] eventlet_server.public_bind_host = 0.0.0.0 {{(pid=80143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 10:40:33.982165 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG uwsgi [-] federation.driver = sql {{(pid=80144) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 10:40:33.982288 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG uwsgi [-] eventlet_server.public_port = 5000 {{(pid=80143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 10:40:33.982406 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG uwsgi [-] federation.federated_domain_name = Federated {{(pid=80144) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 10:40:33.982406 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG uwsgi [-] federation.remote_id_attribute = None {{(pid=80144) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 10:40:33.982634 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG uwsgi [-] federation.assertion_prefix = {{(pid=80143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 10:40:33.982752 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG uwsgi [-] federation.sso_callback_template = /etc/keystone/sso_callback_template.html {{(pid=80144) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 10:40:33.982870 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG uwsgi [-] federation.caching = True {{(pid=80143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 10:40:33.982870 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG uwsgi [-] federation.default_authorization_ttl = 0 {{(pid=80143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 10:40:33.983158 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG uwsgi [-] federation.trusted_dashboard = [] {{(pid=80144) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 10:40:33.983158 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG uwsgi [-] fernet_receipts.key_repository = /etc/keystone/fernet-keys/ {{(pid=80144) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 10:40:33.983393 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG uwsgi [-] federation.driver = sql {{(pid=80143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 10:40:33.983510 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG uwsgi [-] fernet_receipts.max_active_keys = 3 {{(pid=80144) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 10:40:33.983510 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG uwsgi [-] fernet_tokens.key_repository = /etc/keystone/fernet-keys/ {{(pid=80144) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 10:40:33.983728 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG uwsgi [-] federation.federated_domain_name = Federated {{(pid=80143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 10:40:33.983845 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG uwsgi [-] fernet_tokens.max_active_keys = 3 {{(pid=80144) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 10:40:33.983970 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG uwsgi [-] federation.remote_id_attribute = None {{(pid=80143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 10:40:33.984089 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG uwsgi [-] identity.cache_time = 600 {{(pid=80144) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 10:40:33.984206 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG uwsgi [-] federation.sso_callback_template = /etc/keystone/sso_callback_template.html {{(pid=80143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 10:40:33.984323 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG uwsgi [-] identity.caching = True {{(pid=80144) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 10:40:33.984439 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG uwsgi [-] federation.trusted_dashboard = [] {{(pid=80143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 10:40:33.984556 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG uwsgi [-] identity.default_domain_id = default {{(pid=80144) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 10:40:33.984680 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG uwsgi [-] fernet_receipts.key_repository = /etc/keystone/fernet-keys/ {{(pid=80143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 10:40:33.984805 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG uwsgi [-] identity.domain_config_dir = /etc/keystone/domains {{(pid=80144) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 10:40:33.984924 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG uwsgi [-] fernet_receipts.max_active_keys = 3 {{(pid=80143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 10:40:33.985041 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG uwsgi [-] identity.domain_configurations_from_database = False {{(pid=80144) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 10:40:33.985159 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG uwsgi [-] fernet_tokens.key_repository = /etc/keystone/fernet-keys/ {{(pid=80143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 10:40:33.985276 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG uwsgi [-] identity.domain_specific_drivers_enabled = False {{(pid=80144) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 10:40:33.985482 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG uwsgi [-] fernet_tokens.max_active_keys = 3 {{(pid=80143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 10:40:33.985603 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG uwsgi [-] identity.driver = sql {{(pid=80144) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 10:40:33.985730 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG uwsgi [-] identity.cache_time = 600 {{(pid=80143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 10:40:33.985848 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG uwsgi [-] identity.list_limit = None {{(pid=80144) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 10:40:33.985971 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG uwsgi [-] identity.caching = True {{(pid=80143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 10:40:33.986090 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG uwsgi [-] identity.max_password_length = 4096 {{(pid=80144) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 10:40:33.986462 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG uwsgi [-] identity.default_domain_id = default {{(pid=80143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 10:40:33.986584 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG uwsgi [-] identity.password_hash_algorithm = bcrypt {{(pid=80144) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 10:40:33.986701 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG uwsgi [-] identity.domain_config_dir = /etc/keystone/domains {{(pid=80143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 10:40:33.986818 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG uwsgi [-] identity.password_hash_rounds = 4 {{(pid=80144) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 10:40:33.986965 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG uwsgi [-] identity.domain_configurations_from_database = False {{(pid=80143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 10:40:33.987086 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG uwsgi [-] identity.salt_bytesize = None {{(pid=80144) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 10:40:33.987217 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG uwsgi [-] identity.domain_specific_drivers_enabled = False {{(pid=80143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 10:40:33.987337 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG uwsgi [-] identity.scrypt_block_size = None {{(pid=80144) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 10:40:33.987454 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG uwsgi [-] identity.driver = sql {{(pid=80143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 10:40:33.987571 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG uwsgi [-] identity.scrypt_parallelism = None {{(pid=80144) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 10:40:33.987687 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG uwsgi [-] identity.list_limit = None {{(pid=80143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 10:40:33.987803 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG uwsgi [-] identity_mapping.backward_compatible_ids = True {{(pid=80144) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 10:40:33.987928 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG uwsgi [-] identity.max_password_length = 4096 {{(pid=80143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 10:40:33.988046 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG uwsgi [-] identity_mapping.driver = sql {{(pid=80144) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 10:40:33.988162 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG uwsgi [-] identity.password_hash_algorithm = bcrypt {{(pid=80143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 10:40:33.988288 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG uwsgi [-] identity_mapping.generator = sha256 {{(pid=80144) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 10:40:33.988406 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG uwsgi [-] identity.password_hash_rounds = 4 {{(pid=80143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 10:40:33.988529 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG uwsgi [-] jwt_tokens.jws_private_key_repository = /etc/keystone/jws-keys/private {{(pid=80144) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 10:40:33.988655 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG uwsgi [-] identity.salt_bytesize = None {{(pid=80143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 10:40:33.988773 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG uwsgi [-] jwt_tokens.jws_public_key_repository = /etc/keystone/jws-keys/public {{(pid=80144) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 10:40:33.988891 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG uwsgi [-] identity.scrypt_block_size = None {{(pid=80143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 10:40:33.989008 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG uwsgi [-] ldap.alias_dereferencing = default {{(pid=80144) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 10:40:33.989124 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG uwsgi [-] identity.scrypt_parallelism = None {{(pid=80143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 10:40:33.989241 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG uwsgi [-] ldap.auth_pool_connection_lifetime = 60 {{(pid=80144) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 10:40:33.989367 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG uwsgi [-] identity_mapping.backward_compatible_ids = True {{(pid=80143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 10:40:33.989485 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG uwsgi [-] ldap.auth_pool_size = 100 {{(pid=80144) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 10:40:33.989602 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG uwsgi [-] identity_mapping.driver = sql {{(pid=80143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 10:40:33.989718 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG uwsgi [-] ldap.chase_referrals = None {{(pid=80144) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 10:40:33.989835 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG uwsgi [-] identity_mapping.generator = sha256 {{(pid=80143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 10:40:33.989951 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG uwsgi [-] ldap.connection_timeout = -1 {{(pid=80144) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 10:40:33.990075 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG uwsgi [-] jwt_tokens.jws_private_key_repository = /etc/keystone/jws-keys/private {{(pid=80143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 10:40:33.990194 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG uwsgi [-] ldap.debug_level = None {{(pid=80144) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 10:40:33.990311 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG uwsgi [-] jwt_tokens.jws_public_key_repository = /etc/keystone/jws-keys/public {{(pid=80143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 10:40:33.990428 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG uwsgi [-] ldap.group_ad_nesting = False {{(pid=80144) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 10:40:33.990544 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG uwsgi [-] ldap.alias_dereferencing = default {{(pid=80143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 10:40:33.990660 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG uwsgi [-] ldap.group_additional_attribute_mapping = [] {{(pid=80144) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 10:40:33.990785 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG uwsgi [-] ldap.auth_pool_connection_lifetime = 60 {{(pid=80143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 10:40:33.990903 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG uwsgi [-] ldap.group_attribute_ignore = [] {{(pid=80144) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 10:40:33.991041 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG uwsgi [-] ldap.auth_pool_size = 100 {{(pid=80143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 10:40:33.991158 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG uwsgi [-] ldap.group_desc_attribute = description {{(pid=80144) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 10:40:33.991274 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG uwsgi [-] ldap.chase_referrals = None {{(pid=80143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 10:40:33.991391 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG uwsgi [-] ldap.group_filter = None {{(pid=80144) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 10:40:33.991520 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG uwsgi [-] ldap.connection_timeout = -1 {{(pid=80143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 10:40:33.991639 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG uwsgi [-] ldap.group_id_attribute = cn {{(pid=80144) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 10:40:33.991756 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG uwsgi [-] ldap.debug_level = None {{(pid=80143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 10:40:33.991872 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG uwsgi [-] ldap.group_member_attribute = member {{(pid=80144) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 10:40:33.991989 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG uwsgi [-] ldap.group_ad_nesting = False {{(pid=80143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 10:40:33.992105 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG uwsgi [-] ldap.group_members_are_ids = False {{(pid=80144) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 10:40:33.992230 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG uwsgi [-] ldap.group_additional_attribute_mapping = [] {{(pid=80143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 10:40:33.992347 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG uwsgi [-] ldap.group_name_attribute = ou {{(pid=80144) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 10:40:33.992465 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG uwsgi [-] ldap.group_attribute_ignore = [] {{(pid=80143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 10:40:33.992581 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG uwsgi [-] ldap.group_objectclass = groupOfNames {{(pid=80144) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 10:40:33.992698 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG uwsgi [-] ldap.group_desc_attribute = description {{(pid=80143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 10:40:33.992826 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG uwsgi [-] ldap.group_tree_dn = None {{(pid=80144) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 10:40:33.992944 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG uwsgi [-] ldap.group_filter = None {{(pid=80143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 10:40:33.993065 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG uwsgi [-] ldap.page_size = 0 {{(pid=80144) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 10:40:33.993182 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG uwsgi [-] ldap.group_id_attribute = cn {{(pid=80143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 10:40:33.993298 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG uwsgi [-] ldap.password = **** {{(pid=80144) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 10:40:33.993415 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG uwsgi [-] ldap.group_member_attribute = member {{(pid=80143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 10:40:33.993540 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG uwsgi [-] ldap.pool_connection_lifetime = 600 {{(pid=80144) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 10:40:33.993658 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG uwsgi [-] ldap.group_members_are_ids = False {{(pid=80143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 10:40:33.993775 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG uwsgi [-] ldap.pool_connection_timeout = -1 {{(pid=80144) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 10:40:33.993892 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG uwsgi [-] ldap.group_name_attribute = ou {{(pid=80143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 10:40:33.994016 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG uwsgi [-] ldap.pool_retry_delay = 0.1 {{(pid=80144) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 10:40:33.994016 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG uwsgi [-] ldap.pool_retry_max = 3 {{(pid=80144) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 10:40:33.994244 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG uwsgi [-] ldap.group_objectclass = groupOfNames {{(pid=80143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 10:40:33.994244 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG uwsgi [-] ldap.group_tree_dn = None {{(pid=80143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 10:40:33.994464 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG uwsgi [-] ldap.pool_size = 10 {{(pid=80144) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 10:40:33.994581 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG uwsgi [-] ldap.page_size = 0 {{(pid=80143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 10:40:33.994697 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG uwsgi [-] ldap.query_scope = one {{(pid=80144) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 10:40:33.994813 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG uwsgi [-] ldap.password = **** {{(pid=80143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 10:40:33.994954 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG uwsgi [-] ldap.suffix = cn=example,cn=com {{(pid=80144) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 10:40:33.995075 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG uwsgi [-] ldap.pool_connection_lifetime = 600 {{(pid=80143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 10:40:33.995192 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG uwsgi [-] ldap.tls_cacertdir = None {{(pid=80144) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 10:40:33.995308 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG uwsgi [-] ldap.pool_connection_timeout = -1 {{(pid=80143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 10:40:33.995424 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG uwsgi [-] ldap.tls_cacertfile = None {{(pid=80144) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 10:40:33.995540 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG uwsgi [-] ldap.pool_retry_delay = 0.1 {{(pid=80143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 10:40:33.995669 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG uwsgi [-] ldap.tls_req_cert = demand {{(pid=80144) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 10:40:33.995788 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG uwsgi [-] ldap.pool_retry_max = 3 {{(pid=80143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 10:40:33.995911 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG uwsgi [-] ldap.url = ldap://localhost {{(pid=80144) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 10:40:33.996029 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG uwsgi [-] ldap.pool_size = 10 {{(pid=80143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 10:40:33.996152 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG uwsgi [-] ldap.use_auth_pool = True {{(pid=80144) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 10:40:33.996277 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG uwsgi [-] ldap.query_scope = one {{(pid=80143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 10:40:33.996403 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG uwsgi [-] ldap.use_pool = True {{(pid=80144) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 10:40:33.996403 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG uwsgi [-] ldap.use_tls = False {{(pid=80144) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 10:40:33.997294 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG uwsgi [-] ldap.suffix = cn=example,cn=com {{(pid=80143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 10:40:33.997417 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG uwsgi [-] ldap.user = None {{(pid=80144) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 10:40:33.997535 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG uwsgi [-] ldap.tls_cacertdir = None {{(pid=80143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 10:40:33.997654 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG uwsgi [-] ldap.user_additional_attribute_mapping = [] {{(pid=80144) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 10:40:33.997816 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG uwsgi [-] ldap.tls_cacertfile = None {{(pid=80143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 10:40:33.998038 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG uwsgi [-] ldap.user_attribute_ignore = ['default_project_id'] {{(pid=80144) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 10:40:33.998157 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG uwsgi [-] ldap.tls_req_cert = demand {{(pid=80143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 10:40:33.998275 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG uwsgi [-] ldap.user_default_project_id_attribute = None {{(pid=80144) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 10:40:33.998399 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG uwsgi [-] ldap.url = ldap://localhost {{(pid=80143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 10:40:33.998518 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG uwsgi [-] ldap.user_description_attribute = description {{(pid=80144) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 10:40:33.998644 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG uwsgi [-] ldap.use_auth_pool = True {{(pid=80143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 10:40:33.998764 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG uwsgi [-] ldap.user_enabled_attribute = enabled {{(pid=80144) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 10:40:33.998881 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG uwsgi [-] ldap.use_pool = True {{(pid=80143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 10:40:33.999030 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG uwsgi [-] ldap.user_enabled_default = True {{(pid=80144) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 10:40:33.999148 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG uwsgi [-] ldap.use_tls = False {{(pid=80143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 10:40:33.999265 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG uwsgi [-] ldap.user_enabled_emulation = False {{(pid=80144) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 10:40:33.999394 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG uwsgi [-] ldap.user = None {{(pid=80143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 10:40:33.999515 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG uwsgi [-] ldap.user_enabled_emulation_dn = None {{(pid=80144) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 10:40:33.999632 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG uwsgi [-] ldap.user_additional_attribute_mapping = [] {{(pid=80143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 10:40:33.999750 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG uwsgi [-] ldap.user_enabled_emulation_use_group_config = False {{(pid=80144) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 10:40:33.999750 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG uwsgi [-] ldap.user_enabled_invert = False {{(pid=80144) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 10:40:33.999969 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG uwsgi [-] ldap.user_attribute_ignore = ['default_project_id'] {{(pid=80143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 10:40:34.000095 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG uwsgi [-] ldap.user_enabled_mask = 0 {{(pid=80144) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 10:40:34.000214 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG uwsgi [-] ldap.user_default_project_id_attribute = None {{(pid=80143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 10:40:34.000332 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG uwsgi [-] ldap.user_filter = None {{(pid=80144) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 10:40:34.000448 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG uwsgi [-] ldap.user_description_attribute = description {{(pid=80143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 10:40:34.000564 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG uwsgi [-] ldap.user_id_attribute = cn {{(pid=80144) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 10:40:34.000681 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG uwsgi [-] ldap.user_enabled_attribute = enabled {{(pid=80143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 10:40:34.000807 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG uwsgi [-] ldap.user_mail_attribute = mail {{(pid=80144) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 10:40:34.000908 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG uwsgi [-] ldap.user_enabled_default = True {{(pid=80143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 10:40:34.001024 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG uwsgi [-] ldap.user_name_attribute = sn {{(pid=80144) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 10:40:34.001141 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG uwsgi [-] ldap.user_enabled_emulation = False {{(pid=80143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 10:40:34.001257 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG uwsgi [-] ldap.user_objectclass = inetOrgPerson {{(pid=80144) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 10:40:34.001373 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG uwsgi [-] ldap.user_enabled_emulation_dn = None {{(pid=80143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 10:40:34.001497 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG uwsgi [-] ldap.user_pass_attribute = userPassword {{(pid=80144) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 10:40:34.001615 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG uwsgi [-] ldap.user_enabled_emulation_use_group_config = False {{(pid=80143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 10:40:34.001731 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG uwsgi [-] ldap.user_tree_dn = None {{(pid=80144) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 10:40:34.001848 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG uwsgi [-] ldap.user_enabled_invert = False {{(pid=80143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 10:40:34.001971 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG uwsgi [-] memcache.dead_retry = 300 {{(pid=80144) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 10:40:34.002087 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG uwsgi [-] ldap.user_enabled_mask = 0 {{(pid=80143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 10:40:34.002212 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG uwsgi [-] memcache.pool_connection_get_timeout = 10 {{(pid=80144) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 10:40:34.002385 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG uwsgi [-] ldap.user_filter = None {{(pid=80143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 10:40:34.002505 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG uwsgi [-] memcache.pool_maxsize = 10 {{(pid=80144) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 10:40:34.002622 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG uwsgi [-] ldap.user_id_attribute = cn {{(pid=80143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 10:40:34.002739 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG uwsgi [-] memcache.pool_unused_timeout = 60 {{(pid=80144) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 10:40:34.002856 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG uwsgi [-] ldap.user_mail_attribute = mail {{(pid=80143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 10:40:34.003026 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG uwsgi [-] memcache.socket_timeout = 3 {{(pid=80144) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 10:40:34.003148 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG uwsgi [-] ldap.user_name_attribute = sn {{(pid=80143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 10:40:34.003270 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG uwsgi [-] oauth1.access_token_duration = 86400 {{(pid=80144) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 10:40:34.003388 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG uwsgi [-] ldap.user_objectclass = inetOrgPerson {{(pid=80143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 10:40:34.003505 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG uwsgi [-] oauth1.driver = sql {{(pid=80144) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 10:40:34.003621 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG uwsgi [-] ldap.user_pass_attribute = userPassword {{(pid=80143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 10:40:34.003746 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG uwsgi [-] oauth1.request_token_duration = 28800 {{(pid=80144) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 10:40:34.003863 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG uwsgi [-] ldap.user_tree_dn = None {{(pid=80143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 10:40:34.003981 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG uwsgi [-] policy.driver = sql {{(pid=80144) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 10:40:34.004097 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG uwsgi [-] memcache.dead_retry = 300 {{(pid=80143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 10:40:34.004214 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG uwsgi [-] policy.list_limit = None {{(pid=80144) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 10:40:34.004336 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG uwsgi [-] memcache.pool_connection_get_timeout = 10 {{(pid=80143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 10:40:34.004462 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG uwsgi [-] receipt.cache_on_issue = True {{(pid=80144) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 10:40:34.004579 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG uwsgi [-] memcache.pool_maxsize = 10 {{(pid=80143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 10:40:34.004695 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG uwsgi [-] receipt.cache_time = 300 {{(pid=80144) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 10:40:34.004810 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG uwsgi [-] memcache.pool_unused_timeout = 60 {{(pid=80143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 10:40:34.004926 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG uwsgi [-] receipt.caching = True {{(pid=80144) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 10:40:34.005043 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG uwsgi [-] memcache.socket_timeout = 3 {{(pid=80143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 10:40:34.005171 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG uwsgi [-] receipt.expiration = 300 {{(pid=80144) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 10:40:34.005288 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG uwsgi [-] oauth1.access_token_duration = 86400 {{(pid=80143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 10:40:34.005404 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG uwsgi [-] receipt.provider = fernet {{(pid=80144) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 10:40:34.005520 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG uwsgi [-] oauth1.driver = sql {{(pid=80143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 10:40:34.005636 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG uwsgi [-] resource.admin_project_domain_name = None {{(pid=80144) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 10:40:34.005751 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG uwsgi [-] oauth1.request_token_duration = 28800 {{(pid=80143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 10:40:34.005876 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG uwsgi [-] resource.admin_project_name = None {{(pid=80144) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 10:40:34.005993 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG uwsgi [-] policy.driver = sql {{(pid=80143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 10:40:34.006110 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG uwsgi [-] resource.cache_time = None {{(pid=80144) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 10:40:34.006226 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG uwsgi [-] policy.list_limit = None {{(pid=80143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 10:40:34.006342 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG uwsgi [-] resource.caching = True {{(pid=80144) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 10:40:34.006458 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG uwsgi [-] receipt.cache_on_issue = True {{(pid=80143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 10:40:34.006582 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG uwsgi [-] resource.domain_name_url_safe = off {{(pid=80144) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 10:40:34.006699 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG uwsgi [-] receipt.cache_time = 300 {{(pid=80143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 10:40:34.006816 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG uwsgi [-] resource.driver = sql {{(pid=80144) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 10:40:34.006950 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG uwsgi [-] receipt.caching = True {{(pid=80143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 10:40:34.007070 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG uwsgi [-] resource.list_limit = None {{(pid=80144) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 10:40:34.007187 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG uwsgi [-] receipt.expiration = 300 {{(pid=80143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 10:40:34.007315 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG uwsgi [-] resource.project_name_url_safe = off {{(pid=80144) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 10:40:34.007434 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG uwsgi [-] receipt.provider = fernet {{(pid=80143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 10:40:34.007550 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG uwsgi [-] revoke.cache_time = 3600 {{(pid=80144) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 10:40:34.007665 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG uwsgi [-] resource.admin_project_domain_name = None {{(pid=80143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 10:40:34.007780 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG uwsgi [-] revoke.caching = True {{(pid=80144) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 10:40:34.007896 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG uwsgi [-] resource.admin_project_name = None {{(pid=80143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 10:40:34.008019 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG uwsgi [-] revoke.driver = sql {{(pid=80144) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 10:40:34.008137 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG uwsgi [-] resource.cache_time = None {{(pid=80143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 10:40:34.008253 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG uwsgi [-] revoke.expiration_buffer = 1800 {{(pid=80144) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 10:40:34.008369 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG uwsgi [-] resource.caching = True {{(pid=80143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 10:40:34.008485 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG uwsgi [-] role.cache_time = None {{(pid=80144) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 10:40:34.008600 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG uwsgi [-] resource.domain_name_url_safe = off {{(pid=80143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 10:40:34.008724 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG uwsgi [-] role.caching = True {{(pid=80144) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 10:40:34.008841 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG uwsgi [-] resource.driver = sql {{(pid=80143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 10:40:34.008958 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG uwsgi [-] role.driver = sql {{(pid=80144) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 10:40:34.008958 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG uwsgi [-] role.list_limit = None {{(pid=80144) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 10:40:34.009177 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG uwsgi [-] resource.list_limit = None {{(pid=80143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 10:40:34.009369 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG uwsgi [-] saml.assertion_expiration_time = 3600 {{(pid=80144) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 10:40:34.009489 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG uwsgi [-] resource.project_name_url_safe = off {{(pid=80143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 10:40:34.009606 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG uwsgi [-] saml.certfile = /etc/keystone/ssl/certs/signing_cert.pem {{(pid=80144) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 10:40:34.009723 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG uwsgi [-] revoke.cache_time = 3600 {{(pid=80143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 10:40:34.009723 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG uwsgi [-] revoke.caching = True {{(pid=80143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 10:40:34.009947 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG uwsgi [-] saml.idp_contact_company = Example, Inc. {{(pid=80144) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 10:40:34.009947 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG uwsgi [-] saml.idp_contact_email = support@example.com {{(pid=80144) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 10:40:34.010178 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG uwsgi [-] revoke.driver = sql {{(pid=80143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 10:40:34.010178 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG uwsgi [-] revoke.expiration_buffer = 1800 {{(pid=80143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 10:40:34.010397 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG uwsgi [-] saml.idp_contact_name = SAML Identity Provider Support {{(pid=80144) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 10:40:34.010513 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG uwsgi [-] role.cache_time = None {{(pid=80143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 10:40:34.010629 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG uwsgi [-] saml.idp_contact_surname = Support {{(pid=80144) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 10:40:34.010754 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG uwsgi [-] role.caching = True {{(pid=80143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 10:40:34.010871 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG uwsgi [-] saml.idp_contact_telephone = +1 800 555 0100 {{(pid=80144) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 10:40:34.011015 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG uwsgi [-] role.driver = sql {{(pid=80143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 10:40:34.011141 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG uwsgi [-] saml.idp_contact_type = other {{(pid=80144) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 10:40:34.011260 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG uwsgi [-] role.list_limit = None {{(pid=80143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 10:40:34.011382 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG uwsgi [-] saml.idp_entity_id = None {{(pid=80144) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 10:40:34.011516 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG uwsgi [-] saml.assertion_expiration_time = 3600 {{(pid=80143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 10:40:34.011637 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG uwsgi [-] saml.idp_lang = en {{(pid=80144) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 10:40:34.011637 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG uwsgi [-] saml.idp_metadata_path = /etc/keystone/saml2_idp_metadata.xml {{(pid=80144) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 10:40:34.011861 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG uwsgi [-] saml.certfile = /etc/keystone/ssl/certs/signing_cert.pem {{(pid=80143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 10:40:34.011979 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG uwsgi [-] saml.idp_organization_display_name = OpenStack SAML Identity Provider {{(pid=80144) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 10:40:34.012096 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG uwsgi [-] saml.idp_contact_company = Example, Inc. {{(pid=80143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 10:40:34.012443 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG uwsgi [-] saml.idp_organization_name = SAML Identity Provider {{(pid=80144) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 10:40:34.012606 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG uwsgi [-] saml.idp_contact_email = support@example.com {{(pid=80143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 10:40:34.012606 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG uwsgi [-] saml.idp_contact_name = SAML Identity Provider Support {{(pid=80143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 10:40:34.012861 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG uwsgi [-] saml.idp_organization_url = https://example.com/ {{(pid=80144) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 10:40:34.013033 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG uwsgi [-] saml.idp_contact_surname = Support {{(pid=80143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 10:40:34.013159 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG uwsgi [-] saml.idp_sso_endpoint = None {{(pid=80144) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 10:40:34.013286 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG uwsgi [-] saml.idp_contact_telephone = +1 800 555 0100 {{(pid=80143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 10:40:34.013407 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG uwsgi [-] saml.keyfile = /etc/keystone/ssl/private/signing_key.pem {{(pid=80144) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 10:40:34.013530 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG uwsgi [-] saml.idp_contact_type = other {{(pid=80143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 10:40:34.013647 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG uwsgi [-] saml.relay_state_prefix = ss:mem: {{(pid=80144) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 10:40:34.013763 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG uwsgi [-] saml.idp_entity_id = None {{(pid=80143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 10:40:34.013763 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG uwsgi [-] saml.idp_lang = en {{(pid=80143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 10:40:34.013990 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG uwsgi [-] saml.xmlsec1_binary = xmlsec1 {{(pid=80144) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 10:40:34.014108 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG uwsgi [-] saml.idp_metadata_path = /etc/keystone/saml2_idp_metadata.xml {{(pid=80143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 10:40:34.014225 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG uwsgi [-] security_compliance.change_password_upon_first_use = False {{(pid=80144) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 10:40:34.014342 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG uwsgi [-] saml.idp_organization_display_name = OpenStack SAML Identity Provider {{(pid=80143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 10:40:34.014459 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG uwsgi [-] security_compliance.disable_user_account_days_inactive = None {{(pid=80144) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 10:40:34.014576 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG uwsgi [-] saml.idp_organization_name = SAML Identity Provider {{(pid=80143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 10:40:34.014701 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG uwsgi [-] security_compliance.lockout_duration = 10 {{(pid=80144) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 10:40:34.014701 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG uwsgi [-] security_compliance.lockout_failure_attempts = 2 {{(pid=80144) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 10:40:34.014940 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG uwsgi [-] saml.idp_organization_url = https://example.com/ {{(pid=80143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 10:40:34.015059 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG uwsgi [-] security_compliance.minimum_password_age = 0 {{(pid=80144) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 10:40:34.015176 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG uwsgi [-] saml.idp_sso_endpoint = None {{(pid=80143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 10:40:34.015306 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG uwsgi [-] security_compliance.password_expires_days = None {{(pid=80144) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 10:40:34.015425 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG uwsgi [-] saml.keyfile = /etc/keystone/ssl/private/signing_key.pem {{(pid=80143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 10:40:34.015544 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG uwsgi [-] security_compliance.password_regex = None {{(pid=80144) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 10:40:34.015544 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG uwsgi [-] security_compliance.password_regex_description = None {{(pid=80144) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 10:40:34.015761 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG uwsgi [-] saml.relay_state_prefix = ss:mem: {{(pid=80143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 10:40:34.015877 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG uwsgi [-] security_compliance.unique_last_password_count = 2 {{(pid=80144) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 10:40:34.016002 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG uwsgi [-] saml.xmlsec1_binary = xmlsec1 {{(pid=80143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 10:40:34.016119 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG uwsgi [-] shadow_users.driver = sql {{(pid=80144) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 10:40:34.016236 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG uwsgi [-] security_compliance.change_password_upon_first_use = False {{(pid=80143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 10:40:34.016352 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG uwsgi [-] token.allow_expired_window = 172800 {{(pid=80144) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 10:40:34.016468 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG uwsgi [-] security_compliance.disable_user_account_days_inactive = None {{(pid=80143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 10:40:34.016585 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG uwsgi [-] token.allow_rescope_scoped_token = True {{(pid=80144) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 10:40:34.016585 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG uwsgi [-] token.cache_on_issue = True {{(pid=80144) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 10:40:34.016813 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG uwsgi [-] security_compliance.lockout_duration = 10 {{(pid=80143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 10:40:34.016929 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG uwsgi [-] token.cache_time = None {{(pid=80144) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 10:40:34.017044 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG uwsgi [-] security_compliance.lockout_failure_attempts = 2 {{(pid=80143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 10:40:34.017173 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG uwsgi [-] token.caching = True {{(pid=80144) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 10:40:34.017291 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG uwsgi [-] security_compliance.minimum_password_age = 0 {{(pid=80143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 10:40:34.017417 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG uwsgi [-] token.expiration = 3600 {{(pid=80144) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 10:40:34.017534 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG uwsgi [-] security_compliance.password_expires_days = None {{(pid=80143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 10:40:34.017651 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG uwsgi [-] token.provider = fernet {{(pid=80144) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 10:40:34.017767 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG uwsgi [-] security_compliance.password_regex = None {{(pid=80143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 10:40:34.017884 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG uwsgi [-] token.revoke_by_id = True {{(pid=80144) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 10:40:34.018000 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG uwsgi [-] security_compliance.password_regex_description = None {{(pid=80143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 10:40:34.018125 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG uwsgi [-] tokenless_auth.issuer_attribute = SSL_CLIENT_I_DN {{(pid=80144) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 10:40:34.018242 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG uwsgi [-] security_compliance.unique_last_password_count = 2 {{(pid=80143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 10:40:34.018358 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG uwsgi [-] tokenless_auth.protocol = x509 {{(pid=80144) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 10:40:34.018474 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG uwsgi [-] shadow_users.driver = sql {{(pid=80143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 10:40:34.018590 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG uwsgi [-] tokenless_auth.trusted_issuer = [] {{(pid=80144) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 10:40:34.018707 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG uwsgi [-] token.allow_expired_window = 172800 {{(pid=80143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 10:40:34.018831 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG uwsgi [-] totp.included_previous_windows = 1 {{(pid=80144) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 10:40:34.018980 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG uwsgi [-] token.allow_rescope_scoped_token = True {{(pid=80143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 10:40:34.019107 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG uwsgi [-] trust.allow_redelegation = False {{(pid=80144) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 10:40:34.019225 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG uwsgi [-] token.cache_on_issue = True {{(pid=80143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 10:40:34.019345 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG uwsgi [-] trust.driver = sql {{(pid=80144) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 10:40:34.019462 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG uwsgi [-] token.cache_time = None {{(pid=80143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 10:40:34.019590 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG uwsgi [-] trust.max_redelegation_count = 3 {{(pid=80144) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 10:40:34.019711 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG uwsgi [-] token.caching = True {{(pid=80143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 10:40:34.019827 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG uwsgi [-] unified_limit.cache_time = None {{(pid=80144) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 10:40:34.019944 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG uwsgi [-] token.expiration = 3600 {{(pid=80143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 10:40:34.020060 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG uwsgi [-] unified_limit.caching = True {{(pid=80144) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 10:40:34.020176 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG uwsgi [-] token.provider = fernet {{(pid=80143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 10:40:34.020300 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG uwsgi [-] unified_limit.driver = sql {{(pid=80144) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 10:40:34.020418 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG uwsgi [-] token.revoke_by_id = True {{(pid=80143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 10:40:34.020534 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG uwsgi [-] unified_limit.enforcement_model = flat {{(pid=80144) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 10:40:34.020651 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG uwsgi [-] tokenless_auth.issuer_attribute = SSL_CLIENT_I_DN {{(pid=80143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 10:40:34.020768 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG uwsgi [-] unified_limit.list_limit = None {{(pid=80144) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 10:40:34.020883 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG uwsgi [-] tokenless_auth.protocol = x509 {{(pid=80143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 10:40:34.021008 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG uwsgi [-] wsgi.debug_middleware = False {{(pid=80144) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 10:40:34.021126 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG uwsgi [-] tokenless_auth.trusted_issuer = [] {{(pid=80143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 10:40:34.021233 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG uwsgi [-] database.backend = sqlalchemy {{(pid=80144) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 10:40:34.021347 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG uwsgi [-] totp.included_previous_windows = 1 {{(pid=80143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 10:40:34.021463 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG uwsgi [-] database.connection = **** {{(pid=80144) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 10:40:34.021588 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG uwsgi [-] trust.allow_redelegation = False {{(pid=80143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 10:40:34.021714 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG uwsgi [-] database.connection_debug = 0 {{(pid=80144) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 10:40:34.021842 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG uwsgi [-] trust.driver = sql {{(pid=80143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 10:40:34.021960 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG uwsgi [-] database.connection_parameters = {{(pid=80144) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 10:40:34.022076 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG uwsgi [-] trust.max_redelegation_count = 3 {{(pid=80143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 10:40:34.022197 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG uwsgi [-] database.connection_recycle_time = 3600 {{(pid=80144) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 10:40:34.022314 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG uwsgi [-] unified_limit.cache_time = None {{(pid=80143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 10:40:34.022430 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG uwsgi [-] database.connection_trace = False {{(pid=80144) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 10:40:34.022554 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG uwsgi [-] unified_limit.caching = True {{(pid=80143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 10:40:34.022671 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG uwsgi [-] database.db_inc_retry_interval = True {{(pid=80144) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 10:40:34.022787 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG uwsgi [-] unified_limit.driver = sql {{(pid=80143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 10:40:34.022903 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG uwsgi [-] database.db_max_retries = 20 {{(pid=80144) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 10:40:34.023094 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG uwsgi [-] unified_limit.enforcement_model = flat {{(pid=80143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 10:40:34.023210 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG uwsgi [-] database.db_max_retry_interval = 10 {{(pid=80144) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 10:40:34.023340 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG uwsgi [-] unified_limit.list_limit = None {{(pid=80143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 10:40:34.023463 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG uwsgi [-] database.db_retry_interval = 1 {{(pid=80144) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 10:40:34.023581 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG uwsgi [-] wsgi.debug_middleware = False {{(pid=80143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 10:40:34.023697 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG uwsgi [-] database.max_overflow = 50 {{(pid=80144) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 10:40:34.023697 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG uwsgi [-] database.max_pool_size = 5 {{(pid=80144) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 10:40:34.023917 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG uwsgi [-] database.backend = sqlalchemy {{(pid=80143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 10:40:34.024041 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG uwsgi [-] database.max_retries = 10 {{(pid=80144) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 10:40:34.024159 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG uwsgi [-] database.connection = **** {{(pid=80143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 10:40:34.024284 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG uwsgi [-] database.mysql_enable_ndb = False {{(pid=80144) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 10:40:34.024410 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG uwsgi [-] database.connection_debug = 0 {{(pid=80143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 10:40:34.024527 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG uwsgi [-] database.mysql_sql_mode = TRADITIONAL {{(pid=80144) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 10:40:34.024644 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG uwsgi [-] database.connection_parameters = {{(pid=80143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 10:40:34.024760 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG uwsgi [-] database.pool_timeout = None {{(pid=80144) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 10:40:34.024876 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG uwsgi [-] database.connection_recycle_time = 3600 {{(pid=80143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 10:40:34.025001 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG uwsgi [-] database.retry_interval = 10 {{(pid=80144) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 10:40:34.025119 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG uwsgi [-] database.connection_trace = False {{(pid=80143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 10:40:34.025235 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG uwsgi [-] database.slave_connection = **** {{(pid=80144) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 10:40:34.025351 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG uwsgi [-] database.db_inc_retry_interval = True {{(pid=80143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 10:40:34.025467 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG uwsgi [-] database.sqlite_synchronous = True {{(pid=80144) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 10:40:34.025583 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG uwsgi [-] database.db_max_retries = 20 {{(pid=80143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 10:40:34.025708 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG uwsgi [-] database.use_db_reconnect = False {{(pid=80144) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 10:40:34.025825 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG uwsgi [-] database.db_max_retry_interval = 10 {{(pid=80143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 10:40:34.025942 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=80144) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2613}} Aug 07 10:40:34.026316 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG uwsgi [-] database.db_retry_interval = 1 {{(pid=80143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 10:40:34.026316 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG uwsgi [-] database.max_overflow = 50 {{(pid=80143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 10:40:34.026316 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG uwsgi [-] database.max_pool_size = 5 {{(pid=80143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 10:40:34.026316 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG uwsgi [-] database.max_retries = 10 {{(pid=80143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 10:40:34.026316 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG uwsgi [-] database.mysql_enable_ndb = False {{(pid=80143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 10:40:34.026316 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG uwsgi [-] database.mysql_sql_mode = TRADITIONAL {{(pid=80143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 10:40:34.026316 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG uwsgi [-] database.pool_timeout = None {{(pid=80143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 10:40:34.026316 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG uwsgi [-] database.retry_interval = 10 {{(pid=80143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 10:40:34.026316 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG uwsgi [-] database.slave_connection = **** {{(pid=80143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 10:40:34.027305 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG uwsgi [-] database.sqlite_synchronous = True {{(pid=80143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 10:40:34.027305 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG uwsgi [-] database.use_db_reconnect = False {{(pid=80143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 07 10:40:34.027305 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=80143) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2613}} Aug 07 10:40:34.052533 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.deleted`. {{(pid=80144) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 07 10:40:34.052761 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.disabled`. {{(pid=80144) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 07 10:40:34.052979 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_assignment_removal` subscribed to event `identity.remove_application_credentials_for_user.internal`. {{(pid=80144) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 07 10:40:34.054519 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.deleted`. {{(pid=80143) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 07 10:40:34.054755 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.disabled`. {{(pid=80143) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 07 10:40:34.055013 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_assignment_removal` subscribed to event `identity.remove_application_credentials_for_user.internal`. {{(pid=80143) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 07 10:40:34.063486 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.notifications [-] Callback: `keystone.assignment.core.Manager._delete_domain_assignments` subscribed to event `identity.domain.deleted`. {{(pid=80144) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 07 10:40:34.065860 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [-] Callback: `keystone.assignment.core.Manager._delete_domain_assignments` subscribed to event `identity.domain.deleted`. {{(pid=80143) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 07 10:40:34.091609 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.project.deleted`. {{(pid=80144) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 07 10:40:34.091838 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.endpoint.deleted`. {{(pid=80144) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 07 10:40:34.092338 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.project.deleted`. {{(pid=80143) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 07 10:40:34.092572 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.endpoint.deleted`. {{(pid=80143) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 07 10:40:34.144553 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.notifications [-] Callback: `keystone.federation.core.Manager._cleanup_identity_provider` subscribed to event `identity.domain_deleted.internal`. {{(pid=80144) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 07 10:40:34.145622 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [-] Callback: `keystone.federation.core.Manager._cleanup_identity_provider` subscribed to event `identity.domain_deleted.internal`. {{(pid=80143) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 07 10:40:34.194597 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._domain_deleted` subscribed to event `identity.domain_deleted.internal`. {{(pid=80144) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 07 10:40:34.194827 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._unset_default_project` subscribed to event `identity.project.deleted`. {{(pid=80144) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 07 10:40:34.196937 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._domain_deleted` subscribed to event `identity.domain_deleted.internal`. {{(pid=80143) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 07 10:40:34.197170 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._unset_default_project` subscribed to event `identity.project.deleted`. {{(pid=80143) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 07 10:40:34.259652 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._trust_callback` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80144) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 07 10:40:34.259877 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._consumer_callback` subscribed to event `identity.OS-OAUTH1:consumer.deleted`. {{(pid=80144) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 07 10:40:34.260016 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._trust_callback` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80143) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 07 10:40:34.260103 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.deleted`. {{(pid=80144) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 07 10:40:34.260247 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._consumer_callback` subscribed to event `identity.OS-OAUTH1:consumer.deleted`. {{(pid=80143) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 07 10:40:34.260349 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._project_callback` subscribed to event `identity.project.deleted`. {{(pid=80144) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 07 10:40:34.260471 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.deleted`. {{(pid=80143) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 07 10:40:34.260621 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.disabled`. {{(pid=80144) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 07 10:40:34.260744 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._project_callback` subscribed to event `identity.project.deleted`. {{(pid=80143) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 07 10:40:34.260825 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.persist_revocation_event_for_user.internal`. {{(pid=80144) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 07 10:40:34.260911 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.disabled`. {{(pid=80143) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 07 10:40:34.261102 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.persist_revocation_event_for_user.internal`. {{(pid=80143) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 07 10:40:34.283982 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80144) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 07 10:40:34.284223 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.deleted`. {{(pid=80144) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 07 10:40:34.284437 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.deleted`. {{(pid=80144) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 07 10:40:34.284660 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.disabled`. {{(pid=80144) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 07 10:40:34.284784 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80143) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 07 10:40:34.284871 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.disabled`. {{(pid=80144) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 07 10:40:34.285018 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.deleted`. {{(pid=80143) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 07 10:40:34.285103 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.project.disabled`. {{(pid=80144) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 07 10:40:34.285244 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.deleted`. {{(pid=80143) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 07 10:40:34.285338 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=80144) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 07 10:40:34.285469 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.disabled`. {{(pid=80143) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 07 10:40:34.285689 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.disabled`. {{(pid=80143) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 07 10:40:34.285907 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.project.disabled`. {{(pid=80143) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 07 10:40:34.286121 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=80143) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 07 10:40:34.296406 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.notifications [-] Callback: `keystone.trust.core.Manager._on_user_delete` subscribed to event `identity.user.deleted`. {{(pid=80144) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 07 10:40:34.297458 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [-] Callback: `keystone.trust.core.Manager._on_user_delete` subscribed to event `identity.user.deleted`. {{(pid=80143) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 07 10:40:34.308928 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80144) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 07 10:40:34.309694 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.deleted`. {{(pid=80144) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 07 10:40:34.309803 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80143) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 07 10:40:34.309919 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.deleted`. {{(pid=80144) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 07 10:40:34.310150 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.disabled`. {{(pid=80144) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 07 10:40:34.310378 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.deleted`. {{(pid=80143) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 07 10:40:34.310464 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.disabled`. {{(pid=80144) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 07 10:40:34.310539 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.deleted`. {{(pid=80143) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 07 10:40:34.310614 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.project.disabled`. {{(pid=80144) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 07 10:40:34.310698 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.disabled`. {{(pid=80143) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 07 10:40:34.310776 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=80144) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 07 10:40:34.310879 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.disabled`. {{(pid=80143) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 07 10:40:34.311120 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.project.disabled`. {{(pid=80143) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 07 10:40:34.311335 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=80143) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 07 10:40:34.327679 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/projects' {}] {{(pid=80144) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 07 10:40:34.327884 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/projects' {}] {{(pid=80144) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Aug 07 10:40:34.328195 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/projects' {}] {{(pid=80143) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 07 10:40:34.328279 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/domains' {}] {{(pid=80144) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 07 10:40:34.328405 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/domains' {}] {{(pid=80144) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Aug 07 10:40:34.328487 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/projects' {}] {{(pid=80143) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Aug 07 10:40:34.328676 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/system' {}] {{(pid=80144) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 07 10:40:34.328761 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/domains' {}] {{(pid=80143) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 07 10:40:34.328927 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/domains' {}] {{(pid=80143) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Aug 07 10:40:34.329277 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/catalog' {}] {{(pid=80144) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 07 10:40:34.329277 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens/OS-PKI/revoked' {}] {{(pid=80144) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 07 10:40:34.329431 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/system' {}] {{(pid=80143) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 07 10:40:34.329508 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens' {}] {{(pid=80144) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 07 10:40:34.329582 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/catalog' {}] {{(pid=80143) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 07 10:40:34.329686 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens/OS-PKI/revoked' {}] {{(pid=80143) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 07 10:40:34.329942 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens' {}] {{(pid=80143) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 07 10:40:34.341117 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2' {}] {{(pid=80144) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 07 10:40:34.341392 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2/ecp' {}] {{(pid=80144) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 07 10:40:34.341661 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/websso/' {}] {{(pid=80144) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 07 10:40:34.341941 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/identity_providers//protocols//websso' {}] {{(pid=80144) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 07 10:40:34.342052 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2' {}] {{(pid=80143) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 07 10:40:34.342339 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2/ecp' {}] {{(pid=80143) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 07 10:40:34.342612 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/websso/' {}] {{(pid=80143) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 07 10:40:34.342900 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/identity_providers//protocols//websso' {}] {{(pid=80143) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 07 10:40:34.348777 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.common [-] Adding standard routes to API credentials for `type` (API Prefix: ) [/credentials, /credentials/] {{(pid=80144) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 07 10:40:34.349761 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.common [-] Adding standard routes to API credentials for `type` (API Prefix: ) [/credentials, /credentials/] {{(pid=80143) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 07 10:40:34.351819 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.common [-] Adding standard routes to API domains for `type` (API Prefix: ) [/domains, /domains/] {{(pid=80144) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 07 10:40:34.352105 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config' {}] {{(pid=80144) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 07 10:40:34.352384 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config/' {}] {{(pid=80144) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 07 10:40:34.352660 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config//' {}] {{(pid=80144) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 07 10:40:34.352821 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.common [-] Adding standard routes to API domains for `type` (API Prefix: ) [/domains, /domains/] {{(pid=80143) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 07 10:40:34.352936 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config/default' {}] {{(pid=80144) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 07 10:40:34.353108 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config' {}] {{(pid=80143) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 07 10:40:34.353200 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config//default' {}] {{(pid=80144) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 07 10:40:34.353392 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config/' {}] {{(pid=80143) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 07 10:40:34.353476 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config///default' {}] {{(pid=80144) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 07 10:40:34.353669 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config//' {}] {{(pid=80143) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 07 10:40:34.353751 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles' {}] {{(pid=80144) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 07 10:40:34.353946 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles/' {}] {{(pid=80144) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 07 10:40:34.354031 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config/default' {}] {{(pid=80143) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 07 10:40:34.354203 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config//default' {}] {{(pid=80143) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 07 10:40:34.354283 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles' {}] {{(pid=80144) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 07 10:40:34.354471 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config///default' {}] {{(pid=80143) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 07 10:40:34.354562 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles/' {}] {{(pid=80144) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 07 10:40:34.354744 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles' {}] {{(pid=80143) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 07 10:40:34.355049 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles/' {}] {{(pid=80143) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 07 10:40:34.355328 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles' {}] {{(pid=80143) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 07 10:40:34.355595 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles/' {}] {{(pid=80143) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 07 10:40:34.378222 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.common [-] Adding resource routes to API ec2tokens: ['/ec2tokens' {}] {{(pid=80144) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 07 10:40:34.379716 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.common [-] Adding standard routes to API endpoints for `type` (API Prefix: ) [/endpoints, /endpoints/] {{(pid=80144) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 07 10:40:34.379891 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.common [-] Adding resource routes to API ec2tokens: ['/ec2tokens' {}] {{(pid=80143) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 07 10:40:34.379999 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.common [-] Adding resource routes to API endpoints: ['/endpoints//OS-ENDPOINT-POLICY/policy' {}] {{(pid=80144) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 07 10:40:34.381370 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.common [-] Adding standard routes to API endpoints for `type` (API Prefix: ) [/endpoints, /endpoints/] {{(pid=80143) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 07 10:40:34.381655 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.common [-] Adding resource routes to API endpoints: ['/endpoints//OS-ENDPOINT-POLICY/policy' {}] {{(pid=80143) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 07 10:40:34.384527 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.common [-] Adding standard routes to API groups for `type` (API Prefix: ) [/groups, /groups/] {{(pid=80144) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 07 10:40:34.384814 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users' {}] {{(pid=80144) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 07 10:40:34.385098 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users/' {}] {{(pid=80144) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 07 10:40:34.386233 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.common [-] Adding standard routes to API groups for `type` (API Prefix: ) [/groups, /groups/] {{(pid=80143) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 07 10:40:34.386520 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users' {}] {{(pid=80143) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 07 10:40:34.386805 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users/' {}] {{(pid=80143) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 07 10:40:34.391763 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.common [-] Adding standard routes to API limits for `type` (API Prefix: ) [/limits, /limits/] {{(pid=80144) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 07 10:40:34.392047 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.common [-] Adding resource routes to API limits: ['/limits/model' {}] {{(pid=80144) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 07 10:40:34.393678 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.common [-] Adding standard routes to API limits for `type` (API Prefix: ) [/limits, /limits/] {{(pid=80143) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 07 10:40:34.393964 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.common [-] Adding resource routes to API limits: ['/limits/model' {}] {{(pid=80143) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 07 10:40:34.396119 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-EP-FILTER for `type` (API Prefix: /OS-EP-FILTER) [/endpoint_groups, /endpoint_groups/] {{(pid=80144) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 07 10:40:34.396366 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoints//projects' {}] {{(pid=80144) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 07 10:40:34.396665 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints/' {}] {{(pid=80144) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 07 10:40:34.396954 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints' {}] {{(pid=80144) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 07 10:40:34.397227 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoint_groups' {}] {{(pid=80144) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 07 10:40:34.397513 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//endpoints' {}] {{(pid=80144) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 07 10:40:34.397787 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects' {}] {{(pid=80144) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 07 10:40:34.398044 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-EP-FILTER for `type` (API Prefix: /OS-EP-FILTER) [/endpoint_groups, /endpoint_groups/] {{(pid=80143) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 07 10:40:34.398126 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects/' {}] {{(pid=80144) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 07 10:40:34.398334 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoints//projects' {}] {{(pid=80143) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 07 10:40:34.398624 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints/' {}] {{(pid=80143) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 07 10:40:34.398906 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints' {}] {{(pid=80143) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 07 10:40:34.399215 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoint_groups' {}] {{(pid=80143) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 07 10:40:34.399509 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//endpoints' {}] {{(pid=80143) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 07 10:40:34.399791 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects' {}] {{(pid=80143) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 07 10:40:34.400062 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects/' {}] {{(pid=80143) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 07 10:40:34.414675 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/saml2/metadata' {}] {{(pid=80144) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 07 10:40:34.414675 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/identity_providers//protocols//auth' {}] {{(pid=80144) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 07 10:40:34.416675 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/saml2/metadata' {}] {{(pid=80143) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 07 10:40:34.419270 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/identity_providers//protocols//auth' {}] {{(pid=80143) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 07 10:40:34.419877 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.common [-] Adding standard routes to API identity_providers for `type` (API Prefix: /OS-FEDERATION) [/identity_providers, /identity_providers/] {{(pid=80144) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 07 10:40:34.421859 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.common [-] Adding standard routes to API identity_providers for `type` (API Prefix: /OS-FEDERATION) [/identity_providers, /identity_providers/] {{(pid=80143) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 07 10:40:34.422294 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols/' {}] {{(pid=80144) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 07 10:40:34.422588 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols' {}] {{(pid=80144) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 07 10:40:34.425012 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols/' {}] {{(pid=80143) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 07 10:40:34.425312 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols' {}] {{(pid=80143) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 07 10:40:34.426914 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.common [-] Adding standard routes to API mappings for `type` (API Prefix: /OS-FEDERATION) [/mappings, /mappings/] {{(pid=80144) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 07 10:40:34.429709 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.common [-] Adding standard routes to API mappings for `type` (API Prefix: /OS-FEDERATION) [/mappings, /mappings/] {{(pid=80143) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 07 10:40:34.430241 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.common [-] Adding standard routes to API service_providers for `type` (API Prefix: /OS-FEDERATION) [/service_providers, /service_providers/] {{(pid=80144) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 07 10:40:34.432890 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.common [-] Adding standard routes to API service_providers for `type` (API Prefix: /OS-FEDERATION) [/service_providers, /service_providers/] {{(pid=80143) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 07 10:40:34.433184 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles//inherited_to_projects' {}] {{(pid=80144) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 07 10:40:34.433481 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles/inherited_to_projects' {}] {{(pid=80144) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 07 10:40:34.433765 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles//inherited_to_projects' {}] {{(pid=80144) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 07 10:40:34.434089 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles/inherited_to_projects' {}] {{(pid=80144) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 07 10:40:34.434372 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//users//roles//inherited_to_projects' {}] {{(pid=80144) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 07 10:40:34.434646 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//groups//roles//inherited_to_projects' {}] {{(pid=80144) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 07 10:40:34.436471 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles//inherited_to_projects' {}] {{(pid=80143) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 07 10:40:34.436774 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles/inherited_to_projects' {}] {{(pid=80143) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 07 10:40:34.437062 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles//inherited_to_projects' {}] {{(pid=80143) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 07 10:40:34.437360 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles/inherited_to_projects' {}] {{(pid=80143) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 07 10:40:34.437638 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//users//roles//inherited_to_projects' {}] {{(pid=80143) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 07 10:40:34.437926 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//groups//roles//inherited_to_projects' {}] {{(pid=80143) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 07 10:40:34.450793 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-OAUTH1 for `type` (API Prefix: /OS-OAUTH1) [/consumers, /consumers/] {{(pid=80144) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 07 10:40:34.451112 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/request_token' {}] {{(pid=80144) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 07 10:40:34.451386 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/access_token' {}] {{(pid=80144) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 07 10:40:34.451647 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/authorize/' {}] {{(pid=80144) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 07 10:40:34.454353 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-OAUTH1 for `type` (API Prefix: /OS-OAUTH1) [/consumers, /consumers/] {{(pid=80143) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 07 10:40:34.454644 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/request_token' {}] {{(pid=80143) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 07 10:40:34.454915 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/access_token' {}] {{(pid=80143) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 07 10:40:34.455201 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/authorize/' {}] {{(pid=80143) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 07 10:40:34.458423 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.common [-] Adding resource routes to API events: ['/events' {}] {{(pid=80144) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 07 10:40:34.460013 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/ca' {}] {{(pid=80144) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 07 10:40:34.460293 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/certificates' {}] {{(pid=80144) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 07 10:40:34.462063 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.common [-] Adding resource routes to API events: ['/events' {}] {{(pid=80143) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 07 10:40:34.463092 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.common [-] Adding standard routes to API policy for `type` (API Prefix: ) [/policies, /policies/] {{(pid=80144) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 07 10:40:34.463378 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints' {}] {{(pid=80144) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 07 10:40:34.463644 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/ca' {}] {{(pid=80143) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 07 10:40:34.463727 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints/' {}] {{(pid=80144) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 07 10:40:34.463924 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/certificates' {}] {{(pid=80143) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 07 10:40:34.464008 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services/' {}] {{(pid=80144) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 07 10:40:34.464215 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services//regions/' {}] {{(pid=80144) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 07 10:40:34.467576 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.common [-] Adding standard routes to API policy for `type` (API Prefix: ) [/policies, /policies/] {{(pid=80143) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 07 10:40:34.467865 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints' {}] {{(pid=80143) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 07 10:40:34.468164 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints/' {}] {{(pid=80143) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 07 10:40:34.468465 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services/' {}] {{(pid=80143) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 07 10:40:34.468745 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services//regions/' {}] {{(pid=80143) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 07 10:40:34.476063 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.common [-] Adding standard routes to API projects for `type` (API Prefix: ) [/projects, /projects/] {{(pid=80144) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 07 10:40:34.476348 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags' {}] {{(pid=80144) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 07 10:40:34.476629 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags/' {}] {{(pid=80144) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 07 10:40:34.476907 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles/' {}] {{(pid=80144) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 07 10:40:34.477186 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles' {}] {{(pid=80144) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 07 10:40:34.477462 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles/' {}] {{(pid=80144) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 07 10:40:34.477738 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles' {}] {{(pid=80144) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 07 10:40:34.480129 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.common [-] Adding standard routes to API projects for `type` (API Prefix: ) [/projects, /projects/] {{(pid=80143) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 07 10:40:34.480417 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags' {}] {{(pid=80143) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 07 10:40:34.480699 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags/' {}] {{(pid=80143) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 07 10:40:34.481066 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles/' {}] {{(pid=80143) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 07 10:40:34.481555 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles' {}] {{(pid=80143) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 07 10:40:34.481950 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles/' {}] {{(pid=80143) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 07 10:40:34.482247 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles' {}] {{(pid=80143) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 07 10:40:34.494536 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.common [-] Adding standard routes to API regions for `type` (API Prefix: ) [/regions, /regions/] {{(pid=80144) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 07 10:40:34.498000 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.common [-] Adding standard routes to API registered_limit for `type` (API Prefix: ) [/registered_limits, /registered_limits/] {{(pid=80144) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 07 10:40:34.499854 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.common [-] Adding standard routes to API regions for `type` (API Prefix: ) [/regions, /regions/] {{(pid=80143) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 07 10:40:34.500922 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_assignments: ['/role_assignments' {}] {{(pid=80144) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 07 10:40:34.502472 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_inferences: ['/role_inferences' {}] {{(pid=80144) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 07 10:40:34.502790 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.common [-] Adding standard routes to API registered_limit for `type` (API Prefix: ) [/registered_limits, /registered_limits/] {{(pid=80143) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 07 10:40:34.503978 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.common [-] Adding standard routes to API roles for `type` (API Prefix: ) [/roles, /roles/] {{(pid=80144) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 07 10:40:34.504261 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies' {}] {{(pid=80144) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 07 10:40:34.504552 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies/' {}] {{(pid=80144) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 07 10:40:34.505755 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_assignments: ['/role_assignments' {}] {{(pid=80143) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 07 10:40:34.507350 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_inferences: ['/role_inferences' {}] {{(pid=80143) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 07 10:40:34.508840 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.common [-] Adding standard routes to API roles for `type` (API Prefix: ) [/roles, /roles/] {{(pid=80143) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 07 10:40:34.509124 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies' {}] {{(pid=80143) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 07 10:40:34.509405 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies/' {}] {{(pid=80143) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 07 10:40:34.510885 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.common [-] Adding resource routes to API s3tokens: ['/s3tokens' {}] {{(pid=80144) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 07 10:40:34.512707 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.common [-] Adding standard routes to API services for `type` (API Prefix: ) [/services, /services/] {{(pid=80144) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 07 10:40:34.515624 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles' {}] {{(pid=80144) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 07 10:40:34.515921 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles/' {}] {{(pid=80144) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 07 10:40:34.516208 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles' {}] {{(pid=80144) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 07 10:40:34.516302 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.common [-] Adding resource routes to API s3tokens: ['/s3tokens' {}] {{(pid=80143) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 07 10:40:34.516481 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles/' {}] {{(pid=80144) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 07 10:40:34.517816 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.common [-] Adding standard routes to API services for `type` (API Prefix: ) [/services, /services/] {{(pid=80143) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 07 10:40:34.520750 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles' {}] {{(pid=80143) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 07 10:40:34.521044 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles/' {}] {{(pid=80143) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 07 10:40:34.521328 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles' {}] {{(pid=80143) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 07 10:40:34.521601 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles/' {}] {{(pid=80143) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 07 10:40:34.524443 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.common [-] Adding standard routes to API trusts for `type` (API Prefix: /OS-TRUST) [/trusts, /trusts/] {{(pid=80144) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 07 10:40:34.524730 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles' {}] {{(pid=80144) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 07 10:40:34.525006 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles/' {}] {{(pid=80144) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 07 10:40:34.530600 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.common [-] Adding standard routes to API trusts for `type` (API Prefix: /OS-TRUST) [/trusts, /trusts/] {{(pid=80143) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 07 10:40:34.530886 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles' {}] {{(pid=80143) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 07 10:40:34.531230 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles/' {}] {{(pid=80143) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 07 10:40:34.532665 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.common [-] Adding standard routes to API users for `type` (API Prefix: ) [/users, /users/] {{(pid=80144) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 07 10:40:34.533007 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//password' {}] {{(pid=80144) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 07 10:40:34.533291 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//groups' {}] {{(pid=80144) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 07 10:40:34.533571 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//projects' {}] {{(pid=80144) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 07 10:40:34.533847 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2' {}] {{(pid=80144) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 07 10:40:34.534132 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2/' {}] {{(pid=80144) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 07 10:40:34.534408 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens' {}] {{(pid=80144) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 07 10:40:34.534686 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens/' {}] {{(pid=80144) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 07 10:40:34.535091 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles' {}] {{(pid=80144) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 07 10:40:34.535370 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles/' {}] {{(pid=80144) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 07 10:40:34.535644 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials' {}] {{(pid=80144) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 07 10:40:34.535926 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials/' {}] {{(pid=80144) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 07 10:40:34.536202 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules' {}] {{(pid=80144) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 07 10:40:34.536465 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules/' {}] {{(pid=80144) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 07 10:40:34.542134 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.common [-] Adding standard routes to API users for `type` (API Prefix: ) [/users, /users/] {{(pid=80143) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 07 10:40:34.542421 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//password' {}] {{(pid=80143) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 07 10:40:34.542706 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//groups' {}] {{(pid=80143) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 07 10:40:34.543007 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//projects' {}] {{(pid=80143) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 07 10:40:34.543282 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2' {}] {{(pid=80143) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 07 10:40:34.544144 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2/' {}] {{(pid=80143) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 07 10:40:34.544629 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens' {}] {{(pid=80143) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 07 10:40:34.544992 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens/' {}] {{(pid=80143) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 07 10:40:34.545391 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles' {}] {{(pid=80143) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 07 10:40:34.545679 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles/' {}] {{(pid=80143) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 07 10:40:34.545955 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials' {}] {{(pid=80143) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 07 10:40:34.546220 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials/' {}] {{(pid=80143) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 07 10:40:34.546505 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules' {}] {{(pid=80143) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 07 10:40:34.546770 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules/' {}] {{(pid=80143) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 07 10:40:34.570430 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: WSGI app 0 (mountpoint='') ready in 6 seconds on interpreter 0x55a6111579c0 pid: 80144 (default app) Aug 07 10:40:34.573699 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58ebf2e8-8fae-4eb6-bf69-2d4fb83c5d31 None None] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:40:34.573985 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58ebf2e8-8fae-4eb6-bf69-2d4fb83c5d31 None None] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:40:34.574257 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58ebf2e8-8fae-4eb6-bf69-2d4fb83c5d31 None None] PATH_INFO: `/v3` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:40:34.578649 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 1/1] 10.222.0.25 () {50 vars in 791 bytes} [Sun Aug 7 10:40:34 2022] GET /identity/v3/ => generated 255 bytes in 8 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 10:40:34.580678 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: WSGI app 0 (mountpoint='') ready in 6 seconds on interpreter 0x55a6111579c0 pid: 80143 (default app) Aug 07 10:40:42.171284 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97613461-2c45-4c1a-a7bf-c1542617e242 None None] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:40:42.171858 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97613461-2c45-4c1a-a7bf-c1542617e242 None None] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:40:42.171858 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97613461-2c45-4c1a-a7bf-c1542617e242 None None] PATH_INFO: `/` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:40:42.173117 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 2/2] 10.222.0.25 () {54 vars in 927 bytes} [Sun Aug 7 10:40:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 07 10:40:42.180507 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b645da49-9449-4047-9d4e-5e0289af7804 None None] REQUEST_METHOD: `POST` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:40:42.180955 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b645da49-9449-4047-9d4e-5e0289af7804 None None] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:40:42.181048 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b645da49-9449-4047-9d4e-5e0289af7804 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:40:42.253434 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: INFO dbcounter [None req-b645da49-9449-4047-9d4e-5e0289af7804 None None] Registered counter for database keystone Aug 07 10:40:42.272838 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG oslo_db.sqlalchemy.engines [None req-b645da49-9449-4047-9d4e-5e0289af7804 None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=80143) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Aug 07 10:40:42.279364 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writer thread running {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Aug 07 10:40:42.460413 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG passlib.handlers.bcrypt [None req-b645da49-9449-4047-9d4e-5e0289af7804 None None] detected 'bcrypt' backend, version '3.2.2' {{(pid=80143) _load_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:625}} Aug 07 10:40:42.460765 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG passlib.handlers.bcrypt [None req-b645da49-9449-4047-9d4e-5e0289af7804 None None] 'bcrypt' backend lacks $2$ support, enabling workaround {{(pid=80143) _finalize_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:406}} Aug 07 10:40:42.515532 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.auth.core [None req-b645da49-9449-4047-9d4e-5e0289af7804 None None] MFA Rules not processed for user `bac4e6b76fe64217ac8fa44e71cffe6a`. Rule list: `[]` (Enabled: `True`). {{(pid=80143) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 10:40:42.517114 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-b645da49-9449-4047-9d4e-5e0289af7804 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 10:40:42.647951 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1/3] 10.222.0.25 () {58 vars in 1024 bytes} [Sun Aug 7 10:40:42 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 471 msecs (HTTP/1.1 201) 6 headers in 384 bytes (1 switches on core 0) Aug 07 10:40:42.770784 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97613461-2c45-4c1a-a7bf-c1542617e242 None None] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 10:40:42.839860 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: INFO dbcounter [None req-97613461-2c45-4c1a-a7bf-c1542617e242 None None] Registered counter for database keystone Aug 07 10:40:42.861112 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG oslo_db.sqlalchemy.engines [None req-97613461-2c45-4c1a-a7bf-c1542617e242 None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=80144) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Aug 07 10:40:42.867819 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writer thread running {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Aug 07 10:40:42.986724 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97613461-2c45-4c1a-a7bf-c1542617e242 None None] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 10:40:42.991047 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2ad9984-356f-49fe-a556-d398171a7bb6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5ac47e3b09f43d19a23a0f457fe9e23', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 10:40:42.991678 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2ad9984-356f-49fe-a556-d398171a7bb6 admin admin] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:40:42.991964 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2ad9984-356f-49fe-a556-d398171a7bb6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:40:42.992243 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2ad9984-356f-49fe-a556-d398171a7bb6 admin admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:40:43.000328 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2ad9984-356f-49fe-a556-d398171a7bb6 admin admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 10:40:43.052287 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: WARNING keystone.common.rbac_enforcer.enforcer [None req-d2ad9984-356f-49fe-a556-d398171a7bb6 admin admin] Deprecated policy rules found. Use oslopolicy-policy-generator and oslopolicy-policy-upgrade to detect and resolve deprecated policies in your configuration. Aug 07 10:40:43.052581 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2ad9984-356f-49fe-a556-d398171a7bb6 admin admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 10:40:43.062243 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 3/4] 10.222.0.25 () {58 vars in 1324 bytes} [Sun Aug 7 10:40:42 2022] GET /identity/v3/auth/tokens => generated 914 bytes in 292 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 07 10:40:43.069298 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b645da49-9449-4047-9d4e-5e0289af7804 None None] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 10:40:43.084083 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b645da49-9449-4047-9d4e-5e0289af7804 None None] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 10:40:43.088485 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5423eb5f-c0e2-4b1b-a047-e8b6e1618465 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5ac47e3b09f43d19a23a0f457fe9e23', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 10:40:43.089572 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5423eb5f-c0e2-4b1b-a047-e8b6e1618465 admin admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:40:43.089852 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5423eb5f-c0e2-4b1b-a047-e8b6e1618465 admin admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:40:43.090717 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5423eb5f-c0e2-4b1b-a047-e8b6e1618465 admin admin] PATH_INFO: `/v3/projects/d5ac47e3b09f43d19a23a0f457fe9e23` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:40:43.092191 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5423eb5f-c0e2-4b1b-a047-e8b6e1618465 admin admin] RBAC: Authorizing `identity:get_project(project_id=d5ac47e3b09f43d19a23a0f457fe9e23)` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 10:40:43.142566 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: WARNING keystone.common.rbac_enforcer.enforcer [None req-5423eb5f-c0e2-4b1b-a047-e8b6e1618465 admin admin] Deprecated policy rules found. Use oslopolicy-policy-generator and oslopolicy-policy-upgrade to detect and resolve deprecated policies in your configuration. Aug 07 10:40:43.142901 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5423eb5f-c0e2-4b1b-a047-e8b6e1618465 admin admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 10:40:43.147377 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 2/5] 10.222.0.25 () {56 vars in 1207 bytes} [Sun Aug 7 10:40:43 2022] GET /identity/v3/projects/d5ac47e3b09f43d19a23a0f457fe9e23 => generated 344 bytes in 79 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 10:40:45.127465 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a582b73-43d3-44d4-8416-ef1b00900f5e None None] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:40:45.128014 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a582b73-43d3-44d4-8416-ef1b00900f5e None None] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:40:45.128014 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a582b73-43d3-44d4-8416-ef1b00900f5e None None] PATH_INFO: `/` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:40:45.132077 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 4/6] 10.222.0.25 () {54 vars in 927 bytes} [Sun Aug 7 10:40:45 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 07 10:40:45.137828 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a38454e4-f6e0-4f67-b106-e7f6d18a7679 None None] REQUEST_METHOD: `POST` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:40:45.138247 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a38454e4-f6e0-4f67-b106-e7f6d18a7679 None None] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:40:45.138348 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a38454e4-f6e0-4f67-b106-e7f6d18a7679 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:40:45.159346 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.auth.core [None req-a38454e4-f6e0-4f67-b106-e7f6d18a7679 None None] MFA Rules not processed for user `bac4e6b76fe64217ac8fa44e71cffe6a`. Rule list: `[]` (Enabled: `True`). {{(pid=80143) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 10:40:45.160875 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-a38454e4-f6e0-4f67-b106-e7f6d18a7679 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 10:40:45.169238 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 3/7] 10.222.0.25 () {58 vars in 1024 bytes} [Sun Aug 7 10:40:45 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 384 bytes (1 switches on core 0) Aug 07 10:40:45.191113 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a582b73-43d3-44d4-8416-ef1b00900f5e None None] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 10:40:45.203025 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a582b73-43d3-44d4-8416-ef1b00900f5e None None] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 10:40:45.207151 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0ac618c-0318-475a-b2fc-dad4f783fce4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5ac47e3b09f43d19a23a0f457fe9e23', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 10:40:45.207746 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0ac618c-0318-475a-b2fc-dad4f783fce4 admin admin] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:40:45.208024 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0ac618c-0318-475a-b2fc-dad4f783fce4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:40:45.208296 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0ac618c-0318-475a-b2fc-dad4f783fce4 admin admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:40:45.215952 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0ac618c-0318-475a-b2fc-dad4f783fce4 admin admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 10:40:45.217691 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0ac618c-0318-475a-b2fc-dad4f783fce4 admin admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 10:40:45.226223 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 5/8] 10.222.0.25 () {58 vars in 1324 bytes} [Sun Aug 7 10:40:45 2022] GET /identity/v3/auth/tokens => generated 914 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 07 10:40:45.231988 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a38454e4-f6e0-4f67-b106-e7f6d18a7679 None None] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 10:40:45.241185 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a38454e4-f6e0-4f67-b106-e7f6d18a7679 None None] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 10:40:45.245542 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-018d5e5e-61e6-400e-aec1-cfd1276c7b7f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5ac47e3b09f43d19a23a0f457fe9e23', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 10:40:45.246156 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-018d5e5e-61e6-400e-aec1-cfd1276c7b7f admin admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:40:45.247207 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-018d5e5e-61e6-400e-aec1-cfd1276c7b7f admin admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:40:45.247512 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-018d5e5e-61e6-400e-aec1-cfd1276c7b7f admin admin] PATH_INFO: `/v3/users/bac4e6b76fe64217ac8fa44e71cffe6a` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:40:45.248836 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-018d5e5e-61e6-400e-aec1-cfd1276c7b7f admin admin] RBAC: Authorizing `identity:get_user(user_id=bac4e6b76fe64217ac8fa44e71cffe6a)` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 10:40:45.250667 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-018d5e5e-61e6-400e-aec1-cfd1276c7b7f admin admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 10:40:45.252207 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 4/9] 10.222.0.25 () {56 vars in 1198 bytes} [Sun Aug 7 10:40:45 2022] GET /identity/v3/users/bac4e6b76fe64217ac8fa44e71cffe6a => generated 247 bytes in 21 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 10:40:47.296407 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92588fb7-c46d-4cea-8038-60ca8031dda5 None None] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:40:47.296952 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92588fb7-c46d-4cea-8038-60ca8031dda5 None None] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:40:47.296952 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92588fb7-c46d-4cea-8038-60ca8031dda5 None None] PATH_INFO: `/` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:40:47.298273 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 6/10] 10.222.0.25 () {54 vars in 927 bytes} [Sun Aug 7 10:40:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 07 10:40:47.303518 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6be85b96-d6eb-4323-9dd9-2cfb36c83bb5 None None] REQUEST_METHOD: `POST` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:40:47.303929 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6be85b96-d6eb-4323-9dd9-2cfb36c83bb5 None None] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:40:47.304079 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6be85b96-d6eb-4323-9dd9-2cfb36c83bb5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:40:47.314172 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2121ad0-a940-4f4c-9ea6-f6f37dbf6934 None None] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:40:47.314460 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2121ad0-a940-4f4c-9ea6-f6f37dbf6934 None None] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:40:47.314745 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2121ad0-a940-4f4c-9ea6-f6f37dbf6934 None None] PATH_INFO: `/` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:40:47.316083 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 7/11] 10.222.0.25 () {54 vars in 927 bytes} [Sun Aug 7 10:40:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 07 10:40:47.321131 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cf55342-55dd-4536-8be6-dde02e5d1786 None None] REQUEST_METHOD: `POST` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:40:47.321424 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cf55342-55dd-4536-8be6-dde02e5d1786 None None] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:40:47.321701 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cf55342-55dd-4536-8be6-dde02e5d1786 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:40:47.322352 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.auth.core [None req-6be85b96-d6eb-4323-9dd9-2cfb36c83bb5 None None] MFA Rules not processed for user `bac4e6b76fe64217ac8fa44e71cffe6a`. Rule list: `[]` (Enabled: `True`). {{(pid=80143) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 10:40:47.323564 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-6be85b96-d6eb-4323-9dd9-2cfb36c83bb5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 10:40:47.365860 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG passlib.handlers.bcrypt [None req-9cf55342-55dd-4536-8be6-dde02e5d1786 None None] detected 'bcrypt' backend, version '3.2.2' {{(pid=80144) _load_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:625}} Aug 07 10:40:47.366269 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG passlib.handlers.bcrypt [None req-9cf55342-55dd-4536-8be6-dde02e5d1786 None None] 'bcrypt' backend lacks $2$ support, enabling workaround {{(pid=80144) _finalize_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:406}} Aug 07 10:40:47.405927 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 5/12] 10.222.0.25 () {58 vars in 1024 bytes} [Sun Aug 7 10:40:47 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 07 10:40:47.412368 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.auth.core [None req-9cf55342-55dd-4536-8be6-dde02e5d1786 None None] MFA Rules not processed for user `bac4e6b76fe64217ac8fa44e71cffe6a`. Rule list: `[]` (Enabled: `True`). {{(pid=80144) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 10:40:47.413992 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.fernet_utils [None req-9cf55342-55dd-4536-8be6-dde02e5d1786 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80144) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 10:40:47.429176 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6be85b96-d6eb-4323-9dd9-2cfb36c83bb5 None None] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 10:40:47.453367 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=100 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 10:40:47.513407 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 8/13] 10.222.0.25 () {58 vars in 1024 bytes} [Sun Aug 7 10:40:47 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 193 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 07 10:40:47.533746 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6be85b96-d6eb-4323-9dd9-2cfb36c83bb5 None None] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 10:40:47.534335 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cf55342-55dd-4536-8be6-dde02e5d1786 None None] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 10:40:47.568125 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb7a0723-25e5-49ba-bd85-7a4fe76aad52 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 10:40:47.568753 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb7a0723-25e5-49ba-bd85-7a4fe76aad52 None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:40:47.569037 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb7a0723-25e5-49ba-bd85-7a4fe76aad52 None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:40:47.569312 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb7a0723-25e5-49ba-bd85-7a4fe76aad52 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:40:47.580579 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb7a0723-25e5-49ba-bd85-7a4fe76aad52 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 07 10:40:47.589303 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application [None req-bb7a0723-25e5-49ba-bd85-7a4fe76aad52 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 07 10:40:47.589303 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 07 10:40:47.589303 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 07 10:40:47.589303 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 07 10:40:47.589303 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 07 10:40:47.589303 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 07 10:40:47.589303 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 07 10:40:47.589303 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 07 10:40:47.589303 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 07 10:40:47.589303 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 07 10:40:47.589303 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 07 10:40:47.589303 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 07 10:40:47.589303 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 07 10:40:47.589303 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 07 10:40:47.589303 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 07 10:40:47.589303 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 07 10:40:47.590488 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 07 10:40:47.590488 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 07 10:40:47.590488 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 07 10:40:47.590488 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 07 10:40:47.590488 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 07 10:40:47.590488 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 07 10:40:47.590488 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 07 10:40:47.590488 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 07 10:40:47.590488 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 07 10:40:47.590488 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.get_or_create( Aug 07 10:40:47.590488 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 07 10:40:47.590488 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application with Lock( Aug 07 10:40:47.590488 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 07 10:40:47.590488 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self._enter() Aug 07 10:40:47.590488 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 07 10:40:47.590488 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 07 10:40:47.591354 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 07 10:40:47.591354 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.creator() Aug 07 10:40:47.591354 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 07 10:40:47.591354 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application created_value = creator( Aug 07 10:40:47.591354 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 07 10:40:47.591354 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 07 10:40:47.591354 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 07 10:40:47.591354 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 07 10:40:47.591354 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 07 10:40:47.591354 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 07 10:40:47.591354 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 07 10:40:47.591354 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application Aug 07 10:40:47.591354 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 6/14] 10.222.0.25 () {56 vars in 1096 bytes} [Sun Aug 7 10:40:47 2022] GET /identity/v3/roles/admin => generated 83 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 07 10:40:47.596183 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb7a0723-25e5-49ba-bd85-7a4fe76aad52 None admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 10:40:47.633458 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cf55342-55dd-4536-8be6-dde02e5d1786 None None] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 10:40:47.635769 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=100 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 10:40:47.672350 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69498a47-7209-4c38-b828-f6bcf39d1e22 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 10:40:47.673211 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69498a47-7209-4c38-b828-f6bcf39d1e22 None admin] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:40:47.673498 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69498a47-7209-4c38-b828-f6bcf39d1e22 None admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:40:47.673775 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69498a47-7209-4c38-b828-f6bcf39d1e22 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:40:47.678899 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=100 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 10:40:47.697348 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb7a0723-25e5-49ba-bd85-7a4fe76aad52 None admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 10:40:47.731591 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28d924f4-44e0-45f7-a53e-5bb6832f6f7b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 10:40:47.732202 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28d924f4-44e0-45f7-a53e-5bb6832f6f7b None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:40:47.732494 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28d924f4-44e0-45f7-a53e-5bb6832f6f7b None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:40:47.732765 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28d924f4-44e0-45f7-a53e-5bb6832f6f7b None admin] PATH_INFO: `/v3/roles` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:40:47.733437 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28d924f4-44e0-45f7-a53e-5bb6832f6f7b None admin] RBAC: Adding query filter params (name=admin) {{(pid=80143) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 10:40:47.733743 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28d924f4-44e0-45f7-a53e-5bb6832f6f7b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 10:40:47.735151 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28d924f4-44e0-45f7-a53e-5bb6832f6f7b None admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 10:40:47.741848 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69498a47-7209-4c38-b828-f6bcf39d1e22 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 10:40:47.743374 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69498a47-7209-4c38-b828-f6bcf39d1e22 None admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 10:40:47.748096 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 7/15] 10.222.0.25 () {56 vars in 1099 bytes} [Sun Aug 7 10:40:47 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 07 10:40:47.753768 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28d924f4-44e0-45f7-a53e-5bb6832f6f7b None admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 10:40:47.840304 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 9/16] 10.222.0.25 () {58 vars in 1282 bytes} [Sun Aug 7 10:40:47 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 306 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Aug 07 10:40:47.846103 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69498a47-7209-4c38-b828-f6bcf39d1e22 None admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 10:40:47.849354 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=100 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 10:40:47.950875 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69498a47-7209-4c38-b828-f6bcf39d1e22 None admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 10:40:47.964375 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28d924f4-44e0-45f7-a53e-5bb6832f6f7b None admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 10:40:47.983711 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10e76b69-4130-417a-bddc-99fa0e259ffb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 10:40:47.984555 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10e76b69-4130-417a-bddc-99fa0e259ffb None admin] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:40:47.984840 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10e76b69-4130-417a-bddc-99fa0e259ffb None admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:40:47.985117 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10e76b69-4130-417a-bddc-99fa0e259ffb None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:40:47.999622 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c82538c9-5f99-42f4-9c4f-15af63c5b371 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 10:40:48.000252 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c82538c9-5f99-42f4-9c4f-15af63c5b371 None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:40:48.000533 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c82538c9-5f99-42f4-9c4f-15af63c5b371 None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:40:48.000808 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c82538c9-5f99-42f4-9c4f-15af63c5b371 None admin] PATH_INFO: `/v3/users/bac4e6b76fe64217ac8fa44e71cffe6a` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:40:48.002032 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c82538c9-5f99-42f4-9c4f-15af63c5b371 None admin] RBAC: Authorizing `identity:get_user(user_id=bac4e6b76fe64217ac8fa44e71cffe6a)` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 10:40:48.003551 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c82538c9-5f99-42f4-9c4f-15af63c5b371 None admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 10:40:48.005152 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 8/17] 10.222.0.25 () {56 vars in 1177 bytes} [Sun Aug 7 10:40:47 2022] GET /identity/v3/users/bac4e6b76fe64217ac8fa44e71cffe6a => generated 247 bytes in 252 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 07 10:40:48.007536 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10e76b69-4130-417a-bddc-99fa0e259ffb None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 10:40:48.008977 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10e76b69-4130-417a-bddc-99fa0e259ffb None admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 10:40:48.010595 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c82538c9-5f99-42f4-9c4f-15af63c5b371 None admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 10:40:48.025536 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=100 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 10:40:48.027739 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application [None req-10e76b69-4130-417a-bddc-99fa0e259ffb None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 07 10:40:48.027739 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 07 10:40:48.027739 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 07 10:40:48.027739 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 07 10:40:48.027739 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 07 10:40:48.027739 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 07 10:40:48.027739 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 07 10:40:48.027739 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 07 10:40:48.027739 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 07 10:40:48.027739 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 07 10:40:48.027739 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 07 10:40:48.027739 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 07 10:40:48.027739 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 07 10:40:48.027739 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 07 10:40:48.027739 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 07 10:40:48.027739 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 07 10:40:48.028734 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 07 10:40:48.028734 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 07 10:40:48.028734 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 07 10:40:48.028734 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 07 10:40:48.028734 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 07 10:40:48.028734 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self.get_or_create( Aug 07 10:40:48.028734 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 07 10:40:48.028734 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application with Lock( Aug 07 10:40:48.028734 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 07 10:40:48.028734 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self._enter() Aug 07 10:40:48.028734 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 07 10:40:48.028734 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 07 10:40:48.028734 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 07 10:40:48.028734 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self.creator() Aug 07 10:40:48.028734 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 07 10:40:48.028734 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application created_value = creator( Aug 07 10:40:48.028734 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 07 10:40:48.034323 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 07 10:40:48.034323 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 07 10:40:48.034323 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application Aug 07 10:40:48.034323 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 10/18] 10.222.0.25 () {56 vars in 1108 bytes} [Sun Aug 7 10:40:47 2022] GET /identity/v3/domains/Default => generated 87 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 07 10:40:48.034746 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10e76b69-4130-417a-bddc-99fa0e259ffb None admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 10:40:48.092599 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=100 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 10:40:48.118886 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c82538c9-5f99-42f4-9c4f-15af63c5b371 None admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 10:40:48.136694 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10e76b69-4130-417a-bddc-99fa0e259ffb None admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 10:40:48.151988 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c474b9da-5f0c-4816-923f-2ce9cd2c39e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 10:40:48.153180 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c474b9da-5f0c-4816-923f-2ce9cd2c39e2 None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:40:48.153528 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c474b9da-5f0c-4816-923f-2ce9cd2c39e2 None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:40:48.153745 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c474b9da-5f0c-4816-923f-2ce9cd2c39e2 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:40:48.154883 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c474b9da-5f0c-4816-923f-2ce9cd2c39e2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 10:40:48.156357 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c474b9da-5f0c-4816-923f-2ce9cd2c39e2 None admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 10:40:48.157914 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 9/19] 10.222.0.25 () {56 vars in 1108 bytes} [Sun Aug 7 10:40:48 2022] GET /identity/v3/domains/default => generated 199 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 07 10:40:48.163716 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c474b9da-5f0c-4816-923f-2ce9cd2c39e2 None admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 10:40:48.170832 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b288713-9199-4f2b-b538-c89eda16bb04 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 10:40:48.171495 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b288713-9199-4f2b-b538-c89eda16bb04 None admin] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:40:48.171779 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b288713-9199-4f2b-b538-c89eda16bb04 None admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:40:48.172051 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b288713-9199-4f2b-b538-c89eda16bb04 None admin] PATH_INFO: `/v3/domains` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:40:48.172733 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b288713-9199-4f2b-b538-c89eda16bb04 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80144) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 10:40:48.173046 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b288713-9199-4f2b-b538-c89eda16bb04 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 10:40:48.174415 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b288713-9199-4f2b-b538-c89eda16bb04 None admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 10:40:48.191570 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 11/20] 10.222.0.25 () {56 vars in 1109 bytes} [Sun Aug 7 10:40:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 07 10:40:48.248271 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=100 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 10:40:48.269300 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c474b9da-5f0c-4816-923f-2ce9cd2c39e2 None admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 10:40:48.304341 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f8035ff-2e63-4fd4-89e4-e5367cd5ef2d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 10:40:48.304993 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f8035ff-2e63-4fd4-89e4-e5367cd5ef2d None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:40:48.305235 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f8035ff-2e63-4fd4-89e4-e5367cd5ef2d None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:40:48.305506 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f8035ff-2e63-4fd4-89e4-e5367cd5ef2d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:40:48.306248 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f8035ff-2e63-4fd4-89e4-e5367cd5ef2d None admin] RBAC: Adding query filter params (role.id=81b1f1bcee2a473088fbbd327c1c5b23, scope.domain.id=default, user.id=bac4e6b76fe64217ac8fa44e71cffe6a) {{(pid=80143) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 10:40:48.306555 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f8035ff-2e63-4fd4-89e4-e5367cd5ef2d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 10:40:48.308034 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f8035ff-2e63-4fd4-89e4-e5367cd5ef2d None admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 10:40:48.315359 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 10/21] 10.222.0.25 () {56 vars in 1322 bytes} [Sun Aug 7 10:40:48 2022] GET /identity/v3/role_assignments?user.id=bac4e6b76fe64217ac8fa44e71cffe6a&scope.domain.id=default&role.id=81b1f1bcee2a473088fbbd327c1c5b23 => generated 233 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 07 10:40:50.514569 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70ed6d88-809a-445c-9a46-0e4d6606e101 None None] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:40:50.515120 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70ed6d88-809a-445c-9a46-0e4d6606e101 None None] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:40:50.515205 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70ed6d88-809a-445c-9a46-0e4d6606e101 None None] PATH_INFO: `/` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:40:50.516480 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 12/22] 10.222.0.25 () {54 vars in 927 bytes} [Sun Aug 7 10:40:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 07 10:40:50.521840 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea879cf0-e9b9-46b2-b295-b44bc70b545d None None] REQUEST_METHOD: `POST` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:40:50.522232 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea879cf0-e9b9-46b2-b295-b44bc70b545d None None] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:40:50.522418 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea879cf0-e9b9-46b2-b295-b44bc70b545d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:40:50.560090 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.auth.core [None req-ea879cf0-e9b9-46b2-b295-b44bc70b545d None None] MFA Rules not processed for user `bac4e6b76fe64217ac8fa44e71cffe6a`. Rule list: `[]` (Enabled: `True`). {{(pid=80143) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 10:40:50.561293 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-ea879cf0-e9b9-46b2-b295-b44bc70b545d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 10:40:50.608561 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2960d391-f49b-4023-be1f-b1f2a725b1d2 None None] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:40:50.608856 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2960d391-f49b-4023-be1f-b1f2a725b1d2 None None] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:40:50.609120 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2960d391-f49b-4023-be1f-b1f2a725b1d2 None None] PATH_INFO: `/` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:40:50.610680 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 13/23] 10.222.0.25 () {54 vars in 927 bytes} [Sun Aug 7 10:40:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 07 10:40:50.616204 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cd2fd50-78b9-4322-8fe8-9abdae25c071 None None] REQUEST_METHOD: `POST` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:40:50.616490 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cd2fd50-78b9-4322-8fe8-9abdae25c071 None None] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:40:50.616772 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cd2fd50-78b9-4322-8fe8-9abdae25c071 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:40:50.647752 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 11/24] 10.222.0.25 () {58 vars in 1024 bytes} [Sun Aug 7 10:40:50 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 07 10:40:50.663954 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.auth.core [None req-4cd2fd50-78b9-4322-8fe8-9abdae25c071 None None] MFA Rules not processed for user `bac4e6b76fe64217ac8fa44e71cffe6a`. Rule list: `[]` (Enabled: `True`). {{(pid=80144) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 10:40:50.665176 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.fernet_utils [None req-4cd2fd50-78b9-4322-8fe8-9abdae25c071 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80144) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 10:40:50.687724 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea879cf0-e9b9-46b2-b295-b44bc70b545d None None] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 10:40:50.815505 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=100 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 10:40:50.832489 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 14/25] 10.222.0.25 () {58 vars in 1024 bytes} [Sun Aug 7 10:40:50 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 217 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 07 10:40:50.862501 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea879cf0-e9b9-46b2-b295-b44bc70b545d None None] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 10:40:50.865932 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cd2fd50-78b9-4322-8fe8-9abdae25c071 None None] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 10:40:50.935619 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dfdf0c1-76d4-4c61-b91d-d47565873d9a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 10:40:50.936262 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dfdf0c1-76d4-4c61-b91d-d47565873d9a None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:40:50.936541 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dfdf0c1-76d4-4c61-b91d-d47565873d9a None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:40:50.936812 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dfdf0c1-76d4-4c61-b91d-d47565873d9a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:40:50.956061 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dfdf0c1-76d4-4c61-b91d-d47565873d9a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 10:40:50.957480 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dfdf0c1-76d4-4c61-b91d-d47565873d9a None admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 10:40:50.966948 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=100 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 10:40:50.995850 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application [None req-7dfdf0c1-76d4-4c61-b91d-d47565873d9a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 07 10:40:50.995850 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 07 10:40:50.995850 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 07 10:40:50.995850 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 07 10:40:50.995850 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 07 10:40:50.995850 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 07 10:40:50.995850 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 07 10:40:50.995850 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 07 10:40:50.995850 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 07 10:40:50.995850 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 07 10:40:50.995850 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 07 10:40:50.995850 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 07 10:40:50.995850 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 07 10:40:50.995850 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 07 10:40:50.995850 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 07 10:40:50.995850 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 07 10:40:50.998206 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 07 10:40:50.998206 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 07 10:40:50.998206 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 07 10:40:50.998206 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 07 10:40:50.998206 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 07 10:40:50.998206 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.get_or_create( Aug 07 10:40:50.998206 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 07 10:40:50.998206 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application with Lock( Aug 07 10:40:50.998206 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 07 10:40:50.998206 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self._enter() Aug 07 10:40:50.998206 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 07 10:40:50.998206 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 07 10:40:50.998206 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 07 10:40:50.998206 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.creator() Aug 07 10:40:50.998206 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 07 10:40:50.998206 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application created_value = creator( Aug 07 10:40:50.998206 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 07 10:40:50.999248 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 07 10:40:50.999248 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 07 10:40:50.999248 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application Aug 07 10:40:50.999248 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 12/26] 10.222.0.25 () {56 vars in 1108 bytes} [Sun Aug 7 10:40:50 2022] GET /identity/v3/domains/Default => generated 87 bytes in 310 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 07 10:40:51.003725 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dfdf0c1-76d4-4c61-b91d-d47565873d9a None admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 10:40:51.042971 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cd2fd50-78b9-4322-8fe8-9abdae25c071 None None] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 10:40:51.218197 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecde6528-c0ff-465b-a049-5c591a8e2259 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 10:40:51.218782 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecde6528-c0ff-465b-a049-5c591a8e2259 None admin] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:40:51.219102 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecde6528-c0ff-465b-a049-5c591a8e2259 None admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:40:51.219384 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecde6528-c0ff-465b-a049-5c591a8e2259 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:40:51.233031 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecde6528-c0ff-465b-a049-5c591a8e2259 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 07 10:40:51.240667 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application [None req-ecde6528-c0ff-465b-a049-5c591a8e2259 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 07 10:40:51.240667 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 07 10:40:51.240667 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 07 10:40:51.240667 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 07 10:40:51.240667 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 07 10:40:51.240667 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 07 10:40:51.240667 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 07 10:40:51.240667 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 07 10:40:51.240667 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 07 10:40:51.240667 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 07 10:40:51.240667 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 07 10:40:51.240667 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 07 10:40:51.240667 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 07 10:40:51.240667 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 07 10:40:51.240667 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 07 10:40:51.240667 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 07 10:40:51.241503 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 07 10:40:51.241503 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 07 10:40:51.241503 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 07 10:40:51.241503 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 07 10:40:51.241503 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 07 10:40:51.241503 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 07 10:40:51.241503 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 07 10:40:51.241503 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 07 10:40:51.241503 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 07 10:40:51.241503 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self.get_or_create( Aug 07 10:40:51.241503 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 07 10:40:51.241503 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application with Lock( Aug 07 10:40:51.241503 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 07 10:40:51.241503 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self._enter() Aug 07 10:40:51.241503 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 07 10:40:51.241503 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 07 10:40:51.242667 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 07 10:40:51.242667 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self.creator() Aug 07 10:40:51.242667 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 07 10:40:51.242667 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application created_value = creator( Aug 07 10:40:51.242667 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 07 10:40:51.242667 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 07 10:40:51.242667 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 07 10:40:51.242667 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 07 10:40:51.242667 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 07 10:40:51.242667 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 07 10:40:51.242667 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 07 10:40:51.242667 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application Aug 07 10:40:51.242667 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 15/27] 10.222.0.25 () {56 vars in 1096 bytes} [Sun Aug 7 10:40:50 2022] GET /identity/v3/roles/admin => generated 83 bytes in 376 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 07 10:40:51.243574 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85253ab8-ef36-44e0-a983-7ff892e84a75 None None] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:40:51.243574 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85253ab8-ef36-44e0-a983-7ff892e84a75 None None] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:40:51.243688 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85253ab8-ef36-44e0-a983-7ff892e84a75 None None] PATH_INFO: `/` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:40:51.244919 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 16/28] 10.222.0.25 () {54 vars in 927 bytes} [Sun Aug 7 10:40:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 07 10:40:51.256096 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85253ab8-ef36-44e0-a983-7ff892e84a75 None None] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 10:40:51.257928 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dfdf0c1-76d4-4c61-b91d-d47565873d9a None admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 10:40:51.288566 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=100 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 10:40:51.299505 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb845a47-ebac-4b27-8d45-dc5f64bf34d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 10:40:51.300443 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb845a47-ebac-4b27-8d45-dc5f64bf34d2 None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:40:51.300947 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb845a47-ebac-4b27-8d45-dc5f64bf34d2 None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:40:51.301421 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb845a47-ebac-4b27-8d45-dc5f64bf34d2 None admin] PATH_INFO: `/v3/domains` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:40:51.302278 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb845a47-ebac-4b27-8d45-dc5f64bf34d2 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80143) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 10:40:51.302829 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb845a47-ebac-4b27-8d45-dc5f64bf34d2 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 10:40:51.304523 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb845a47-ebac-4b27-8d45-dc5f64bf34d2 None admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 10:40:51.326582 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 13/29] 10.222.0.25 () {56 vars in 1109 bytes} [Sun Aug 7 10:40:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 323 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 10:40:51.328369 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c40d191-ff93-44b4-89fb-7bb1b3656279 None None] REQUEST_METHOD: `POST` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:40:51.328857 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c40d191-ff93-44b4-89fb-7bb1b3656279 None None] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:40:51.329358 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c40d191-ff93-44b4-89fb-7bb1b3656279 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:40:51.350328 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.auth.core [None req-3c40d191-ff93-44b4-89fb-7bb1b3656279 None None] MFA Rules not processed for user `bac4e6b76fe64217ac8fa44e71cffe6a`. Rule list: `[]` (Enabled: `True`). {{(pid=80143) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 10:40:51.351981 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-3c40d191-ff93-44b4-89fb-7bb1b3656279 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 10:40:51.401697 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85253ab8-ef36-44e0-a983-7ff892e84a75 None None] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 10:40:51.409932 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=100 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 10:40:51.438677 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73624657-54ea-4e6a-893c-807d83261925 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 10:40:51.439407 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73624657-54ea-4e6a-893c-807d83261925 None admin] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:40:51.439663 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73624657-54ea-4e6a-893c-807d83261925 None admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:40:51.439880 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73624657-54ea-4e6a-893c-807d83261925 None admin] PATH_INFO: `/v3/roles` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:40:51.440546 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73624657-54ea-4e6a-893c-807d83261925 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80144) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 10:40:51.440846 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73624657-54ea-4e6a-893c-807d83261925 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 10:40:51.442214 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73624657-54ea-4e6a-893c-807d83261925 None admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 10:40:51.444576 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 14/30] 10.222.0.25 () {58 vars in 1024 bytes} [Sun Aug 7 10:40:51 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 07 10:40:51.445777 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c40d191-ff93-44b4-89fb-7bb1b3656279 None None] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 10:40:51.458956 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 17/31] 10.222.0.25 () {56 vars in 1099 bytes} [Sun Aug 7 10:40:51 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 10:40:51.458956 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-813835a6-10ff-467b-9a4d-17092838a11f None None] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:40:51.458956 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-813835a6-10ff-467b-9a4d-17092838a11f None None] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:40:51.458956 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-813835a6-10ff-467b-9a4d-17092838a11f None None] PATH_INFO: `/` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:40:51.458956 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 18/32] 10.222.0.25 () {54 vars in 927 bytes} [Sun Aug 7 10:40:51 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 07 10:40:51.466966 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-813835a6-10ff-467b-9a4d-17092838a11f None None] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 10:40:51.583436 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-813835a6-10ff-467b-9a4d-17092838a11f None None] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 10:40:51.593472 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c40d191-ff93-44b4-89fb-7bb1b3656279 None None] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 10:40:51.615786 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=100 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 10:40:51.624897 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b72da796-8086-4b16-a606-955052e78410 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 10:40:51.625764 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b72da796-8086-4b16-a606-955052e78410 None admin] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:40:51.626222 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b72da796-8086-4b16-a606-955052e78410 None admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:40:51.626752 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b72da796-8086-4b16-a606-955052e78410 None admin] PATH_INFO: `/v3/users/bac4e6b76fe64217ac8fa44e71cffe6a` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:40:51.628549 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b72da796-8086-4b16-a606-955052e78410 None admin] RBAC: Authorizing `identity:get_user(user_id=bac4e6b76fe64217ac8fa44e71cffe6a)` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 10:40:51.630314 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b72da796-8086-4b16-a606-955052e78410 None admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 10:40:51.632571 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 19/33] 10.222.0.25 () {56 vars in 1177 bytes} [Sun Aug 7 10:40:51 2022] GET /identity/v3/users/bac4e6b76fe64217ac8fa44e71cffe6a => generated 247 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 10:40:51.633905 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-346b82a2-f801-4c51-ad06-5a08eb277472 None None] REQUEST_METHOD: `POST` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:40:51.634359 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-346b82a2-f801-4c51-ad06-5a08eb277472 None None] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:40:51.634828 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-346b82a2-f801-4c51-ad06-5a08eb277472 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:40:51.640285 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e453c0d-d3fe-47bc-8bd0-388afc3ed4c5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 10:40:51.640911 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e453c0d-d3fe-47bc-8bd0-388afc3ed4c5 None admin] REQUEST_METHOD: `POST` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:40:51.641195 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e453c0d-d3fe-47bc-8bd0-388afc3ed4c5 None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:40:51.641470 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e453c0d-d3fe-47bc-8bd0-388afc3ed4c5 None admin] PATH_INFO: `/v3/projects` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:40:51.642119 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e453c0d-d3fe-47bc-8bd0-388afc3ed4c5 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 10:40:51.644036 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e453c0d-d3fe-47bc-8bd0-388afc3ed4c5 None admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 10:40:51.655941 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.auth.core [None req-346b82a2-f801-4c51-ad06-5a08eb277472 None None] MFA Rules not processed for user `bac4e6b76fe64217ac8fa44e71cffe6a`. Rule list: `[]` (Enabled: `True`). {{(pid=80144) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 10:40:51.657595 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.fernet_utils [None req-346b82a2-f801-4c51-ad06-5a08eb277472 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80144) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 10:40:51.691125 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=100 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 10:40:51.733871 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 20/34] 10.222.0.25 () {58 vars in 1024 bytes} [Sun Aug 7 10:40:51 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 07 10:40:51.735117 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-346b82a2-f801-4c51-ad06-5a08eb277472 None None] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 10:40:51.806014 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 15/35] 10.222.0.25 () {60 vars in 1141 bytes} [Sun Aug 7 10:40:51 2022] POST /identity/v3/projects => generated 301 bytes in 361 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 07 10:40:51.807385 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e453c0d-d3fe-47bc-8bd0-388afc3ed4c5 None admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 10:40:51.859955 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-346b82a2-f801-4c51-ad06-5a08eb277472 None None] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 10:40:51.900276 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa57f9f7-b1a8-4faa-bf5f-219bc0ffac75 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 10:40:51.901273 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa57f9f7-b1a8-4faa-bf5f-219bc0ffac75 None admin] REQUEST_METHOD: `POST` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:40:51.901866 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa57f9f7-b1a8-4faa-bf5f-219bc0ffac75 None admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:40:51.902406 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa57f9f7-b1a8-4faa-bf5f-219bc0ffac75 None admin] PATH_INFO: `/v3/roles` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:40:51.903832 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa57f9f7-b1a8-4faa-bf5f-219bc0ffac75 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 10:40:51.905563 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa57f9f7-b1a8-4faa-bf5f-219bc0ffac75 None admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 10:40:51.928504 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e453c0d-d3fe-47bc-8bd0-388afc3ed4c5 None admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 10:40:51.968536 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7793623-1914-49c0-8221-e4ed2a88d72e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 10:40:51.969230 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7793623-1914-49c0-8221-e4ed2a88d72e None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:40:51.969522 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7793623-1914-49c0-8221-e4ed2a88d72e None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:40:51.969800 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7793623-1914-49c0-8221-e4ed2a88d72e None admin] PATH_INFO: `/v3/domains/default` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:40:51.971138 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7793623-1914-49c0-8221-e4ed2a88d72e None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 10:40:51.972749 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7793623-1914-49c0-8221-e4ed2a88d72e None admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 10:40:51.974388 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 16/36] 10.222.0.25 () {56 vars in 1108 bytes} [Sun Aug 7 10:40:51 2022] GET /identity/v3/domains/default => generated 199 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 10:40:51.975176 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7793623-1914-49c0-8221-e4ed2a88d72e None admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 10:40:52.022041 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 10:40:52.041984 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 21/37] 10.222.0.25 () {60 vars in 1131 bytes} [Sun Aug 7 10:40:51 2022] POST /identity/v3/roles => generated 219 bytes in 307 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 07 10:40:52.044056 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdc4461c-0b4b-4875-9b34-848231832e6c None None] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:40:52.044333 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdc4461c-0b4b-4875-9b34-848231832e6c None None] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:40:52.044611 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdc4461c-0b4b-4875-9b34-848231832e6c None None] PATH_INFO: `/` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:40:52.046159 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 22/38] 10.222.0.25 () {54 vars in 927 bytes} [Sun Aug 7 10:40:52 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 07 10:40:52.046970 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdc4461c-0b4b-4875-9b34-848231832e6c None None] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 10:40:52.086121 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 10:40:52.165103 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7793623-1914-49c0-8221-e4ed2a88d72e None admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 10:40:52.233807 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dfab27c-24f6-404f-8300-30cce39ab9cb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 10:40:52.234738 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dfab27c-24f6-404f-8300-30cce39ab9cb None admin] REQUEST_METHOD: `POST` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:40:52.235237 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dfab27c-24f6-404f-8300-30cce39ab9cb None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:40:52.235709 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dfab27c-24f6-404f-8300-30cce39ab9cb None admin] PATH_INFO: `/v3/roles` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:40:52.236719 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdc4461c-0b4b-4875-9b34-848231832e6c None None] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 10:40:52.237032 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dfab27c-24f6-404f-8300-30cce39ab9cb None admin] RBAC: Authorizing `identity:create_role()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 10:40:52.239445 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dfab27c-24f6-404f-8300-30cce39ab9cb None admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 10:40:52.266990 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 17/39] 10.222.0.25 () {60 vars in 1131 bytes} [Sun Aug 7 10:40:51 2022] POST /identity/v3/roles => generated 225 bytes in 292 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 07 10:40:52.268914 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bef0bdd-3b22-4f14-870a-867f0f52d26f None None] REQUEST_METHOD: `POST` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:40:52.269362 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bef0bdd-3b22-4f14-870a-867f0f52d26f None None] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:40:52.269829 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bef0bdd-3b22-4f14-870a-867f0f52d26f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:40:52.299256 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.auth.core [None req-1bef0bdd-3b22-4f14-870a-867f0f52d26f None None] MFA Rules not processed for user `bac4e6b76fe64217ac8fa44e71cffe6a`. Rule list: `[]` (Enabled: `True`). {{(pid=80143) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 10:40:52.301787 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-1bef0bdd-3b22-4f14-870a-867f0f52d26f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 10:40:52.302091 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ff1aeca-ba88-4553-8128-87a0355b434c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 10:40:52.303248 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ff1aeca-ba88-4553-8128-87a0355b434c None admin] REQUEST_METHOD: `PUT` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:40:52.303751 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ff1aeca-ba88-4553-8128-87a0355b434c None admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:40:52.304401 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ff1aeca-ba88-4553-8128-87a0355b434c None admin] PATH_INFO: `/v3/domains/default/users/bac4e6b76fe64217ac8fa44e71cffe6a/roles/81b1f1bcee2a473088fbbd327c1c5b23` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:40:52.308488 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ff1aeca-ba88-4553-8128-87a0355b434c None admin] RBAC: Authorizing `identity:create_grant(domain_id=default, user_id=bac4e6b76fe64217ac8fa44e71cffe6a, role_id=81b1f1bcee2a473088fbbd327c1c5b23)` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 10:40:52.312589 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ff1aeca-ba88-4553-8128-87a0355b434c None admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 10:40:52.340353 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 23/40] 10.222.0.25 () {58 vars in 1361 bytes} [Sun Aug 7 10:40:52 2022] PUT /identity/v3/domains/default/users/bac4e6b76fe64217ac8fa44e71cffe6a/roles/81b1f1bcee2a473088fbbd327c1c5b23 => generated 0 bytes in 293 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 07 10:40:52.342527 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3d2927a-8544-463b-bef9-d7644236dee6 None None] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:40:52.343021 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3d2927a-8544-463b-bef9-d7644236dee6 None None] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:40:52.343547 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3d2927a-8544-463b-bef9-d7644236dee6 None None] PATH_INFO: `/` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:40:52.346155 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 24/41] 10.222.0.25 () {54 vars in 927 bytes} [Sun Aug 7 10:40:52 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 07 10:40:52.352372 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-260fd510-2acb-44bb-a738-c2fef8d1518c None None] REQUEST_METHOD: `POST` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:40:52.352654 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-260fd510-2acb-44bb-a738-c2fef8d1518c None None] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:40:52.352934 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-260fd510-2acb-44bb-a738-c2fef8d1518c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:40:52.381371 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.auth.core [None req-260fd510-2acb-44bb-a738-c2fef8d1518c None None] MFA Rules not processed for user `bac4e6b76fe64217ac8fa44e71cffe6a`. Rule list: `[]` (Enabled: `True`). {{(pid=80144) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 10:40:52.383199 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.fernet_utils [None req-260fd510-2acb-44bb-a738-c2fef8d1518c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80144) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 10:40:52.414304 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 18/42] 10.222.0.25 () {58 vars in 1024 bytes} [Sun Aug 7 10:40:52 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 147 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 07 10:40:52.444993 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bef0bdd-3b22-4f14-870a-867f0f52d26f None None] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 10:40:52.482499 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 10:40:52.502718 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 25/43] 10.222.0.25 () {58 vars in 1024 bytes} [Sun Aug 7 10:40:52 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 151 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 07 10:40:52.526811 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-260fd510-2acb-44bb-a738-c2fef8d1518c None None] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 10:40:52.549403 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 10:40:52.577509 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bef0bdd-3b22-4f14-870a-867f0f52d26f None None] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 10:40:52.614171 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0018712e-f2f1-48d3-9909-cdd6de3a9463 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 10:40:52.614814 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0018712e-f2f1-48d3-9909-cdd6de3a9463 None admin] REQUEST_METHOD: `POST` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:40:52.615091 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0018712e-f2f1-48d3-9909-cdd6de3a9463 None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:40:52.615370 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0018712e-f2f1-48d3-9909-cdd6de3a9463 None admin] PATH_INFO: `/v3/roles` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:40:52.616046 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0018712e-f2f1-48d3-9909-cdd6de3a9463 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 10:40:52.617436 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0018712e-f2f1-48d3-9909-cdd6de3a9463 None admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 10:40:52.633267 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 19/44] 10.222.0.25 () {60 vars in 1131 bytes} [Sun Aug 7 10:40:52 2022] POST /identity/v3/roles => generated 223 bytes in 189 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 07 10:40:52.646132 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-260fd510-2acb-44bb-a738-c2fef8d1518c None None] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 10:40:52.689351 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc7cab25-d256-43db-9af6-88e2b22d1cac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 10:40:52.690315 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc7cab25-d256-43db-9af6-88e2b22d1cac None admin] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:40:52.690776 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc7cab25-d256-43db-9af6-88e2b22d1cac None admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:40:52.691331 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc7cab25-d256-43db-9af6-88e2b22d1cac None admin] PATH_INFO: `/v3/domains/default` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:40:52.692857 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc7cab25-d256-43db-9af6-88e2b22d1cac None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 10:40:52.694518 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc7cab25-d256-43db-9af6-88e2b22d1cac None admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 10:40:52.696418 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 26/45] 10.222.0.25 () {56 vars in 1108 bytes} [Sun Aug 7 10:40:52 2022] GET /identity/v3/domains/default => generated 199 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 07 10:40:52.702484 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0018712e-f2f1-48d3-9909-cdd6de3a9463 None admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 10:40:52.801825 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 10:40:52.819058 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0018712e-f2f1-48d3-9909-cdd6de3a9463 None admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 10:40:52.854630 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6802ffed-8e96-4bab-a8da-686e3083ab77 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 10:40:52.855284 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6802ffed-8e96-4bab-a8da-686e3083ab77 None admin] REQUEST_METHOD: `POST` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:40:52.856216 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6802ffed-8e96-4bab-a8da-686e3083ab77 None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:40:52.856497 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6802ffed-8e96-4bab-a8da-686e3083ab77 None admin] PATH_INFO: `/v3/projects` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:40:52.857154 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6802ffed-8e96-4bab-a8da-686e3083ab77 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 10:40:52.858601 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6802ffed-8e96-4bab-a8da-686e3083ab77 None admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 10:40:52.885584 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 20/46] 10.222.0.25 () {60 vars in 1141 bytes} [Sun Aug 7 10:40:52 2022] POST /identity/v3/projects => generated 312 bytes in 184 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 07 10:40:54.389834 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0b5df57-814b-44f8-8e6f-7ce6511162ce None None] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:40:54.390378 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0b5df57-814b-44f8-8e6f-7ce6511162ce None None] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:40:54.390467 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0b5df57-814b-44f8-8e6f-7ce6511162ce None None] PATH_INFO: `/` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:40:54.391825 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 27/47] 10.222.0.25 () {54 vars in 927 bytes} [Sun Aug 7 10:40:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 07 10:40:54.397067 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65fd43ef-6610-470d-a0e7-c7c60e22e62f None None] REQUEST_METHOD: `POST` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:40:54.397459 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65fd43ef-6610-470d-a0e7-c7c60e22e62f None None] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:40:54.397654 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65fd43ef-6610-470d-a0e7-c7c60e22e62f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:40:54.416146 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.auth.core [None req-65fd43ef-6610-470d-a0e7-c7c60e22e62f None None] MFA Rules not processed for user `bac4e6b76fe64217ac8fa44e71cffe6a`. Rule list: `[]` (Enabled: `True`). {{(pid=80143) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 10:40:54.417328 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-65fd43ef-6610-470d-a0e7-c7c60e22e62f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 10:40:54.482667 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 21/48] 10.222.0.25 () {58 vars in 1024 bytes} [Sun Aug 7 10:40:54 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 07 10:40:54.505655 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0b5df57-814b-44f8-8e6f-7ce6511162ce None None] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 10:40:54.591172 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=100 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 10:40:54.609154 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0b5df57-814b-44f8-8e6f-7ce6511162ce None None] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 10:40:54.641972 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1343b5c2-b692-4e5e-a183-12ea2787af20 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 10:40:54.642537 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1343b5c2-b692-4e5e-a183-12ea2787af20 None admin] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:40:54.642819 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1343b5c2-b692-4e5e-a183-12ea2787af20 None admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:40:54.643131 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1343b5c2-b692-4e5e-a183-12ea2787af20 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:40:54.653445 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1343b5c2-b692-4e5e-a183-12ea2787af20 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 07 10:40:54.658132 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application [None req-1343b5c2-b692-4e5e-a183-12ea2787af20 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 07 10:40:54.658132 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 07 10:40:54.658132 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 07 10:40:54.658132 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 07 10:40:54.658132 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 07 10:40:54.658132 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 07 10:40:54.658132 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 07 10:40:54.658132 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 07 10:40:54.658132 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 07 10:40:54.658132 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 07 10:40:54.658132 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 07 10:40:54.658132 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 07 10:40:54.658132 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 07 10:40:54.658132 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 07 10:40:54.658132 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 07 10:40:54.658132 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 07 10:40:54.659072 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 07 10:40:54.659072 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 07 10:40:54.659072 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 07 10:40:54.659072 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 07 10:40:54.659072 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 07 10:40:54.659072 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 07 10:40:54.659072 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 07 10:40:54.659072 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 07 10:40:54.659072 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 07 10:40:54.659072 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self.get_or_create( Aug 07 10:40:54.659072 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 07 10:40:54.659072 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application with Lock( Aug 07 10:40:54.659072 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 07 10:40:54.659072 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self._enter() Aug 07 10:40:54.659072 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 07 10:40:54.659072 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 07 10:40:54.659880 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 07 10:40:54.659880 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self.creator() Aug 07 10:40:54.659880 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 07 10:40:54.659880 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application created_value = creator( Aug 07 10:40:54.659880 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 07 10:40:54.659880 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 07 10:40:54.659880 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 07 10:40:54.659880 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 07 10:40:54.659880 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 07 10:40:54.659880 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 07 10:40:54.659880 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 07 10:40:54.659880 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application Aug 07 10:40:54.659880 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 28/49] 10.222.0.25 () {56 vars in 1096 bytes} [Sun Aug 7 10:40:54 2022] GET /identity/v3/roles/admin => generated 83 bytes in 154 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 07 10:40:54.665184 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65fd43ef-6610-470d-a0e7-c7c60e22e62f None None] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 10:40:54.728005 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 10:40:54.762145 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65fd43ef-6610-470d-a0e7-c7c60e22e62f None None] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 10:40:54.794666 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fb317e7-de83-4c57-b131-f6a6d28f9577 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 10:40:54.795309 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fb317e7-de83-4c57-b131-f6a6d28f9577 None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:40:54.795593 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fb317e7-de83-4c57-b131-f6a6d28f9577 None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:40:54.795865 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fb317e7-de83-4c57-b131-f6a6d28f9577 None admin] PATH_INFO: `/v3/roles` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:40:54.796529 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fb317e7-de83-4c57-b131-f6a6d28f9577 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80143) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 10:40:54.796831 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fb317e7-de83-4c57-b131-f6a6d28f9577 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 10:40:54.798202 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fb317e7-de83-4c57-b131-f6a6d28f9577 None admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 10:40:54.806032 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 22/50] 10.222.0.25 () {56 vars in 1099 bytes} [Sun Aug 7 10:40:54 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 10:40:54.812313 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1343b5c2-b692-4e5e-a183-12ea2787af20 None admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 10:40:54.883055 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-728801b6-b41d-4614-a3be-e44aa6fec0fa None None] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:40:54.883343 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-728801b6-b41d-4614-a3be-e44aa6fec0fa None None] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:40:54.883619 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-728801b6-b41d-4614-a3be-e44aa6fec0fa None None] PATH_INFO: `/` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:40:54.884946 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 23/51] 10.222.0.25 () {54 vars in 927 bytes} [Sun Aug 7 10:40:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 07 10:40:54.889964 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0af61e92-081f-4570-8ea0-44a74775237e None None] REQUEST_METHOD: `POST` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:40:54.890248 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0af61e92-081f-4570-8ea0-44a74775237e None None] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:40:54.890525 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0af61e92-081f-4570-8ea0-44a74775237e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:40:54.907724 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.auth.core [None req-0af61e92-081f-4570-8ea0-44a74775237e None None] MFA Rules not processed for user `bac4e6b76fe64217ac8fa44e71cffe6a`. Rule list: `[]` (Enabled: `True`). {{(pid=80143) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 10:40:54.908838 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-0af61e92-081f-4570-8ea0-44a74775237e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 10:40:54.909617 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1343b5c2-b692-4e5e-a183-12ea2787af20 None admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 10:40:54.943406 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc81d6c5-6704-4acb-87a1-9c16a6610243 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 10:40:54.944047 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc81d6c5-6704-4acb-87a1-9c16a6610243 None admin] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:40:54.944332 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc81d6c5-6704-4acb-87a1-9c16a6610243 None admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:40:54.944612 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc81d6c5-6704-4acb-87a1-9c16a6610243 None admin] PATH_INFO: `/v3/users/bac4e6b76fe64217ac8fa44e71cffe6a` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:40:54.945745 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc81d6c5-6704-4acb-87a1-9c16a6610243 None admin] RBAC: Authorizing `identity:get_user(user_id=bac4e6b76fe64217ac8fa44e71cffe6a)` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 10:40:54.947249 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc81d6c5-6704-4acb-87a1-9c16a6610243 None admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 10:40:54.948828 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 29/52] 10.222.0.25 () {56 vars in 1177 bytes} [Sun Aug 7 10:40:54 2022] GET /identity/v3/users/bac4e6b76fe64217ac8fa44e71cffe6a => generated 247 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 10:40:54.954242 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc81d6c5-6704-4acb-87a1-9c16a6610243 None admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 10:40:54.968957 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=100 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 10:40:54.975895 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 24/53] 10.222.0.25 () {58 vars in 1024 bytes} [Sun Aug 7 10:40:54 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 07 10:40:54.998991 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0af61e92-081f-4570-8ea0-44a74775237e None None] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 10:40:55.053308 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc81d6c5-6704-4acb-87a1-9c16a6610243 None admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 10:40:55.057090 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=100 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 10:40:55.092693 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6c97a86-77e6-4b54-acdb-53d5e586553d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 10:40:55.093334 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6c97a86-77e6-4b54-acdb-53d5e586553d None admin] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:40:55.093615 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6c97a86-77e6-4b54-acdb-53d5e586553d None admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:40:55.093893 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6c97a86-77e6-4b54-acdb-53d5e586553d None admin] PATH_INFO: `/v3/domains/default` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:40:55.095039 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6c97a86-77e6-4b54-acdb-53d5e586553d None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 10:40:55.096483 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6c97a86-77e6-4b54-acdb-53d5e586553d None admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 10:40:55.098223 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 30/54] 10.222.0.25 () {56 vars in 1108 bytes} [Sun Aug 7 10:40:54 2022] GET /identity/v3/domains/default => generated 199 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 07 10:40:55.100555 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0af61e92-081f-4570-8ea0-44a74775237e None None] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 10:40:55.103648 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6c97a86-77e6-4b54-acdb-53d5e586553d None admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 10:40:55.133873 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-504ec048-27ff-4610-9152-83e25288c6d7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 10:40:55.134514 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-504ec048-27ff-4610-9152-83e25288c6d7 None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:40:55.134801 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-504ec048-27ff-4610-9152-83e25288c6d7 None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:40:55.135109 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-504ec048-27ff-4610-9152-83e25288c6d7 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:40:55.136243 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-504ec048-27ff-4610-9152-83e25288c6d7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 10:40:55.138328 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-504ec048-27ff-4610-9152-83e25288c6d7 None admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 10:40:55.139916 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 25/55] 10.222.0.25 () {56 vars in 1108 bytes} [Sun Aug 7 10:40:54 2022] GET /identity/v3/domains/default => generated 199 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 07 10:40:55.145437 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-504ec048-27ff-4610-9152-83e25288c6d7 None admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 10:40:55.184628 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=100 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 10:40:55.204243 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6c97a86-77e6-4b54-acdb-53d5e586553d None admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 10:40:55.237613 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-916b821a-d06a-4b06-a26d-8abcedc9e701 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 10:40:55.238223 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-916b821a-d06a-4b06-a26d-8abcedc9e701 None admin] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:40:55.238512 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-916b821a-d06a-4b06-a26d-8abcedc9e701 None admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:40:55.238789 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-916b821a-d06a-4b06-a26d-8abcedc9e701 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:40:55.239545 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-916b821a-d06a-4b06-a26d-8abcedc9e701 None admin] RBAC: Adding query filter params (role.id=81b1f1bcee2a473088fbbd327c1c5b23, scope.domain.id=default, user.id=bac4e6b76fe64217ac8fa44e71cffe6a) {{(pid=80144) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 10:40:55.239855 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-916b821a-d06a-4b06-a26d-8abcedc9e701 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 10:40:55.241291 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-916b821a-d06a-4b06-a26d-8abcedc9e701 None admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 10:40:55.244662 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-504ec048-27ff-4610-9152-83e25288c6d7 None admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 10:40:55.249068 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 31/56] 10.222.0.25 () {56 vars in 1322 bytes} [Sun Aug 7 10:40:55 2022] GET /identity/v3/role_assignments?user.id=bac4e6b76fe64217ac8fa44e71cffe6a&scope.domain.id=default&role.id=81b1f1bcee2a473088fbbd327c1c5b23 => generated 530 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 07 10:40:55.273599 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=100 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 10:40:55.282600 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-080faed4-d3fa-423a-85d9-fbc25f20f177 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 10:40:55.283269 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-080faed4-d3fa-423a-85d9-fbc25f20f177 None admin] REQUEST_METHOD: `POST` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:40:55.283553 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-080faed4-d3fa-423a-85d9-fbc25f20f177 None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:40:55.283827 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-080faed4-d3fa-423a-85d9-fbc25f20f177 None admin] PATH_INFO: `/v3/projects` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:40:55.284496 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-080faed4-d3fa-423a-85d9-fbc25f20f177 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 10:40:55.285971 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-080faed4-d3fa-423a-85d9-fbc25f20f177 None admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 10:40:55.315038 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 26/57] 10.222.0.25 () {60 vars in 1140 bytes} [Sun Aug 7 10:40:55 2022] POST /identity/v3/projects => generated 298 bytes in 170 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 07 10:40:57.330140 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-908dc29e-e56b-42fb-878c-922bd9d51218 None None] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:40:57.330656 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-908dc29e-e56b-42fb-878c-922bd9d51218 None None] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:40:57.330762 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-908dc29e-e56b-42fb-878c-922bd9d51218 None None] PATH_INFO: `/` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:40:57.332066 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 32/58] 10.222.0.25 () {54 vars in 927 bytes} [Sun Aug 7 10:40:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 07 10:40:57.337112 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01171408-78aa-4f34-90d7-86b18721e2cc None None] REQUEST_METHOD: `POST` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:40:57.337493 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01171408-78aa-4f34-90d7-86b18721e2cc None None] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:40:57.337668 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01171408-78aa-4f34-90d7-86b18721e2cc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:40:57.356907 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.auth.core [None req-01171408-78aa-4f34-90d7-86b18721e2cc None None] MFA Rules not processed for user `bac4e6b76fe64217ac8fa44e71cffe6a`. Rule list: `[]` (Enabled: `True`). {{(pid=80143) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 10:40:57.358072 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-01171408-78aa-4f34-90d7-86b18721e2cc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 10:40:57.423252 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 27/59] 10.222.0.25 () {58 vars in 1024 bytes} [Sun Aug 7 10:40:57 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 07 10:40:57.447321 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-908dc29e-e56b-42fb-878c-922bd9d51218 None None] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 10:40:57.544001 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-908dc29e-e56b-42fb-878c-922bd9d51218 None None] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 10:40:57.576501 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7eb00771-db7e-43ca-bbff-f28d3e00769d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 10:40:57.577114 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eb00771-db7e-43ca-bbff-f28d3e00769d None admin] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:40:57.577397 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eb00771-db7e-43ca-bbff-f28d3e00769d None admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:40:57.577669 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eb00771-db7e-43ca-bbff-f28d3e00769d None admin] PATH_INFO: `/v3/domains/default` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:40:57.578762 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7eb00771-db7e-43ca-bbff-f28d3e00769d None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 10:40:57.580251 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7eb00771-db7e-43ca-bbff-f28d3e00769d None admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 10:40:57.581709 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 33/60] 10.222.0.25 () {56 vars in 1108 bytes} [Sun Aug 7 10:40:57 2022] GET /identity/v3/domains/default => generated 199 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 10:40:57.587253 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01171408-78aa-4f34-90d7-86b18721e2cc None None] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 10:40:57.683234 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01171408-78aa-4f34-90d7-86b18721e2cc None None] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 10:40:57.693382 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 10:40:57.716714 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ccd3069-c55e-4a85-b53b-52ddfe8b2841 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 10:40:57.717311 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ccd3069-c55e-4a85-b53b-52ddfe8b2841 None admin] REQUEST_METHOD: `POST` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:40:57.717589 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ccd3069-c55e-4a85-b53b-52ddfe8b2841 None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:40:57.717861 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ccd3069-c55e-4a85-b53b-52ddfe8b2841 None admin] PATH_INFO: `/v3/users` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:40:57.718513 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ccd3069-c55e-4a85-b53b-52ddfe8b2841 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 10:40:57.719982 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ccd3069-c55e-4a85-b53b-52ddfe8b2841 None admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 10:40:57.745127 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 28/61] 10.222.0.25 () {60 vars in 1132 bytes} [Sun Aug 7 10:40:57 2022] POST /identity/v3/users => generated 275 bytes in 158 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 07 10:41:00.791973 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e34c6650-f341-4395-9578-03ee76737f81 None None] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:41:00.793414 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e34c6650-f341-4395-9578-03ee76737f81 None None] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:41:00.793844 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e34c6650-f341-4395-9578-03ee76737f81 None None] PATH_INFO: `/` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:41:00.795393 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 34/62] 10.222.0.25 () {54 vars in 927 bytes} [Sun Aug 7 10:41:00 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 07 10:41:00.801082 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f671a0b9-293f-4c24-9b53-76d76431d6bd None None] REQUEST_METHOD: `POST` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:41:00.801820 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f671a0b9-293f-4c24-9b53-76d76431d6bd None None] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:41:00.802252 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f671a0b9-293f-4c24-9b53-76d76431d6bd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:41:00.827897 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.auth.core [None req-f671a0b9-293f-4c24-9b53-76d76431d6bd None None] MFA Rules not processed for user `bac4e6b76fe64217ac8fa44e71cffe6a`. Rule list: `[]` (Enabled: `True`). {{(pid=80143) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 10:41:00.829361 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-f671a0b9-293f-4c24-9b53-76d76431d6bd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 10:41:00.932637 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 29/63] 10.222.0.25 () {58 vars in 1024 bytes} [Sun Aug 7 10:41:00 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 07 10:41:00.956855 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1703868-a7f1-4502-9f4d-e7532e8e0d93 None None] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:41:00.957368 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1703868-a7f1-4502-9f4d-e7532e8e0d93 None None] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:41:00.957800 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1703868-a7f1-4502-9f4d-e7532e8e0d93 None None] PATH_INFO: `/` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:41:00.959341 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 35/64] 10.222.0.25 () {54 vars in 927 bytes} [Sun Aug 7 10:41:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 07 10:41:00.960485 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f671a0b9-293f-4c24-9b53-76d76431d6bd None None] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 10:41:00.964807 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56b03124-9b0b-492c-a1d3-03f8b4ea77ae None None] REQUEST_METHOD: `POST` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:41:00.965109 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56b03124-9b0b-492c-a1d3-03f8b4ea77ae None None] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:41:00.965398 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56b03124-9b0b-492c-a1d3-03f8b4ea77ae None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:41:00.993815 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.auth.core [None req-56b03124-9b0b-492c-a1d3-03f8b4ea77ae None None] MFA Rules not processed for user `bac4e6b76fe64217ac8fa44e71cffe6a`. Rule list: `[]` (Enabled: `True`). {{(pid=80144) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 10:41:00.995087 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.fernet_utils [None req-56b03124-9b0b-492c-a1d3-03f8b4ea77ae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80144) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 10:41:01.007292 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=100 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 10:41:01.123230 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f671a0b9-293f-4c24-9b53-76d76431d6bd None None] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 10:41:01.127263 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 10:41:01.153939 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 36/65] 10.222.0.25 () {58 vars in 1024 bytes} [Sun Aug 7 10:41:00 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 190 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 07 10:41:01.156413 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-052b95e7-51c4-4bf0-9fe3-a3b101ee7bec None None] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:41:01.156846 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-052b95e7-51c4-4bf0-9fe3-a3b101ee7bec None None] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:41:01.157259 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-052b95e7-51c4-4bf0-9fe3-a3b101ee7bec None None] PATH_INFO: `/` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:41:01.158642 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 37/66] 10.222.0.25 () {54 vars in 927 bytes} [Sun Aug 7 10:41:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 07 10:41:01.166060 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf39aaa6-5020-4367-93ee-2232d1618787 None None] REQUEST_METHOD: `POST` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:41:01.167192 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf39aaa6-5020-4367-93ee-2232d1618787 None None] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:41:01.168824 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf39aaa6-5020-4367-93ee-2232d1618787 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:41:01.181180 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9064745-2b41-4d85-a2c6-668648fba934 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 10:41:01.182028 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9064745-2b41-4d85-a2c6-668648fba934 None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:41:01.182448 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9064745-2b41-4d85-a2c6-668648fba934 None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:41:01.182881 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9064745-2b41-4d85-a2c6-668648fba934 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:41:01.196037 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9064745-2b41-4d85-a2c6-668648fba934 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 07 10:41:01.200156 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.auth.core [None req-bf39aaa6-5020-4367-93ee-2232d1618787 None None] MFA Rules not processed for user `bac4e6b76fe64217ac8fa44e71cffe6a`. Rule list: `[]` (Enabled: `True`). {{(pid=80144) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 10:41:01.201485 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.fernet_utils [None req-bf39aaa6-5020-4367-93ee-2232d1618787 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80144) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 10:41:01.202024 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application [None req-d9064745-2b41-4d85-a2c6-668648fba934 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 07 10:41:01.202024 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 07 10:41:01.202024 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 07 10:41:01.202024 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 07 10:41:01.202024 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 07 10:41:01.202024 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 07 10:41:01.202024 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 07 10:41:01.202024 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 07 10:41:01.202024 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 07 10:41:01.202024 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 07 10:41:01.202024 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 07 10:41:01.202024 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 07 10:41:01.202024 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 07 10:41:01.202024 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 07 10:41:01.202024 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 07 10:41:01.202024 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 07 10:41:01.202899 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 07 10:41:01.202899 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 07 10:41:01.202899 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 07 10:41:01.202899 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 07 10:41:01.202899 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 07 10:41:01.202899 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 07 10:41:01.202899 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 07 10:41:01.202899 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 07 10:41:01.202899 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 07 10:41:01.202899 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.get_or_create( Aug 07 10:41:01.202899 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 07 10:41:01.202899 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application with Lock( Aug 07 10:41:01.202899 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 07 10:41:01.202899 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self._enter() Aug 07 10:41:01.202899 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 07 10:41:01.202899 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 07 10:41:01.203801 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 07 10:41:01.203801 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.creator() Aug 07 10:41:01.203801 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 07 10:41:01.203801 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application created_value = creator( Aug 07 10:41:01.203801 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 07 10:41:01.203801 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 07 10:41:01.203801 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 07 10:41:01.203801 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 07 10:41:01.203801 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 07 10:41:01.203801 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 07 10:41:01.203801 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 07 10:41:01.203801 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application Aug 07 10:41:01.205878 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 30/67] 10.222.0.25 () {56 vars in 1099 bytes} [Sun Aug 7 10:41:00 2022] GET /identity/v3/roles/member => generated 84 bytes in 246 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 07 10:41:01.207151 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9064745-2b41-4d85-a2c6-668648fba934 None admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 10:41:01.298175 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 38/68] 10.222.0.25 () {58 vars in 1024 bytes} [Sun Aug 7 10:41:01 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 07 10:41:01.299400 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf39aaa6-5020-4367-93ee-2232d1618787 None None] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 10:41:01.332724 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9064745-2b41-4d85-a2c6-668648fba934 None admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 10:41:01.373846 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-256a1630-150e-411d-ba3e-c0d337deca0f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 10:41:01.374948 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-256a1630-150e-411d-ba3e-c0d337deca0f None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:41:01.375424 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-256a1630-150e-411d-ba3e-c0d337deca0f None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:41:01.375886 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-256a1630-150e-411d-ba3e-c0d337deca0f None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:41:01.383509 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=100 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 10:41:01.387458 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-256a1630-150e-411d-ba3e-c0d337deca0f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 07 10:41:01.392194 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application [None req-256a1630-150e-411d-ba3e-c0d337deca0f None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 07 10:41:01.392194 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 07 10:41:01.392194 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 07 10:41:01.392194 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 07 10:41:01.392194 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 07 10:41:01.392194 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 07 10:41:01.392194 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 07 10:41:01.392194 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 07 10:41:01.392194 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 07 10:41:01.392194 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 07 10:41:01.392194 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 07 10:41:01.392194 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 07 10:41:01.392194 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 07 10:41:01.392194 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 07 10:41:01.392194 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 07 10:41:01.392194 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 07 10:41:01.393058 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 07 10:41:01.393058 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 07 10:41:01.393058 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 07 10:41:01.393058 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 07 10:41:01.393058 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 07 10:41:01.393058 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 07 10:41:01.393058 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 07 10:41:01.393058 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 07 10:41:01.393058 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 07 10:41:01.393058 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.get_or_create( Aug 07 10:41:01.393058 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 07 10:41:01.393058 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application with Lock( Aug 07 10:41:01.393058 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 07 10:41:01.393058 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self._enter() Aug 07 10:41:01.393058 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 07 10:41:01.393058 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 07 10:41:01.393845 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 07 10:41:01.393845 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.creator() Aug 07 10:41:01.393845 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 07 10:41:01.393845 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application created_value = creator( Aug 07 10:41:01.393845 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 07 10:41:01.393845 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 07 10:41:01.393845 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 07 10:41:01.393845 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 07 10:41:01.393845 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 07 10:41:01.393845 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 07 10:41:01.393845 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 07 10:41:01.393845 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application Aug 07 10:41:01.393845 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 31/69] 10.222.0.25 () {56 vars in 1096 bytes} [Sun Aug 7 10:41:01 2022] GET /identity/v3/roles/admin => generated 83 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 07 10:41:01.394463 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-256a1630-150e-411d-ba3e-c0d337deca0f None admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 10:41:01.414405 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=100 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 10:41:01.425404 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf39aaa6-5020-4367-93ee-2232d1618787 None None] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 10:41:01.462721 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb270975-be77-4569-8247-867cd6d2ae31 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 10:41:01.463376 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb270975-be77-4569-8247-867cd6d2ae31 None admin] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:41:01.463671 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb270975-be77-4569-8247-867cd6d2ae31 None admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:41:01.463954 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb270975-be77-4569-8247-867cd6d2ae31 None admin] PATH_INFO: `/v3/roles` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:41:01.464650 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb270975-be77-4569-8247-867cd6d2ae31 None admin] RBAC: Adding query filter params (name=member) {{(pid=80144) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 10:41:01.465032 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb270975-be77-4569-8247-867cd6d2ae31 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 10:41:01.466407 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb270975-be77-4569-8247-867cd6d2ae31 None admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 10:41:01.474473 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 39/70] 10.222.0.25 () {56 vars in 1101 bytes} [Sun Aug 7 10:41:01 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 10:41:01.475661 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c986f1e-1d87-448b-805f-3a4cb0f9ffb7 None None] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:41:01.475953 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c986f1e-1d87-448b-805f-3a4cb0f9ffb7 None None] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:41:01.476231 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c986f1e-1d87-448b-805f-3a4cb0f9ffb7 None None] PATH_INFO: `/` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:41:01.477597 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 40/71] 10.222.0.25 () {54 vars in 927 bytes} [Sun Aug 7 10:41:01 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 07 10:41:01.479707 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c986f1e-1d87-448b-805f-3a4cb0f9ffb7 None None] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 10:41:01.502623 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-256a1630-150e-411d-ba3e-c0d337deca0f None admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 10:41:01.537171 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6abad30-be09-42a0-8727-11b8b7107b7b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 10:41:01.537808 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6abad30-be09-42a0-8727-11b8b7107b7b None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:41:01.538093 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6abad30-be09-42a0-8727-11b8b7107b7b None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:41:01.538373 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6abad30-be09-42a0-8727-11b8b7107b7b None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:41:01.548701 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6abad30-be09-42a0-8727-11b8b7107b7b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 07 10:41:01.553498 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application [None req-b6abad30-be09-42a0-8727-11b8b7107b7b None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 07 10:41:01.553498 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 07 10:41:01.553498 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 07 10:41:01.553498 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 07 10:41:01.553498 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 07 10:41:01.553498 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 07 10:41:01.553498 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 07 10:41:01.553498 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 07 10:41:01.553498 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 07 10:41:01.553498 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 07 10:41:01.553498 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 07 10:41:01.553498 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 07 10:41:01.553498 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 07 10:41:01.553498 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 07 10:41:01.553498 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 07 10:41:01.553498 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 07 10:41:01.554456 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 07 10:41:01.554456 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 07 10:41:01.554456 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 07 10:41:01.554456 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 07 10:41:01.554456 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 07 10:41:01.554456 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 07 10:41:01.554456 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 07 10:41:01.554456 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 07 10:41:01.554456 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 07 10:41:01.554456 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.get_or_create( Aug 07 10:41:01.554456 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 07 10:41:01.554456 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application with Lock( Aug 07 10:41:01.554456 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 07 10:41:01.554456 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self._enter() Aug 07 10:41:01.554456 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 07 10:41:01.554456 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 07 10:41:01.555247 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 07 10:41:01.555247 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.creator() Aug 07 10:41:01.555247 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 07 10:41:01.555247 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application created_value = creator( Aug 07 10:41:01.555247 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 07 10:41:01.555247 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 07 10:41:01.555247 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 07 10:41:01.555247 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 07 10:41:01.555247 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 07 10:41:01.555247 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 07 10:41:01.555247 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 07 10:41:01.555247 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application Aug 07 10:41:01.555247 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 32/72] 10.222.0.25 () {56 vars in 1114 bytes} [Sun Aug 7 10:41:01 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 07 10:41:01.557583 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97c0ee2f-5873-4022-aaa2-86bb542789f9 None None] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:41:01.558299 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97c0ee2f-5873-4022-aaa2-86bb542789f9 None None] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:41:01.559317 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97c0ee2f-5873-4022-aaa2-86bb542789f9 None None] PATH_INFO: `/` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:41:01.560919 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 33/73] 10.222.0.25 () {54 vars in 927 bytes} [Sun Aug 7 10:41:01 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 07 10:41:01.562077 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97c0ee2f-5873-4022-aaa2-86bb542789f9 None None] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 10:41:01.582536 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c986f1e-1d87-448b-805f-3a4cb0f9ffb7 None None] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 10:41:01.616336 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a5f97a9-2b40-4694-aa64-eca25f14c67e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 10:41:01.616945 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a5f97a9-2b40-4694-aa64-eca25f14c67e None admin] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:41:01.617231 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a5f97a9-2b40-4694-aa64-eca25f14c67e None admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:41:01.617507 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a5f97a9-2b40-4694-aa64-eca25f14c67e None admin] PATH_INFO: `/v3/roles` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:41:01.618179 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a5f97a9-2b40-4694-aa64-eca25f14c67e None admin] RBAC: Adding query filter params (name=admin) {{(pid=80144) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 10:41:01.618488 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a5f97a9-2b40-4694-aa64-eca25f14c67e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 10:41:01.619893 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a5f97a9-2b40-4694-aa64-eca25f14c67e None admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 10:41:01.621790 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=100 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 10:41:01.629365 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 41/74] 10.222.0.25 () {56 vars in 1099 bytes} [Sun Aug 7 10:41:01 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 10:41:01.635027 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8365a91f-4c2a-4522-bb1c-7c1c10887401 None None] REQUEST_METHOD: `POST` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:41:01.635324 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8365a91f-4c2a-4522-bb1c-7c1c10887401 None None] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:41:01.635612 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8365a91f-4c2a-4522-bb1c-7c1c10887401 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:41:01.641330 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=100 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 10:41:01.654045 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.auth.core [None req-8365a91f-4c2a-4522-bb1c-7c1c10887401 None None] MFA Rules not processed for user `bac4e6b76fe64217ac8fa44e71cffe6a`. Rule list: `[]` (Enabled: `True`). {{(pid=80144) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 10:41:01.655277 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.fernet_utils [None req-8365a91f-4c2a-4522-bb1c-7c1c10887401 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80144) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 10:41:01.665862 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97c0ee2f-5873-4022-aaa2-86bb542789f9 None None] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 10:41:01.699964 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79ecd0a5-2af2-480a-97be-44eb224c7fd9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 10:41:01.700721 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79ecd0a5-2af2-480a-97be-44eb224c7fd9 None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:41:01.701013 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79ecd0a5-2af2-480a-97be-44eb224c7fd9 None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:41:01.701298 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79ecd0a5-2af2-480a-97be-44eb224c7fd9 None admin] PATH_INFO: `/v3/users/cfdc7e8b78b84c6aa44d115a396152bd` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:41:01.714391 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79ecd0a5-2af2-480a-97be-44eb224c7fd9 None admin] RBAC: Authorizing `identity:get_user(user_id=cfdc7e8b78b84c6aa44d115a396152bd)` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 10:41:01.715881 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79ecd0a5-2af2-480a-97be-44eb224c7fd9 None admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 10:41:01.717472 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 34/75] 10.222.0.25 () {56 vars in 1177 bytes} [Sun Aug 7 10:41:01 2022] GET /identity/v3/users/cfdc7e8b78b84c6aa44d115a396152bd => generated 275 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 10:41:01.718236 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79ecd0a5-2af2-480a-97be-44eb224c7fd9 None admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 10:41:01.719886 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 42/76] 10.222.0.25 () {58 vars in 1024 bytes} [Sun Aug 7 10:41:01 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 07 10:41:01.721113 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9acb4b1e-2aaf-4d1d-8a40-37d8f74f49c6 None None] REQUEST_METHOD: `POST` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:41:01.721406 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9acb4b1e-2aaf-4d1d-8a40-37d8f74f49c6 None None] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:41:01.721687 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9acb4b1e-2aaf-4d1d-8a40-37d8f74f49c6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:41:01.739784 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.auth.core [None req-9acb4b1e-2aaf-4d1d-8a40-37d8f74f49c6 None None] MFA Rules not processed for user `bac4e6b76fe64217ac8fa44e71cffe6a`. Rule list: `[]` (Enabled: `True`). {{(pid=80144) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 10:41:01.740953 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.fernet_utils [None req-9acb4b1e-2aaf-4d1d-8a40-37d8f74f49c6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80144) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 10:41:01.807575 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 43/77] 10.222.0.25 () {58 vars in 1024 bytes} [Sun Aug 7 10:41:01 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 07 10:41:01.808455 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9acb4b1e-2aaf-4d1d-8a40-37d8f74f49c6 None None] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 10:41:01.832945 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79ecd0a5-2af2-480a-97be-44eb224c7fd9 None admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 10:41:01.863389 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=100 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 10:41:01.866545 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d7df6d5-4a34-4986-b0f3-a8fb4e4f2715 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 10:41:01.867193 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d7df6d5-4a34-4986-b0f3-a8fb4e4f2715 None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:41:01.867480 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d7df6d5-4a34-4986-b0f3-a8fb4e4f2715 None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:41:01.867758 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d7df6d5-4a34-4986-b0f3-a8fb4e4f2715 None admin] PATH_INFO: `/v3/roles` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:41:01.868427 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d7df6d5-4a34-4986-b0f3-a8fb4e4f2715 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80143) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 10:41:01.868736 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d7df6d5-4a34-4986-b0f3-a8fb4e4f2715 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 10:41:01.870117 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d7df6d5-4a34-4986-b0f3-a8fb4e4f2715 None admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 10:41:01.871499 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=100 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 10:41:01.878962 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 35/78] 10.222.0.25 () {56 vars in 1111 bytes} [Sun Aug 7 10:41:01 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 10:41:01.878962 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d7df6d5-4a34-4986-b0f3-a8fb4e4f2715 None admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 10:41:01.908374 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9acb4b1e-2aaf-4d1d-8a40-37d8f74f49c6 None None] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 10:41:01.942677 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e9c0aad-33bf-464a-ace7-241a90240957 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 10:41:01.943332 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e9c0aad-33bf-464a-ace7-241a90240957 None admin] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:41:01.943612 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e9c0aad-33bf-464a-ace7-241a90240957 None admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:41:01.943883 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e9c0aad-33bf-464a-ace7-241a90240957 None admin] PATH_INFO: `/v3/users/bac4e6b76fe64217ac8fa44e71cffe6a` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:41:01.945017 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e9c0aad-33bf-464a-ace7-241a90240957 None admin] RBAC: Authorizing `identity:get_user(user_id=bac4e6b76fe64217ac8fa44e71cffe6a)` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 10:41:01.946426 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e9c0aad-33bf-464a-ace7-241a90240957 None admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 10:41:01.948012 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 44/79] 10.222.0.25 () {56 vars in 1177 bytes} [Sun Aug 7 10:41:01 2022] GET /identity/v3/users/bac4e6b76fe64217ac8fa44e71cffe6a => generated 247 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 10:41:01.948768 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e9c0aad-33bf-464a-ace7-241a90240957 None admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 10:41:01.983688 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d7df6d5-4a34-4986-b0f3-a8fb4e4f2715 None admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 10:41:02.016352 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e8d63b0-0720-4ffa-8f41-cd9c39d7e9eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 10:41:02.016962 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e8d63b0-0720-4ffa-8f41-cd9c39d7e9eb None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:41:02.017252 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e8d63b0-0720-4ffa-8f41-cd9c39d7e9eb None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:41:02.017533 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e8d63b0-0720-4ffa-8f41-cd9c39d7e9eb None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:41:02.047066 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e9c0aad-33bf-464a-ace7-241a90240957 None admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 10:41:02.070489 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=100 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 10:41:02.078941 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=100 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 10:41:02.082197 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-146468b8-cebf-4b9c-9dab-990aeaa0db67 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 10:41:02.082835 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-146468b8-cebf-4b9c-9dab-990aeaa0db67 None admin] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:41:02.083144 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-146468b8-cebf-4b9c-9dab-990aeaa0db67 None admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:41:02.083419 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-146468b8-cebf-4b9c-9dab-990aeaa0db67 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:41:02.084543 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-146468b8-cebf-4b9c-9dab-990aeaa0db67 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 10:41:02.085874 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e8d63b0-0720-4ffa-8f41-cd9c39d7e9eb None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 10:41:02.085994 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-146468b8-cebf-4b9c-9dab-990aeaa0db67 None admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 10:41:02.087371 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e8d63b0-0720-4ffa-8f41-cd9c39d7e9eb None admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 10:41:02.087555 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 45/80] 10.222.0.25 () {56 vars in 1108 bytes} [Sun Aug 7 10:41:01 2022] GET /identity/v3/domains/default => generated 199 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 10:41:02.088317 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-146468b8-cebf-4b9c-9dab-990aeaa0db67 None admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 10:41:02.185311 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 36/81] 10.222.0.25 () {58 vars in 1282 bytes} [Sun Aug 7 10:41:01 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 306 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 07 10:41:02.186074 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e8d63b0-0720-4ffa-8f41-cd9c39d7e9eb None admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 10:41:02.186675 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-146468b8-cebf-4b9c-9dab-990aeaa0db67 None admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 10:41:02.229039 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3391dc1-252e-49d0-991f-b39c82d343c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 10:41:02.229675 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3391dc1-252e-49d0-991f-b39c82d343c9 None admin] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:41:02.229966 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3391dc1-252e-49d0-991f-b39c82d343c9 None admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:41:02.230347 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3391dc1-252e-49d0-991f-b39c82d343c9 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:41:02.241286 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3391dc1-252e-49d0-991f-b39c82d343c9 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 07 10:41:02.247252 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application [None req-e3391dc1-252e-49d0-991f-b39c82d343c9 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 07 10:41:02.247252 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 07 10:41:02.247252 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 07 10:41:02.247252 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 07 10:41:02.247252 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 07 10:41:02.247252 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 07 10:41:02.247252 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 07 10:41:02.247252 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 07 10:41:02.247252 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 07 10:41:02.247252 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 07 10:41:02.247252 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 07 10:41:02.247252 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 07 10:41:02.247252 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 07 10:41:02.247252 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 07 10:41:02.247252 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 07 10:41:02.247252 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 07 10:41:02.248276 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 07 10:41:02.248276 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 07 10:41:02.248276 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 07 10:41:02.248276 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 07 10:41:02.248276 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 07 10:41:02.248276 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 07 10:41:02.248276 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 07 10:41:02.248276 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 07 10:41:02.248276 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 07 10:41:02.248276 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self.get_or_create( Aug 07 10:41:02.248276 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 07 10:41:02.248276 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application with Lock( Aug 07 10:41:02.248276 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 07 10:41:02.248276 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self._enter() Aug 07 10:41:02.248276 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 07 10:41:02.248276 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 07 10:41:02.249175 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 07 10:41:02.249175 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self.creator() Aug 07 10:41:02.249175 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 07 10:41:02.249175 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application created_value = creator( Aug 07 10:41:02.249175 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 07 10:41:02.249175 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 07 10:41:02.249175 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 07 10:41:02.249175 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 07 10:41:02.249175 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 07 10:41:02.249175 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 07 10:41:02.249175 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 07 10:41:02.249175 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application Aug 07 10:41:02.249175 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 46/82] 10.222.0.25 () {56 vars in 1099 bytes} [Sun Aug 7 10:41:02 2022] GET /identity/v3/roles/member => generated 84 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 07 10:41:02.249940 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3391dc1-252e-49d0-991f-b39c82d343c9 None admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 10:41:02.283002 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=100 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 10:41:02.289057 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e8d63b0-0720-4ffa-8f41-cd9c39d7e9eb None admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 10:41:02.303492 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=100 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 10:41:02.324041 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91904bf9-f897-4721-9b3e-0c0f5c4f4760 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 10:41:02.324739 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91904bf9-f897-4721-9b3e-0c0f5c4f4760 None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:41:02.325024 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91904bf9-f897-4721-9b3e-0c0f5c4f4760 None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:41:02.325303 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91904bf9-f897-4721-9b3e-0c0f5c4f4760 None admin] PATH_INFO: `/v3/users/cfdc7e8b78b84c6aa44d115a396152bd` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:41:02.326438 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91904bf9-f897-4721-9b3e-0c0f5c4f4760 None admin] RBAC: Authorizing `identity:get_user(user_id=cfdc7e8b78b84c6aa44d115a396152bd)` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 10:41:02.327919 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91904bf9-f897-4721-9b3e-0c0f5c4f4760 None admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 10:41:02.329480 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 37/83] 10.222.0.25 () {56 vars in 1177 bytes} [Sun Aug 7 10:41:02 2022] GET /identity/v3/users/cfdc7e8b78b84c6aa44d115a396152bd => generated 275 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 10:41:02.330219 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91904bf9-f897-4721-9b3e-0c0f5c4f4760 None admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 10:41:02.352911 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3391dc1-252e-49d0-991f-b39c82d343c9 None admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 10:41:02.387147 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42a038c1-c907-4db6-bac1-59abb6bf88ba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 10:41:02.387744 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42a038c1-c907-4db6-bac1-59abb6bf88ba None admin] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:41:02.388026 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42a038c1-c907-4db6-bac1-59abb6bf88ba None admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:41:02.388293 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42a038c1-c907-4db6-bac1-59abb6bf88ba None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:41:02.431656 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91904bf9-f897-4721-9b3e-0c0f5c4f4760 None admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 10:41:02.456995 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42a038c1-c907-4db6-bac1-59abb6bf88ba None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 10:41:02.458402 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42a038c1-c907-4db6-bac1-59abb6bf88ba None admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 10:41:02.464420 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fef6f9d9-9888-48e9-bde6-02def23cbbc3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 10:41:02.465076 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fef6f9d9-9888-48e9-bde6-02def23cbbc3 None admin] REQUEST_METHOD: `POST` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:41:02.465358 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fef6f9d9-9888-48e9-bde6-02def23cbbc3 None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:41:02.465632 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fef6f9d9-9888-48e9-bde6-02def23cbbc3 None admin] PATH_INFO: `/v3/users` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:41:02.466274 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fef6f9d9-9888-48e9-bde6-02def23cbbc3 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 10:41:02.467721 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fef6f9d9-9888-48e9-bde6-02def23cbbc3 None admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 10:41:02.488437 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 38/84] 10.222.0.25 () {60 vars in 1132 bytes} [Sun Aug 7 10:41:02 2022] POST /identity/v3/users => generated 289 bytes in 158 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 07 10:41:02.489237 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fef6f9d9-9888-48e9-bde6-02def23cbbc3 None admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 10:41:02.510140 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=100 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 10:41:02.532598 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 10:41:02.576615 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 47/85] 10.222.0.25 () {58 vars in 1282 bytes} [Sun Aug 7 10:41:02 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 328 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 07 10:41:02.577632 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42a038c1-c907-4db6-bac1-59abb6bf88ba None admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 10:41:02.628382 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fef6f9d9-9888-48e9-bde6-02def23cbbc3 None admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 10:41:02.673416 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0d88b64-79ab-478c-8744-8c77435d4b76 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 10:41:02.674392 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0d88b64-79ab-478c-8744-8c77435d4b76 None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:41:02.674745 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0d88b64-79ab-478c-8744-8c77435d4b76 None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:41:02.675294 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0d88b64-79ab-478c-8744-8c77435d4b76 None admin] PATH_INFO: `/v3/projects/90718391b09e493bb296767dc8a3c0ce` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:41:02.676913 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0d88b64-79ab-478c-8744-8c77435d4b76 None admin] RBAC: Authorizing `identity:get_project(project_id=90718391b09e493bb296767dc8a3c0ce)` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 10:41:02.677764 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0d88b64-79ab-478c-8744-8c77435d4b76 None admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 10:41:02.679538 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 39/86] 10.222.0.25 () {56 vars in 1186 bytes} [Sun Aug 7 10:41:02 2022] GET /identity/v3/projects/90718391b09e493bb296767dc8a3c0ce => generated 298 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 10:41:02.680169 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0d88b64-79ab-478c-8744-8c77435d4b76 None admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 10:41:02.699914 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42a038c1-c907-4db6-bac1-59abb6bf88ba None admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 10:41:02.748573 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2699c8f5-4457-42a4-97f2-095c88998aaa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 10:41:02.749519 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2699c8f5-4457-42a4-97f2-095c88998aaa None admin] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:41:02.750010 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2699c8f5-4457-42a4-97f2-095c88998aaa None admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:41:02.750476 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2699c8f5-4457-42a4-97f2-095c88998aaa None admin] PATH_INFO: `/v3/roles` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:41:02.751398 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2699c8f5-4457-42a4-97f2-095c88998aaa None admin] RBAC: Adding query filter params (name=member) {{(pid=80144) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 10:41:02.752046 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2699c8f5-4457-42a4-97f2-095c88998aaa None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 10:41:02.753678 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2699c8f5-4457-42a4-97f2-095c88998aaa None admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 10:41:02.765701 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 48/87] 10.222.0.25 () {56 vars in 1101 bytes} [Sun Aug 7 10:41:02 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 10:41:02.767443 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2699c8f5-4457-42a4-97f2-095c88998aaa None admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 10:41:02.788071 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=100 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 10:41:02.800150 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=100 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 10:41:02.818396 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0d88b64-79ab-478c-8744-8c77435d4b76 None admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 10:41:02.860776 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbece9bb-33ed-4aeb-acf7-8be337351ce4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 10:41:02.861415 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbece9bb-33ed-4aeb-acf7-8be337351ce4 None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:41:02.861937 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbece9bb-33ed-4aeb-acf7-8be337351ce4 None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:41:02.862221 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbece9bb-33ed-4aeb-acf7-8be337351ce4 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:41:02.887672 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2699c8f5-4457-42a4-97f2-095c88998aaa None admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 10:41:02.924529 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b8efcd6-873a-4b0f-beee-0c3f360c2803 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 10:41:02.925168 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b8efcd6-873a-4b0f-beee-0c3f360c2803 None admin] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:41:02.925452 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b8efcd6-873a-4b0f-beee-0c3f360c2803 None admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:41:02.925728 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b8efcd6-873a-4b0f-beee-0c3f360c2803 None admin] PATH_INFO: `/v3/projects/90718391b09e493bb296767dc8a3c0ce` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:41:02.926858 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b8efcd6-873a-4b0f-beee-0c3f360c2803 None admin] RBAC: Authorizing `identity:get_project(project_id=90718391b09e493bb296767dc8a3c0ce)` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 10:41:02.928332 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b8efcd6-873a-4b0f-beee-0c3f360c2803 None admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 10:41:02.930160 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 49/88] 10.222.0.25 () {56 vars in 1186 bytes} [Sun Aug 7 10:41:02 2022] GET /identity/v3/projects/90718391b09e493bb296767dc8a3c0ce => generated 298 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 10:41:02.930707 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b8efcd6-873a-4b0f-beee-0c3f360c2803 None admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 10:41:02.933953 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbece9bb-33ed-4aeb-acf7-8be337351ce4 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 10:41:02.935460 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbece9bb-33ed-4aeb-acf7-8be337351ce4 None admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 10:41:03.021008 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=100 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 10:41:03.040668 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 40/89] 10.222.0.25 () {58 vars in 1282 bytes} [Sun Aug 7 10:41:02 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 361 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 07 10:41:03.041550 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbece9bb-33ed-4aeb-acf7-8be337351ce4 None admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 10:41:03.042417 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b8efcd6-873a-4b0f-beee-0c3f360c2803 None admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 10:41:03.045731 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=100 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 10:41:03.082030 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2bc323d-10ce-4d7b-99e5-e5e37acb6fb6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 10:41:03.082643 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2bc323d-10ce-4d7b-99e5-e5e37acb6fb6 None admin] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:41:03.082969 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2bc323d-10ce-4d7b-99e5-e5e37acb6fb6 None admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:41:03.083224 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2bc323d-10ce-4d7b-99e5-e5e37acb6fb6 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:41:03.083948 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2bc323d-10ce-4d7b-99e5-e5e37acb6fb6 None admin] RBAC: Adding query filter params (role.id=b98dfb3b9fee4644af2887a3ac3fd2e6, scope.project.id=90718391b09e493bb296767dc8a3c0ce, user.id=cfdc7e8b78b84c6aa44d115a396152bd) {{(pid=80144) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 10:41:03.084260 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2bc323d-10ce-4d7b-99e5-e5e37acb6fb6 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 10:41:03.085686 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2bc323d-10ce-4d7b-99e5-e5e37acb6fb6 None admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 10:41:03.091381 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 50/90] 10.222.0.25 () {56 vars in 1374 bytes} [Sun Aug 7 10:41:02 2022] GET /identity/v3/role_assignments?user.id=cfdc7e8b78b84c6aa44d115a396152bd&scope.project.id=90718391b09e493bb296767dc8a3c0ce&role.id=b98dfb3b9fee4644af2887a3ac3fd2e6 => generated 259 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 10:41:03.098121 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2bc323d-10ce-4d7b-99e5-e5e37acb6fb6 None admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 10:41:03.203878 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2bc323d-10ce-4d7b-99e5-e5e37acb6fb6 None admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 10:41:03.243205 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a4db901-b4ab-487f-8174-4e349a93aec1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 10:41:03.243828 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a4db901-b4ab-487f-8174-4e349a93aec1 None admin] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:41:03.244111 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a4db901-b4ab-487f-8174-4e349a93aec1 None admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:41:03.244388 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a4db901-b4ab-487f-8174-4e349a93aec1 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:41:03.245131 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a4db901-b4ab-487f-8174-4e349a93aec1 None admin] RBAC: Adding query filter params (role.id=81b1f1bcee2a473088fbbd327c1c5b23, scope.project.id=90718391b09e493bb296767dc8a3c0ce, user.id=bac4e6b76fe64217ac8fa44e71cffe6a) {{(pid=80144) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 10:41:03.245445 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a4db901-b4ab-487f-8174-4e349a93aec1 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 10:41:03.247371 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a4db901-b4ab-487f-8174-4e349a93aec1 None admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 10:41:03.252179 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbece9bb-33ed-4aeb-acf7-8be337351ce4 None admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 10:41:03.266299 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 51/91] 10.222.0.25 () {56 vars in 1374 bytes} [Sun Aug 7 10:41:03 2022] GET /identity/v3/role_assignments?user.id=bac4e6b76fe64217ac8fa44e71cffe6a&scope.project.id=90718391b09e493bb296767dc8a3c0ce&role.id=81b1f1bcee2a473088fbbd327c1c5b23 => generated 259 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 10:41:03.267096 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a4db901-b4ab-487f-8174-4e349a93aec1 None admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 10:41:03.291431 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=100 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 10:41:03.324431 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a999d97b-32b5-4b71-a44e-d3ea9a27189a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 10:41:03.326339 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a999d97b-32b5-4b71-a44e-d3ea9a27189a None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:41:03.328057 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a999d97b-32b5-4b71-a44e-d3ea9a27189a None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:41:03.328507 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a999d97b-32b5-4b71-a44e-d3ea9a27189a None admin] PATH_INFO: `/v3/users/cfdc7e8b78b84c6aa44d115a396152bd` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:41:03.331144 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a999d97b-32b5-4b71-a44e-d3ea9a27189a None admin] RBAC: Authorizing `identity:get_user(user_id=cfdc7e8b78b84c6aa44d115a396152bd)` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 10:41:03.333783 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a999d97b-32b5-4b71-a44e-d3ea9a27189a None admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 10:41:03.337858 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 41/92] 10.222.0.25 () {56 vars in 1177 bytes} [Sun Aug 7 10:41:03 2022] GET /identity/v3/users/cfdc7e8b78b84c6aa44d115a396152bd => generated 275 bytes in 297 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 10:41:03.344407 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a999d97b-32b5-4b71-a44e-d3ea9a27189a None admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 10:41:03.418560 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=100 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 10:41:03.450951 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a4db901-b4ab-487f-8174-4e349a93aec1 None admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 10:41:03.481842 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4bc5f34-3dd2-4139-8c73-a574c0cf42b2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 10:41:03.482428 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4bc5f34-3dd2-4139-8c73-a574c0cf42b2 None admin] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:41:03.482710 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4bc5f34-3dd2-4139-8c73-a574c0cf42b2 None admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:41:03.483015 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4bc5f34-3dd2-4139-8c73-a574c0cf42b2 None admin] PATH_INFO: `/v3/projects/90718391b09e493bb296767dc8a3c0ce` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:41:03.484179 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4bc5f34-3dd2-4139-8c73-a574c0cf42b2 None admin] RBAC: Authorizing `identity:get_project(project_id=90718391b09e493bb296767dc8a3c0ce)` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 10:41:03.485617 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4bc5f34-3dd2-4139-8c73-a574c0cf42b2 None admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 10:41:03.494948 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 52/93] 10.222.0.25 () {56 vars in 1186 bytes} [Sun Aug 7 10:41:03 2022] GET /identity/v3/projects/90718391b09e493bb296767dc8a3c0ce => generated 298 bytes in 221 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 10:41:03.501386 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a999d97b-32b5-4b71-a44e-d3ea9a27189a None admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 10:41:03.503056 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4bc5f34-3dd2-4139-8c73-a574c0cf42b2 None admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 10:41:03.543163 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b13deef-faee-4104-a55b-749ae831b33f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 10:41:03.543163 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b13deef-faee-4104-a55b-749ae831b33f None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:41:03.543429 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b13deef-faee-4104-a55b-749ae831b33f None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:41:03.543785 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b13deef-faee-4104-a55b-749ae831b33f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:41:03.622799 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b13deef-faee-4104-a55b-749ae831b33f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 10:41:03.624454 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b13deef-faee-4104-a55b-749ae831b33f None admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 10:41:03.635542 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=100 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 10:41:03.666761 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4bc5f34-3dd2-4139-8c73-a574c0cf42b2 None admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 10:41:03.688313 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=100 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 10:41:03.727640 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ecb6cec-f63f-4ec3-a041-ef3b2c7cdd2b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 10:41:03.728493 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ecb6cec-f63f-4ec3-a041-ef3b2c7cdd2b None admin] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:41:03.728935 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ecb6cec-f63f-4ec3-a041-ef3b2c7cdd2b None admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:41:03.729340 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ecb6cec-f63f-4ec3-a041-ef3b2c7cdd2b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:41:03.730266 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ecb6cec-f63f-4ec3-a041-ef3b2c7cdd2b None admin] RBAC: Adding query filter params (role.id=1512d531ebe248ea8e2d7d56d754b230, scope.project.id=90718391b09e493bb296767dc8a3c0ce, user.id=cfdc7e8b78b84c6aa44d115a396152bd) {{(pid=80144) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 10:41:03.730719 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ecb6cec-f63f-4ec3-a041-ef3b2c7cdd2b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 10:41:03.732679 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ecb6cec-f63f-4ec3-a041-ef3b2c7cdd2b None admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 10:41:03.738852 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 53/94] 10.222.0.25 () {56 vars in 1374 bytes} [Sun Aug 7 10:41:03 2022] GET /identity/v3/role_assignments?user.id=cfdc7e8b78b84c6aa44d115a396152bd&scope.project.id=90718391b09e493bb296767dc8a3c0ce&role.id=1512d531ebe248ea8e2d7d56d754b230 => generated 259 bytes in 236 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 07 10:41:03.744290 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 42/95] 10.222.0.25 () {58 vars in 1282 bytes} [Sun Aug 7 10:41:03 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 400 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Aug 07 10:41:03.750606 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ecb6cec-f63f-4ec3-a041-ef3b2c7cdd2b None admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 10:41:03.893469 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ecb6cec-f63f-4ec3-a041-ef3b2c7cdd2b None admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 10:41:03.939764 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fadb972b-da0e-432c-a9ad-f34f23021033 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 10:41:03.940681 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fadb972b-da0e-432c-a9ad-f34f23021033 None admin] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:41:03.941105 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fadb972b-da0e-432c-a9ad-f34f23021033 None admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:41:03.941518 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fadb972b-da0e-432c-a9ad-f34f23021033 None admin] PATH_INFO: `/v3/projects/6974c88728de4be899fbc38155f70bb9` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:41:03.942806 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fadb972b-da0e-432c-a9ad-f34f23021033 None admin] RBAC: Authorizing `identity:get_project(project_id=6974c88728de4be899fbc38155f70bb9)` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 10:41:03.944434 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fadb972b-da0e-432c-a9ad-f34f23021033 None admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 10:41:03.946241 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 54/96] 10.222.0.25 () {56 vars in 1186 bytes} [Sun Aug 7 10:41:03 2022] GET /identity/v3/projects/6974c88728de4be899fbc38155f70bb9 => generated 312 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 10:41:03.953229 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b13deef-faee-4104-a55b-749ae831b33f None admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 10:41:04.068395 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b13deef-faee-4104-a55b-749ae831b33f None admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 10:41:04.106516 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdc0566f-fb08-4b9f-8f80-3a99f44dded2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 10:41:04.107172 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdc0566f-fb08-4b9f-8f80-3a99f44dded2 None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:41:04.107464 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdc0566f-fb08-4b9f-8f80-3a99f44dded2 None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:41:04.107747 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdc0566f-fb08-4b9f-8f80-3a99f44dded2 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:41:04.108556 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdc0566f-fb08-4b9f-8f80-3a99f44dded2 None admin] RBAC: Adding query filter params (role.id=b98dfb3b9fee4644af2887a3ac3fd2e6, scope.project.id=6974c88728de4be899fbc38155f70bb9, user.id=cfdc7e8b78b84c6aa44d115a396152bd) {{(pid=80143) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 10:41:04.108881 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdc0566f-fb08-4b9f-8f80-3a99f44dded2 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 10:41:04.110328 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdc0566f-fb08-4b9f-8f80-3a99f44dded2 None admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 10:41:04.121619 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 43/97] 10.222.0.25 () {56 vars in 1374 bytes} [Sun Aug 7 10:41:03 2022] GET /identity/v3/role_assignments?user.id=cfdc7e8b78b84c6aa44d115a396152bd&scope.project.id=6974c88728de4be899fbc38155f70bb9&role.id=b98dfb3b9fee4644af2887a3ac3fd2e6 => generated 259 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 10:41:04.766899 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a455733-702a-4cb1-9ed9-140b918823b1 None None] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:41:04.767281 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a455733-702a-4cb1-9ed9-140b918823b1 None None] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:41:04.767533 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a455733-702a-4cb1-9ed9-140b918823b1 None None] PATH_INFO: `/` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:41:04.768992 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 55/98] 10.222.0.25 () {54 vars in 927 bytes} [Sun Aug 7 10:41:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 07 10:41:04.774052 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5c89c4c-1d48-4c2d-b217-f76511475b01 None None] REQUEST_METHOD: `POST` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:41:04.774347 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5c89c4c-1d48-4c2d-b217-f76511475b01 None None] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:41:04.774627 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5c89c4c-1d48-4c2d-b217-f76511475b01 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:41:04.793944 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.auth.core [None req-d5c89c4c-1d48-4c2d-b217-f76511475b01 None None] MFA Rules not processed for user `bac4e6b76fe64217ac8fa44e71cffe6a`. Rule list: `[]` (Enabled: `True`). {{(pid=80143) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 10:41:04.799306 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-d5c89c4c-1d48-4c2d-b217-f76511475b01 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 10:41:04.819413 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=100 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 10:41:04.889667 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 44/99] 10.222.0.25 () {58 vars in 1024 bytes} [Sun Aug 7 10:41:04 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 07 10:41:04.935830 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a455733-702a-4cb1-9ed9-140b918823b1 None None] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 10:41:04.964992 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=100 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 10:41:05.085443 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a455733-702a-4cb1-9ed9-140b918823b1 None None] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 10:41:05.140094 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aecf0163-a6fe-4a11-81d1-3e461eac584e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 10:41:05.140748 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aecf0163-a6fe-4a11-81d1-3e461eac584e None admin] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:41:05.141037 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aecf0163-a6fe-4a11-81d1-3e461eac584e None admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:41:05.141315 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aecf0163-a6fe-4a11-81d1-3e461eac584e None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:41:05.164722 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aecf0163-a6fe-4a11-81d1-3e461eac584e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 07 10:41:05.170607 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application [None req-aecf0163-a6fe-4a11-81d1-3e461eac584e None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Aug 07 10:41:05.170607 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 07 10:41:05.170607 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 07 10:41:05.170607 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 07 10:41:05.170607 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 07 10:41:05.170607 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 07 10:41:05.170607 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 07 10:41:05.170607 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 07 10:41:05.170607 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 07 10:41:05.170607 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 07 10:41:05.170607 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 07 10:41:05.170607 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 07 10:41:05.170607 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 07 10:41:05.170607 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 07 10:41:05.170607 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 07 10:41:05.170607 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 07 10:41:05.171633 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 07 10:41:05.171633 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 07 10:41:05.171633 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 07 10:41:05.171633 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 07 10:41:05.171633 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 07 10:41:05.171633 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 07 10:41:05.171633 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 07 10:41:05.171633 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 07 10:41:05.171633 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 07 10:41:05.171633 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self.get_or_create( Aug 07 10:41:05.171633 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 07 10:41:05.171633 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application with Lock( Aug 07 10:41:05.171633 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 07 10:41:05.171633 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self._enter() Aug 07 10:41:05.171633 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 07 10:41:05.171633 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 07 10:41:05.172394 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 07 10:41:05.172394 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self.creator() Aug 07 10:41:05.172394 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 07 10:41:05.172394 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application created_value = creator( Aug 07 10:41:05.172394 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 07 10:41:05.172394 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 07 10:41:05.172394 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 07 10:41:05.172394 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 07 10:41:05.172394 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 07 10:41:05.172394 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 07 10:41:05.172394 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Aug 07 10:41:05.172394 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application Aug 07 10:41:05.172394 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 56/100] 10.222.0.25 () {56 vars in 1099 bytes} [Sun Aug 7 10:41:04 2022] GET /identity/v3/roles/reader => generated 84 bytes in 237 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 07 10:41:05.180590 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5c89c4c-1d48-4c2d-b217-f76511475b01 None None] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 10:41:05.349932 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5c89c4c-1d48-4c2d-b217-f76511475b01 None None] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 10:41:05.402365 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd077cca-71ad-44e5-b7ea-3ca26da2b173 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 10:41:05.403239 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd077cca-71ad-44e5-b7ea-3ca26da2b173 None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:41:05.403531 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd077cca-71ad-44e5-b7ea-3ca26da2b173 None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:41:05.403814 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd077cca-71ad-44e5-b7ea-3ca26da2b173 None admin] PATH_INFO: `/v3/roles` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:41:05.404651 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd077cca-71ad-44e5-b7ea-3ca26da2b173 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80143) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 10:41:05.405223 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd077cca-71ad-44e5-b7ea-3ca26da2b173 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 10:41:05.406891 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd077cca-71ad-44e5-b7ea-3ca26da2b173 None admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 10:41:05.424163 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 45/101] 10.222.0.25 () {56 vars in 1101 bytes} [Sun Aug 7 10:41:05 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 243 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 10:41:05.430527 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aecf0163-a6fe-4a11-81d1-3e461eac584e None admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 10:41:05.554099 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=100 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 10:41:05.587934 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aecf0163-a6fe-4a11-81d1-3e461eac584e None admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 10:41:05.645196 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94bba5a5-1a20-44ac-82f9-16b618088238 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 10:41:05.645740 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94bba5a5-1a20-44ac-82f9-16b618088238 None admin] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:41:05.646028 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94bba5a5-1a20-44ac-82f9-16b618088238 None admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:41:05.646308 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94bba5a5-1a20-44ac-82f9-16b618088238 None admin] PATH_INFO: `/v3/users/b1626c83857e4f2091030226ccd3ca7e` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:41:05.675170 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94bba5a5-1a20-44ac-82f9-16b618088238 None admin] RBAC: Authorizing `identity:get_user(user_id=b1626c83857e4f2091030226ccd3ca7e)` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 10:41:05.676650 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94bba5a5-1a20-44ac-82f9-16b618088238 None admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 10:41:05.678606 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 57/102] 10.222.0.25 () {56 vars in 1177 bytes} [Sun Aug 7 10:41:05 2022] GET /identity/v3/users/b1626c83857e4f2091030226ccd3ca7e => generated 289 bytes in 249 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 10:41:05.686634 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd077cca-71ad-44e5-b7ea-3ca26da2b173 None admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 10:41:05.703328 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b895ef6-34b1-4e3b-8cad-83bf787d418d None None] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:41:05.703729 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b895ef6-34b1-4e3b-8cad-83bf787d418d None None] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:41:05.704075 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b895ef6-34b1-4e3b-8cad-83bf787d418d None None] PATH_INFO: `/` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:41:05.705426 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 58/103] 10.222.0.25 () {54 vars in 927 bytes} [Sun Aug 7 10:41:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 07 10:41:05.712538 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eba91ba0-9e63-4ae9-a834-a7d60279891c None None] REQUEST_METHOD: `POST` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:41:05.713070 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eba91ba0-9e63-4ae9-a834-a7d60279891c None None] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:41:05.713648 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eba91ba0-9e63-4ae9-a834-a7d60279891c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:41:05.715432 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=100 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 10:41:05.768712 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.auth.core [None req-eba91ba0-9e63-4ae9-a834-a7d60279891c None None] MFA Rules not processed for user `bac4e6b76fe64217ac8fa44e71cffe6a`. Rule list: `[]` (Enabled: `True`). {{(pid=80144) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 10:41:05.769972 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.fernet_utils [None req-eba91ba0-9e63-4ae9-a834-a7d60279891c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80144) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 10:41:05.942994 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd077cca-71ad-44e5-b7ea-3ca26da2b173 None admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 10:41:05.952865 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 59/104] 10.222.0.25 () {58 vars in 1024 bytes} [Sun Aug 7 10:41:05 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 242 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 07 10:41:06.000321 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50b15ba6-320e-44ae-be97-0ec90869f833 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 10:41:06.001057 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50b15ba6-320e-44ae-be97-0ec90869f833 None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:41:06.001351 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50b15ba6-320e-44ae-be97-0ec90869f833 None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:41:06.001628 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50b15ba6-320e-44ae-be97-0ec90869f833 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:41:06.002898 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eba91ba0-9e63-4ae9-a834-a7d60279891c None None] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 10:41:06.109656 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50b15ba6-320e-44ae-be97-0ec90869f833 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 10:41:06.111191 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50b15ba6-320e-44ae-be97-0ec90869f833 None admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 10:41:06.127829 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=100 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 10:41:06.140246 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=100 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 10:41:06.241146 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eba91ba0-9e63-4ae9-a834-a7d60279891c None None] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 10:41:06.323624 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceac6231-b248-4d50-a4a6-6d073cd94a83 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 10:41:06.324624 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceac6231-b248-4d50-a4a6-6d073cd94a83 None admin] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:41:06.325232 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceac6231-b248-4d50-a4a6-6d073cd94a83 None admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:41:06.325710 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceac6231-b248-4d50-a4a6-6d073cd94a83 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:41:06.327146 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ceac6231-b248-4d50-a4a6-6d073cd94a83 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 10:41:06.328793 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ceac6231-b248-4d50-a4a6-6d073cd94a83 None admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 10:41:06.330608 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 60/105] 10.222.0.25 () {56 vars in 1108 bytes} [Sun Aug 7 10:41:06 2022] GET /identity/v3/domains/default => generated 199 bytes in 328 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 07 10:41:06.337003 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceac6231-b248-4d50-a4a6-6d073cd94a83 None admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 10:41:06.363017 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 46/106] 10.222.0.25 () {58 vars in 1282 bytes} [Sun Aug 7 10:41:05 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 677 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 07 10:41:06.368506 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50b15ba6-320e-44ae-be97-0ec90869f833 None admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 10:41:06.530641 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50b15ba6-320e-44ae-be97-0ec90869f833 None admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 10:41:06.558406 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=100 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 10:41:06.575384 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7394fef7-ed8e-4c0b-9d44-031874cf122f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 10:41:06.576042 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7394fef7-ed8e-4c0b-9d44-031874cf122f None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:41:06.576473 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7394fef7-ed8e-4c0b-9d44-031874cf122f None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:41:06.576763 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7394fef7-ed8e-4c0b-9d44-031874cf122f None admin] PATH_INFO: `/v3/projects/90718391b09e493bb296767dc8a3c0ce` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:41:06.578515 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7394fef7-ed8e-4c0b-9d44-031874cf122f None admin] RBAC: Authorizing `identity:get_project(project_id=90718391b09e493bb296767dc8a3c0ce)` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 10:41:06.580671 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7394fef7-ed8e-4c0b-9d44-031874cf122f None admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 10:41:06.582949 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 47/107] 10.222.0.25 () {56 vars in 1186 bytes} [Sun Aug 7 10:41:06 2022] GET /identity/v3/projects/90718391b09e493bb296767dc8a3c0ce => generated 298 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 07 10:41:06.583127 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceac6231-b248-4d50-a4a6-6d073cd94a83 None admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 10:41:06.584249 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-025001f0-f83c-4ceb-8854-5ba1938d989e None None] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:41:06.584563 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-025001f0-f83c-4ceb-8854-5ba1938d989e None None] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:41:06.585080 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-025001f0-f83c-4ceb-8854-5ba1938d989e None None] PATH_INFO: `/` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:41:06.587709 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 48/108] 10.222.0.25 () {54 vars in 927 bytes} [Sun Aug 7 10:41:06 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 07 10:41:06.588908 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28728520-2494-48c0-a522-708a8c7e0290 None None] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:41:06.589197 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28728520-2494-48c0-a522-708a8c7e0290 None None] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:41:06.590028 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28728520-2494-48c0-a522-708a8c7e0290 None None] PATH_INFO: `/` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:41:06.591476 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 49/109] 10.222.0.25 () {54 vars in 927 bytes} [Sun Aug 7 10:41:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 07 10:41:06.593158 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28728520-2494-48c0-a522-708a8c7e0290 None None] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 10:41:06.619824 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=100 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 10:41:06.631853 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e4cf99d-7167-4082-835b-8b3d0291c6a5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 10:41:06.632478 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e4cf99d-7167-4082-835b-8b3d0291c6a5 None admin] REQUEST_METHOD: `POST` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:41:06.632766 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e4cf99d-7167-4082-835b-8b3d0291c6a5 None admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:41:06.633048 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e4cf99d-7167-4082-835b-8b3d0291c6a5 None admin] PATH_INFO: `/v3/projects` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:41:06.633886 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e4cf99d-7167-4082-835b-8b3d0291c6a5 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 10:41:06.635963 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e4cf99d-7167-4082-835b-8b3d0291c6a5 None admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 10:41:06.679189 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 61/110] 10.222.0.25 () {60 vars in 1141 bytes} [Sun Aug 7 10:41:06 2022] POST /identity/v3/projects => generated 302 bytes in 336 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 07 10:41:06.681649 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c93d1b0c-aab6-4d87-88d9-3e159b54e181 None None] REQUEST_METHOD: `POST` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:41:06.681944 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c93d1b0c-aab6-4d87-88d9-3e159b54e181 None None] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:41:06.682225 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c93d1b0c-aab6-4d87-88d9-3e159b54e181 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:41:06.724420 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.auth.core [None req-c93d1b0c-aab6-4d87-88d9-3e159b54e181 None None] MFA Rules not processed for user `bac4e6b76fe64217ac8fa44e71cffe6a`. Rule list: `[]` (Enabled: `True`). {{(pid=80144) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 10:41:06.727371 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.fernet_utils [None req-c93d1b0c-aab6-4d87-88d9-3e159b54e181 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80144) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 10:41:06.763507 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28728520-2494-48c0-a522-708a8c7e0290 None None] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 10:41:06.847665 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-621eab99-e3d8-42c9-9f86-98783f747ddf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 10:41:06.850818 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-621eab99-e3d8-42c9-9f86-98783f747ddf None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:41:06.851194 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-621eab99-e3d8-42c9-9f86-98783f747ddf None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:41:06.851478 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-621eab99-e3d8-42c9-9f86-98783f747ddf None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:41:06.852230 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-621eab99-e3d8-42c9-9f86-98783f747ddf None admin] RBAC: Adding query filter params (role.id=567b9b8669874c9986d68642dc9ef845, scope.project.id=90718391b09e493bb296767dc8a3c0ce, user.id=b1626c83857e4f2091030226ccd3ca7e) {{(pid=80143) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 10:41:06.853916 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-621eab99-e3d8-42c9-9f86-98783f747ddf None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 10:41:06.857263 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-621eab99-e3d8-42c9-9f86-98783f747ddf None admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 10:41:06.866807 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 50/111] 10.222.0.25 () {56 vars in 1374 bytes} [Sun Aug 7 10:41:06 2022] GET /identity/v3/role_assignments?user.id=b1626c83857e4f2091030226ccd3ca7e&scope.project.id=90718391b09e493bb296767dc8a3c0ce&role.id=567b9b8669874c9986d68642dc9ef845 => generated 259 bytes in 275 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 10:41:06.867846 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62bf8b90-6ec0-44bf-bb32-aae67e78e2c4 None None] REQUEST_METHOD: `POST` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:41:06.868139 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62bf8b90-6ec0-44bf-bb32-aae67e78e2c4 None None] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:41:06.868414 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62bf8b90-6ec0-44bf-bb32-aae67e78e2c4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:41:06.883018 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 62/112] 10.222.0.25 () {58 vars in 1024 bytes} [Sun Aug 7 10:41:06 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 203 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 07 10:41:06.894896 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.auth.core [None req-62bf8b90-6ec0-44bf-bb32-aae67e78e2c4 None None] MFA Rules not processed for user `bac4e6b76fe64217ac8fa44e71cffe6a`. Rule list: `[]` (Enabled: `True`). {{(pid=80143) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 10:41:06.896383 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-62bf8b90-6ec0-44bf-bb32-aae67e78e2c4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 10:41:06.908200 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c93d1b0c-aab6-4d87-88d9-3e159b54e181 None None] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 10:41:06.942667 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=100 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 10:41:06.974133 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 51/113] 10.222.0.25 () {58 vars in 1024 bytes} [Sun Aug 7 10:41:06 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 07 10:41:07.000174 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62bf8b90-6ec0-44bf-bb32-aae67e78e2c4 None None] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 10:41:07.027069 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c93d1b0c-aab6-4d87-88d9-3e159b54e181 None None] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 10:41:07.038804 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 10:41:07.074077 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd0fb703-cbc9-4711-b8cf-7d12d553ad42 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 10:41:07.075461 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd0fb703-cbc9-4711-b8cf-7d12d553ad42 None admin] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:41:07.075974 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd0fb703-cbc9-4711-b8cf-7d12d553ad42 None admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:41:07.076446 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd0fb703-cbc9-4711-b8cf-7d12d553ad42 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:41:07.088764 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd0fb703-cbc9-4711-b8cf-7d12d553ad42 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 07 10:41:07.094493 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application [None req-dd0fb703-cbc9-4711-b8cf-7d12d553ad42 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 07 10:41:07.094493 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 07 10:41:07.094493 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 07 10:41:07.094493 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 07 10:41:07.094493 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 07 10:41:07.094493 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 07 10:41:07.094493 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 07 10:41:07.094493 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 07 10:41:07.094493 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 07 10:41:07.094493 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 07 10:41:07.094493 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 07 10:41:07.094493 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 07 10:41:07.094493 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 07 10:41:07.094493 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 07 10:41:07.094493 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 07 10:41:07.094493 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 07 10:41:07.095383 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 07 10:41:07.095383 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 07 10:41:07.095383 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 07 10:41:07.095383 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 07 10:41:07.095383 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 07 10:41:07.095383 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 07 10:41:07.095383 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 07 10:41:07.095383 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 07 10:41:07.095383 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 07 10:41:07.095383 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self.get_or_create( Aug 07 10:41:07.095383 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 07 10:41:07.095383 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application with Lock( Aug 07 10:41:07.095383 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 07 10:41:07.095383 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self._enter() Aug 07 10:41:07.095383 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 07 10:41:07.095383 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 07 10:41:07.096129 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 07 10:41:07.096129 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self.creator() Aug 07 10:41:07.096129 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 07 10:41:07.096129 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application created_value = creator( Aug 07 10:41:07.096129 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 07 10:41:07.096129 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 07 10:41:07.096129 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 07 10:41:07.096129 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 07 10:41:07.096129 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 07 10:41:07.096129 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 07 10:41:07.096129 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 07 10:41:07.096129 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application Aug 07 10:41:07.098843 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 63/114] 10.222.0.25 () {56 vars in 1096 bytes} [Sun Aug 7 10:41:06 2022] GET /identity/v3/roles/admin => generated 83 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 07 10:41:07.107515 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd0fb703-cbc9-4711-b8cf-7d12d553ad42 None admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 10:41:07.114474 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62bf8b90-6ec0-44bf-bb32-aae67e78e2c4 None None] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 10:41:07.154548 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2af431df-3b21-4b34-a055-9d902eae5bde None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 10:41:07.155599 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2af431df-3b21-4b34-a055-9d902eae5bde None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:41:07.155599 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2af431df-3b21-4b34-a055-9d902eae5bde None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:41:07.155897 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2af431df-3b21-4b34-a055-9d902eae5bde None admin] PATH_INFO: `/v3/roles/member` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:41:07.168600 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2af431df-3b21-4b34-a055-9d902eae5bde None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 07 10:41:07.174114 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application [None req-2af431df-3b21-4b34-a055-9d902eae5bde None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 07 10:41:07.174114 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 07 10:41:07.174114 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 07 10:41:07.174114 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 07 10:41:07.174114 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 07 10:41:07.174114 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 07 10:41:07.174114 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 07 10:41:07.174114 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 07 10:41:07.174114 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 07 10:41:07.174114 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 07 10:41:07.174114 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 07 10:41:07.174114 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 07 10:41:07.174114 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 07 10:41:07.174114 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 07 10:41:07.174114 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 07 10:41:07.174114 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 07 10:41:07.175043 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 07 10:41:07.175043 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 07 10:41:07.175043 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 07 10:41:07.175043 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 07 10:41:07.175043 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 07 10:41:07.175043 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 07 10:41:07.175043 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 07 10:41:07.175043 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 07 10:41:07.175043 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 07 10:41:07.175043 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.get_or_create( Aug 07 10:41:07.175043 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 07 10:41:07.175043 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application with Lock( Aug 07 10:41:07.175043 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 07 10:41:07.175043 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self._enter() Aug 07 10:41:07.175043 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 07 10:41:07.175043 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 07 10:41:07.175774 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 07 10:41:07.175774 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.creator() Aug 07 10:41:07.175774 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 07 10:41:07.175774 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application created_value = creator( Aug 07 10:41:07.175774 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 07 10:41:07.175774 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 07 10:41:07.175774 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 07 10:41:07.175774 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 07 10:41:07.175774 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 07 10:41:07.175774 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 07 10:41:07.175774 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 07 10:41:07.175774 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application Aug 07 10:41:07.179065 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 52/115] 10.222.0.25 () {56 vars in 1099 bytes} [Sun Aug 7 10:41:06 2022] GET /identity/v3/roles/member => generated 84 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 07 10:41:07.180415 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-396c4b31-aee3-4bc5-bc30-1c86211e7223 None None] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:41:07.180803 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-396c4b31-aee3-4bc5-bc30-1c86211e7223 None None] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:41:07.181046 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-396c4b31-aee3-4bc5-bc30-1c86211e7223 None None] PATH_INFO: `/` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:41:07.185445 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 53/116] 10.222.0.25 () {54 vars in 927 bytes} [Sun Aug 7 10:41:07 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 07 10:41:07.187518 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-396c4b31-aee3-4bc5-bc30-1c86211e7223 None None] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 10:41:07.234883 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd0fb703-cbc9-4711-b8cf-7d12d553ad42 None admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 10:41:07.251303 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=100 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 10:41:07.271725 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72d45589-4ae6-44de-9ea9-7bf4c495bc6f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 10:41:07.272540 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72d45589-4ae6-44de-9ea9-7bf4c495bc6f None admin] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:41:07.272961 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72d45589-4ae6-44de-9ea9-7bf4c495bc6f None admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:41:07.273302 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72d45589-4ae6-44de-9ea9-7bf4c495bc6f None admin] PATH_INFO: `/v3/roles` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:41:07.274150 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72d45589-4ae6-44de-9ea9-7bf4c495bc6f None admin] RBAC: Adding query filter params (name=admin) {{(pid=80144) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 10:41:07.274562 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72d45589-4ae6-44de-9ea9-7bf4c495bc6f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 10:41:07.276213 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72d45589-4ae6-44de-9ea9-7bf4c495bc6f None admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 10:41:07.285993 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 64/117] 10.222.0.25 () {56 vars in 1099 bytes} [Sun Aug 7 10:41:07 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 07 10:41:07.287608 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df5b4f0e-9209-41ed-8481-d7e340554996 None None] REQUEST_METHOD: `POST` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:41:07.288001 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df5b4f0e-9209-41ed-8481-d7e340554996 None None] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:41:07.288385 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df5b4f0e-9209-41ed-8481-d7e340554996 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:41:07.307336 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.auth.core [None req-df5b4f0e-9209-41ed-8481-d7e340554996 None None] MFA Rules not processed for user `bac4e6b76fe64217ac8fa44e71cffe6a`. Rule list: `[]` (Enabled: `True`). {{(pid=80144) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 10:41:07.309432 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.fernet_utils [None req-df5b4f0e-9209-41ed-8481-d7e340554996 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80144) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 10:41:07.321493 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-396c4b31-aee3-4bc5-bc30-1c86211e7223 None None] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 10:41:07.335888 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=100 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 10:41:07.372052 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00bb4501-eb69-4700-a906-d3eac1b6e7b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 10:41:07.373854 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00bb4501-eb69-4700-a906-d3eac1b6e7b5 None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:41:07.374696 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00bb4501-eb69-4700-a906-d3eac1b6e7b5 None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:41:07.375135 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00bb4501-eb69-4700-a906-d3eac1b6e7b5 None admin] PATH_INFO: `/v3/roles` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:41:07.376228 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00bb4501-eb69-4700-a906-d3eac1b6e7b5 None admin] RBAC: Adding query filter params (name=member) {{(pid=80143) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 10:41:07.376905 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00bb4501-eb69-4700-a906-d3eac1b6e7b5 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 10:41:07.379130 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00bb4501-eb69-4700-a906-d3eac1b6e7b5 None admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 10:41:07.385457 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 65/118] 10.222.0.25 () {58 vars in 1024 bytes} [Sun Aug 7 10:41:07 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 07 10:41:07.386618 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df5b4f0e-9209-41ed-8481-d7e340554996 None None] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 10:41:07.391235 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 54/119] 10.222.0.25 () {56 vars in 1101 bytes} [Sun Aug 7 10:41:07 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 10:41:07.392652 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-532574f4-d9c6-4db8-8b57-af5ac47a0964 None None] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:41:07.393071 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-532574f4-d9c6-4db8-8b57-af5ac47a0964 None None] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:41:07.393466 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-532574f4-d9c6-4db8-8b57-af5ac47a0964 None None] PATH_INFO: `/` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:41:07.394776 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 55/120] 10.222.0.25 () {54 vars in 927 bytes} [Sun Aug 7 10:41:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 07 10:41:07.403207 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-532574f4-d9c6-4db8-8b57-af5ac47a0964 None None] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 10:41:07.522233 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=100 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 10:41:07.530805 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df5b4f0e-9209-41ed-8481-d7e340554996 None None] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 10:41:07.534457 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-532574f4-d9c6-4db8-8b57-af5ac47a0964 None None] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 10:41:07.571576 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65485fe8-7ac3-4a49-bd18-968d38b1e3d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 10:41:07.572260 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65485fe8-7ac3-4a49-bd18-968d38b1e3d2 None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:41:07.572546 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65485fe8-7ac3-4a49-bd18-968d38b1e3d2 None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:41:07.572828 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65485fe8-7ac3-4a49-bd18-968d38b1e3d2 None admin] PATH_INFO: `/v3/users/cfdc7e8b78b84c6aa44d115a396152bd` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:41:07.574219 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65485fe8-7ac3-4a49-bd18-968d38b1e3d2 None admin] RBAC: Authorizing `identity:get_user(user_id=cfdc7e8b78b84c6aa44d115a396152bd)` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 10:41:07.575710 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65485fe8-7ac3-4a49-bd18-968d38b1e3d2 None admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 10:41:07.577463 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 56/121] 10.222.0.25 () {56 vars in 1177 bytes} [Sun Aug 7 10:41:07 2022] GET /identity/v3/users/cfdc7e8b78b84c6aa44d115a396152bd => generated 275 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 10:41:07.577802 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95e41e64-6769-4c6f-86f2-421f06a759ba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 10:41:07.578687 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95e41e64-6769-4c6f-86f2-421f06a759ba None admin] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:41:07.579191 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95e41e64-6769-4c6f-86f2-421f06a759ba None admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:41:07.579684 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95e41e64-6769-4c6f-86f2-421f06a759ba None admin] PATH_INFO: `/v3/users/bac4e6b76fe64217ac8fa44e71cffe6a` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:41:07.581033 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95e41e64-6769-4c6f-86f2-421f06a759ba None admin] RBAC: Authorizing `identity:get_user(user_id=bac4e6b76fe64217ac8fa44e71cffe6a)` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 10:41:07.582679 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95e41e64-6769-4c6f-86f2-421f06a759ba None admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 10:41:07.583054 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73d6dcdd-cb05-44ee-8d74-e2faa5b177a0 None None] REQUEST_METHOD: `POST` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:41:07.583349 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73d6dcdd-cb05-44ee-8d74-e2faa5b177a0 None None] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:41:07.583633 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73d6dcdd-cb05-44ee-8d74-e2faa5b177a0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:41:07.585111 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 66/122] 10.222.0.25 () {56 vars in 1177 bytes} [Sun Aug 7 10:41:07 2022] GET /identity/v3/users/bac4e6b76fe64217ac8fa44e71cffe6a => generated 247 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 10:41:07.591101 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95e41e64-6769-4c6f-86f2-421f06a759ba None admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 10:41:07.607144 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.auth.core [None req-73d6dcdd-cb05-44ee-8d74-e2faa5b177a0 None None] MFA Rules not processed for user `bac4e6b76fe64217ac8fa44e71cffe6a`. Rule list: `[]` (Enabled: `True`). {{(pid=80143) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 10:41:07.608593 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-73d6dcdd-cb05-44ee-8d74-e2faa5b177a0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 10:41:07.625879 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=100 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 10:41:07.695053 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 57/123] 10.222.0.25 () {58 vars in 1024 bytes} [Sun Aug 7 10:41:07 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 07 10:41:07.696138 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73d6dcdd-cb05-44ee-8d74-e2faa5b177a0 None None] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 10:41:07.728083 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95e41e64-6769-4c6f-86f2-421f06a759ba None admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 10:41:07.776187 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69d06c37-5ffd-4932-94be-7cc30e340772 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 10:41:07.777357 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69d06c37-5ffd-4932-94be-7cc30e340772 None admin] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:41:07.777838 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69d06c37-5ffd-4932-94be-7cc30e340772 None admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:41:07.778397 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69d06c37-5ffd-4932-94be-7cc30e340772 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:41:07.794433 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69d06c37-5ffd-4932-94be-7cc30e340772 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 07 10:41:07.802561 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application [None req-69d06c37-5ffd-4932-94be-7cc30e340772 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 07 10:41:07.802561 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 07 10:41:07.802561 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 07 10:41:07.802561 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 07 10:41:07.802561 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 07 10:41:07.802561 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 07 10:41:07.802561 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 07 10:41:07.802561 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 07 10:41:07.802561 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 07 10:41:07.802561 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 07 10:41:07.802561 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 07 10:41:07.802561 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 07 10:41:07.802561 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 07 10:41:07.802561 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 07 10:41:07.802561 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 07 10:41:07.802561 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 07 10:41:07.803532 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 07 10:41:07.803532 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 07 10:41:07.803532 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 07 10:41:07.803532 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 07 10:41:07.803532 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 07 10:41:07.803532 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 07 10:41:07.803532 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 07 10:41:07.803532 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 07 10:41:07.803532 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 07 10:41:07.803532 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self.get_or_create( Aug 07 10:41:07.803532 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 07 10:41:07.803532 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application with Lock( Aug 07 10:41:07.803532 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 07 10:41:07.803532 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self._enter() Aug 07 10:41:07.803532 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 07 10:41:07.803532 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 07 10:41:07.804331 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 07 10:41:07.804331 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self.creator() Aug 07 10:41:07.804331 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 07 10:41:07.804331 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application created_value = creator( Aug 07 10:41:07.804331 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 07 10:41:07.804331 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 07 10:41:07.804331 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 07 10:41:07.804331 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 07 10:41:07.804331 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 07 10:41:07.804331 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 07 10:41:07.804331 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 07 10:41:07.804331 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application Aug 07 10:41:07.806731 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 67/124] 10.222.0.25 () {56 vars in 1114 bytes} [Sun Aug 7 10:41:07 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 216 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 07 10:41:07.807784 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69d06c37-5ffd-4932-94be-7cc30e340772 None admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 10:41:07.828794 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=100 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 10:41:07.836445 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73d6dcdd-cb05-44ee-8d74-e2faa5b177a0 None None] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 10:41:07.873872 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfe320de-f1c5-45db-a0d1-1687a90c6a32 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 10:41:07.874968 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfe320de-f1c5-45db-a0d1-1687a90c6a32 None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:41:07.875476 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfe320de-f1c5-45db-a0d1-1687a90c6a32 None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:41:07.876061 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfe320de-f1c5-45db-a0d1-1687a90c6a32 None admin] PATH_INFO: `/v3/projects/90718391b09e493bb296767dc8a3c0ce` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:41:07.877403 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfe320de-f1c5-45db-a0d1-1687a90c6a32 None admin] RBAC: Authorizing `identity:get_project(project_id=90718391b09e493bb296767dc8a3c0ce)` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 10:41:07.879147 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfe320de-f1c5-45db-a0d1-1687a90c6a32 None admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 10:41:07.885994 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 58/125] 10.222.0.25 () {56 vars in 1186 bytes} [Sun Aug 7 10:41:07 2022] GET /identity/v3/projects/90718391b09e493bb296767dc8a3c0ce => generated 298 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 10:41:07.887711 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfe320de-f1c5-45db-a0d1-1687a90c6a32 None admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 10:41:07.910428 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=100 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 10:41:07.924670 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69d06c37-5ffd-4932-94be-7cc30e340772 None admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 10:41:07.961705 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3f9945d-69fe-41fd-8456-a8c12b7c150f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 10:41:07.962611 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3f9945d-69fe-41fd-8456-a8c12b7c150f None admin] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:41:07.963134 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3f9945d-69fe-41fd-8456-a8c12b7c150f None admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:41:07.963588 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3f9945d-69fe-41fd-8456-a8c12b7c150f None admin] PATH_INFO: `/v3/projects/90718391b09e493bb296767dc8a3c0ce` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:41:07.964944 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3f9945d-69fe-41fd-8456-a8c12b7c150f None admin] RBAC: Authorizing `identity:get_project(project_id=90718391b09e493bb296767dc8a3c0ce)` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 10:41:07.966833 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3f9945d-69fe-41fd-8456-a8c12b7c150f None admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 10:41:07.968759 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 68/126] 10.222.0.25 () {56 vars in 1186 bytes} [Sun Aug 7 10:41:07 2022] GET /identity/v3/projects/90718391b09e493bb296767dc8a3c0ce => generated 298 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 10:41:07.974532 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3f9945d-69fe-41fd-8456-a8c12b7c150f None admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 10:41:07.992989 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfe320de-f1c5-45db-a0d1-1687a90c6a32 None admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 10:41:08.028967 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59c2c32b-412c-4238-8e0f-000f0f421a46 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 10:41:08.029678 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59c2c32b-412c-4238-8e0f-000f0f421a46 None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:41:08.029891 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59c2c32b-412c-4238-8e0f-000f0f421a46 None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:41:08.030170 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59c2c32b-412c-4238-8e0f-000f0f421a46 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:41:08.046946 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59c2c32b-412c-4238-8e0f-000f0f421a46 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 07 10:41:08.047083 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application [None req-59c2c32b-412c-4238-8e0f-000f0f421a46 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 07 10:41:08.047083 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 07 10:41:08.047083 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 07 10:41:08.047083 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 07 10:41:08.047083 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 07 10:41:08.047083 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 07 10:41:08.047083 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 07 10:41:08.047083 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 07 10:41:08.047083 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 07 10:41:08.047083 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 07 10:41:08.047083 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 07 10:41:08.047083 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 07 10:41:08.047083 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 07 10:41:08.047083 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 07 10:41:08.047083 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 07 10:41:08.047083 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 07 10:41:08.047874 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 07 10:41:08.047874 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 07 10:41:08.047874 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 07 10:41:08.047874 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 07 10:41:08.047874 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 07 10:41:08.047874 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 07 10:41:08.047874 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 07 10:41:08.047874 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 07 10:41:08.047874 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 07 10:41:08.047874 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.get_or_create( Aug 07 10:41:08.047874 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 07 10:41:08.047874 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application with Lock( Aug 07 10:41:08.047874 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 07 10:41:08.047874 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self._enter() Aug 07 10:41:08.047874 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 07 10:41:08.047874 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 07 10:41:08.048633 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 07 10:41:08.048633 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.creator() Aug 07 10:41:08.048633 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 07 10:41:08.048633 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application created_value = creator( Aug 07 10:41:08.048633 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 07 10:41:08.048633 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 07 10:41:08.048633 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 07 10:41:08.048633 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 07 10:41:08.048633 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 07 10:41:08.048633 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 07 10:41:08.048633 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 07 10:41:08.048633 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application Aug 07 10:41:08.048633 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 59/127] 10.222.0.25 () {56 vars in 1099 bytes} [Sun Aug 7 10:41:07 2022] GET /identity/v3/roles/member => generated 84 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 07 10:41:08.049248 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59c2c32b-412c-4238-8e0f-000f0f421a46 None admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 10:41:08.080394 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=100 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 10:41:08.095213 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3f9945d-69fe-41fd-8456-a8c12b7c150f None admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 10:41:08.131191 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0dc6ea8-0fd2-422f-af20-71ae639ac062 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 10:41:08.131776 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0dc6ea8-0fd2-422f-af20-71ae639ac062 None admin] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:41:08.132024 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0dc6ea8-0fd2-422f-af20-71ae639ac062 None admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:41:08.132290 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0dc6ea8-0fd2-422f-af20-71ae639ac062 None admin] PATH_INFO: `/v3/roles` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:41:08.132965 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0dc6ea8-0fd2-422f-af20-71ae639ac062 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80144) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 10:41:08.133273 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0dc6ea8-0fd2-422f-af20-71ae639ac062 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 10:41:08.134638 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0dc6ea8-0fd2-422f-af20-71ae639ac062 None admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 10:41:08.144001 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 69/128] 10.222.0.25 () {56 vars in 1111 bytes} [Sun Aug 7 10:41:07 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 10:41:08.144913 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0dc6ea8-0fd2-422f-af20-71ae639ac062 None admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 10:41:08.178433 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59c2c32b-412c-4238-8e0f-000f0f421a46 None admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 10:41:08.179316 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=100 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 10:41:08.227632 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a85eacd8-d3a5-498f-9327-fe8e40ec84b3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 10:41:08.228318 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a85eacd8-d3a5-498f-9327-fe8e40ec84b3 None admin] REQUEST_METHOD: `PUT` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:41:08.228603 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a85eacd8-d3a5-498f-9327-fe8e40ec84b3 None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:41:08.228910 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a85eacd8-d3a5-498f-9327-fe8e40ec84b3 None admin] PATH_INFO: `/v3/projects/90718391b09e493bb296767dc8a3c0ce/users/cfdc7e8b78b84c6aa44d115a396152bd/roles/b98dfb3b9fee4644af2887a3ac3fd2e6` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:41:08.234495 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a85eacd8-d3a5-498f-9327-fe8e40ec84b3 None admin] RBAC: Authorizing `identity:create_grant(project_id=90718391b09e493bb296767dc8a3c0ce, user_id=cfdc7e8b78b84c6aa44d115a396152bd, role_id=b98dfb3b9fee4644af2887a3ac3fd2e6)` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 10:41:08.236417 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a85eacd8-d3a5-498f-9327-fe8e40ec84b3 None admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 10:41:08.263569 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 60/129] 10.222.0.25 () {58 vars in 1439 bytes} [Sun Aug 7 10:41:08 2022] PUT /identity/v3/projects/90718391b09e493bb296767dc8a3c0ce/users/cfdc7e8b78b84c6aa44d115a396152bd/roles/b98dfb3b9fee4644af2887a3ac3fd2e6 => generated 0 bytes in 215 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 07 10:41:08.266553 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a85eacd8-d3a5-498f-9327-fe8e40ec84b3 None admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 10:41:08.271496 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0dc6ea8-0fd2-422f-af20-71ae639ac062 None admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 10:41:08.325130 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e2fe4f3-c4f3-435d-91c5-0ce38f50d956 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 10:41:08.326544 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e2fe4f3-c4f3-435d-91c5-0ce38f50d956 None admin] REQUEST_METHOD: `PUT` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:41:08.327080 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e2fe4f3-c4f3-435d-91c5-0ce38f50d956 None admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:41:08.327560 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e2fe4f3-c4f3-435d-91c5-0ce38f50d956 None admin] PATH_INFO: `/v3/projects/90718391b09e493bb296767dc8a3c0ce/users/bac4e6b76fe64217ac8fa44e71cffe6a/roles/81b1f1bcee2a473088fbbd327c1c5b23` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:41:08.330520 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e2fe4f3-c4f3-435d-91c5-0ce38f50d956 None admin] RBAC: Authorizing `identity:create_grant(project_id=90718391b09e493bb296767dc8a3c0ce, user_id=bac4e6b76fe64217ac8fa44e71cffe6a, role_id=81b1f1bcee2a473088fbbd327c1c5b23)` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 10:41:08.332739 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e2fe4f3-c4f3-435d-91c5-0ce38f50d956 None admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 10:41:08.370067 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 10:41:08.381792 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 70/130] 10.222.0.25 () {58 vars in 1439 bytes} [Sun Aug 7 10:41:08 2022] PUT /identity/v3/projects/90718391b09e493bb296767dc8a3c0ce/users/bac4e6b76fe64217ac8fa44e71cffe6a/roles/81b1f1bcee2a473088fbbd327c1c5b23 => generated 0 bytes in 237 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 07 10:41:08.382507 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e2fe4f3-c4f3-435d-91c5-0ce38f50d956 None admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 10:41:08.393180 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a85eacd8-d3a5-498f-9327-fe8e40ec84b3 None admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 10:41:08.436771 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0de8e307-a428-4962-8dfe-0de0fdb8e966 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 10:41:08.437682 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0de8e307-a428-4962-8dfe-0de0fdb8e966 None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:41:08.438281 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0de8e307-a428-4962-8dfe-0de0fdb8e966 None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:41:08.438770 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0de8e307-a428-4962-8dfe-0de0fdb8e966 None admin] PATH_INFO: `/v3/roles` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:41:08.440056 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0de8e307-a428-4962-8dfe-0de0fdb8e966 None admin] RBAC: Adding query filter params (name=member) {{(pid=80143) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 10:41:08.440682 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0de8e307-a428-4962-8dfe-0de0fdb8e966 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 10:41:08.442363 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0de8e307-a428-4962-8dfe-0de0fdb8e966 None admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 10:41:08.454294 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 61/131] 10.222.0.25 () {56 vars in 1101 bytes} [Sun Aug 7 10:41:08 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 10:41:08.462216 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0de8e307-a428-4962-8dfe-0de0fdb8e966 None admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 10:41:08.619177 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 10:41:08.629626 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0de8e307-a428-4962-8dfe-0de0fdb8e966 None admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 10:41:08.673736 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e0aa112-e96e-4b0e-917f-0164ac0f26c4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 10:41:08.674481 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e0aa112-e96e-4b0e-917f-0164ac0f26c4 None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:41:08.674878 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e0aa112-e96e-4b0e-917f-0164ac0f26c4 None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:41:08.675355 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e0aa112-e96e-4b0e-917f-0164ac0f26c4 None admin] PATH_INFO: `/v3/users/cfdc7e8b78b84c6aa44d115a396152bd` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:41:08.676889 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e0aa112-e96e-4b0e-917f-0164ac0f26c4 None admin] RBAC: Authorizing `identity:get_user(user_id=cfdc7e8b78b84c6aa44d115a396152bd)` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 10:41:08.679023 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e0aa112-e96e-4b0e-917f-0164ac0f26c4 None admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 10:41:08.680911 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 62/132] 10.222.0.25 () {56 vars in 1177 bytes} [Sun Aug 7 10:41:08 2022] GET /identity/v3/users/cfdc7e8b78b84c6aa44d115a396152bd => generated 275 bytes in 219 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 07 10:41:08.682973 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e2fe4f3-c4f3-435d-91c5-0ce38f50d956 None admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 10:41:08.691391 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e0aa112-e96e-4b0e-917f-0164ac0f26c4 None admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 10:41:08.711761 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=100 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 10:41:08.768937 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-569c214b-a477-4c28-b7ee-b2416f0d1f96 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 10:41:08.769823 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-569c214b-a477-4c28-b7ee-b2416f0d1f96 None admin] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:41:08.770245 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-569c214b-a477-4c28-b7ee-b2416f0d1f96 None admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:41:08.770685 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-569c214b-a477-4c28-b7ee-b2416f0d1f96 None admin] PATH_INFO: `/v3/users/cfdc7e8b78b84c6aa44d115a396152bd` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:41:08.773430 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-569c214b-a477-4c28-b7ee-b2416f0d1f96 None admin] RBAC: Authorizing `identity:get_user(user_id=cfdc7e8b78b84c6aa44d115a396152bd)` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 10:41:08.775077 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-569c214b-a477-4c28-b7ee-b2416f0d1f96 None admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 10:41:08.780800 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 71/133] 10.222.0.25 () {56 vars in 1177 bytes} [Sun Aug 7 10:41:08 2022] GET /identity/v3/users/cfdc7e8b78b84c6aa44d115a396152bd => generated 275 bytes in 399 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 10:41:08.787201 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-569c214b-a477-4c28-b7ee-b2416f0d1f96 None admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 10:41:08.903825 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e0aa112-e96e-4b0e-917f-0164ac0f26c4 None admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 10:41:08.974599 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae374c6b-6313-4951-a07e-1ea3bb89babd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 10:41:08.975521 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae374c6b-6313-4951-a07e-1ea3bb89babd None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:41:08.975945 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae374c6b-6313-4951-a07e-1ea3bb89babd None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:41:08.976366 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae374c6b-6313-4951-a07e-1ea3bb89babd None admin] PATH_INFO: `/v3/projects/6974c88728de4be899fbc38155f70bb9` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:41:08.978446 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae374c6b-6313-4951-a07e-1ea3bb89babd None admin] RBAC: Authorizing `identity:get_project(project_id=6974c88728de4be899fbc38155f70bb9)` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 10:41:08.980460 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae374c6b-6313-4951-a07e-1ea3bb89babd None admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 10:41:08.984532 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 63/134] 10.222.0.25 () {56 vars in 1186 bytes} [Sun Aug 7 10:41:08 2022] GET /identity/v3/projects/6974c88728de4be899fbc38155f70bb9 => generated 312 bytes in 295 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 07 10:41:08.994970 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae374c6b-6313-4951-a07e-1ea3bb89babd None admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 10:41:09.017779 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-569c214b-a477-4c28-b7ee-b2416f0d1f96 None admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 10:41:09.088348 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90b32ed6-62dc-454b-aa0b-af96b36d079c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 10:41:09.089276 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90b32ed6-62dc-454b-aa0b-af96b36d079c None admin] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:41:09.090352 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90b32ed6-62dc-454b-aa0b-af96b36d079c None admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:41:09.090776 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90b32ed6-62dc-454b-aa0b-af96b36d079c None admin] PATH_INFO: `/v3/projects/90718391b09e493bb296767dc8a3c0ce` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:41:09.093927 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90b32ed6-62dc-454b-aa0b-af96b36d079c None admin] RBAC: Authorizing `identity:get_project(project_id=90718391b09e493bb296767dc8a3c0ce)` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 10:41:09.096419 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90b32ed6-62dc-454b-aa0b-af96b36d079c None admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 10:41:09.102639 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 72/135] 10.222.0.25 () {56 vars in 1186 bytes} [Sun Aug 7 10:41:08 2022] GET /identity/v3/projects/90718391b09e493bb296767dc8a3c0ce => generated 298 bytes in 316 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 10:41:09.108802 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90b32ed6-62dc-454b-aa0b-af96b36d079c None admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 10:41:09.150284 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=100 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 10:41:09.151293 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=100 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 10:41:09.193935 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae374c6b-6313-4951-a07e-1ea3bb89babd None admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 10:41:09.245733 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6adaf7cd-e228-4f46-b0ea-915b52297834 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 10:41:09.246615 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6adaf7cd-e228-4f46-b0ea-915b52297834 None admin] REQUEST_METHOD: `PUT` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:41:09.246912 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6adaf7cd-e228-4f46-b0ea-915b52297834 None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:41:09.247224 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6adaf7cd-e228-4f46-b0ea-915b52297834 None admin] PATH_INFO: `/v3/projects/6974c88728de4be899fbc38155f70bb9/users/cfdc7e8b78b84c6aa44d115a396152bd/roles/b98dfb3b9fee4644af2887a3ac3fd2e6` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:41:09.249938 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6adaf7cd-e228-4f46-b0ea-915b52297834 None admin] RBAC: Authorizing `identity:create_grant(project_id=6974c88728de4be899fbc38155f70bb9, user_id=cfdc7e8b78b84c6aa44d115a396152bd, role_id=b98dfb3b9fee4644af2887a3ac3fd2e6)` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 10:41:09.251446 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6adaf7cd-e228-4f46-b0ea-915b52297834 None admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 10:41:09.273840 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 64/136] 10.222.0.25 () {58 vars in 1439 bytes} [Sun Aug 7 10:41:08 2022] PUT /identity/v3/projects/6974c88728de4be899fbc38155f70bb9/users/cfdc7e8b78b84c6aa44d115a396152bd/roles/b98dfb3b9fee4644af2887a3ac3fd2e6 => generated 0 bytes in 279 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 07 10:41:09.275218 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1929e226-5ebc-4200-bcf9-ea8310b6de6f None None] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:41:09.276695 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1929e226-5ebc-4200-bcf9-ea8310b6de6f None None] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:41:09.277242 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1929e226-5ebc-4200-bcf9-ea8310b6de6f None None] PATH_INFO: `/` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:41:09.285782 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 65/137] 10.222.0.25 () {54 vars in 927 bytes} [Sun Aug 7 10:41:09 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 07 10:41:09.297471 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e86039d-c669-4fc4-87c9-88c80de8fcc2 None None] REQUEST_METHOD: `POST` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:41:09.297757 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e86039d-c669-4fc4-87c9-88c80de8fcc2 None None] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:41:09.298043 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e86039d-c669-4fc4-87c9-88c80de8fcc2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:41:09.331656 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.auth.core [None req-2e86039d-c669-4fc4-87c9-88c80de8fcc2 None None] MFA Rules not processed for user `bac4e6b76fe64217ac8fa44e71cffe6a`. Rule list: `[]` (Enabled: `True`). {{(pid=80143) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 10:41:09.332900 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-2e86039d-c669-4fc4-87c9-88c80de8fcc2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 10:41:09.336546 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90b32ed6-62dc-454b-aa0b-af96b36d079c None admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 10:41:09.403985 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e2585de-b7ff-4543-9091-e2a17930a022 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 10:41:09.404678 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e2585de-b7ff-4543-9091-e2a17930a022 None admin] REQUEST_METHOD: `PUT` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:41:09.404960 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e2585de-b7ff-4543-9091-e2a17930a022 None admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:41:09.405235 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e2585de-b7ff-4543-9091-e2a17930a022 None admin] PATH_INFO: `/v3/projects/90718391b09e493bb296767dc8a3c0ce/users/cfdc7e8b78b84c6aa44d115a396152bd/roles/1512d531ebe248ea8e2d7d56d754b230` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:41:09.419922 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e2585de-b7ff-4543-9091-e2a17930a022 None admin] RBAC: Authorizing `identity:create_grant(project_id=90718391b09e493bb296767dc8a3c0ce, user_id=cfdc7e8b78b84c6aa44d115a396152bd, role_id=1512d531ebe248ea8e2d7d56d754b230)` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 10:41:09.421426 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e2585de-b7ff-4543-9091-e2a17930a022 None admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 10:41:09.441354 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 73/138] 10.222.0.25 () {58 vars in 1439 bytes} [Sun Aug 7 10:41:09 2022] PUT /identity/v3/projects/90718391b09e493bb296767dc8a3c0ce/users/cfdc7e8b78b84c6aa44d115a396152bd/roles/1512d531ebe248ea8e2d7d56d754b230 => generated 0 bytes in 333 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 07 10:41:09.442637 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-360ff11b-3d15-46ee-9424-f6a8f92ee0c7 None None] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:41:09.442954 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-360ff11b-3d15-46ee-9424-f6a8f92ee0c7 None None] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:41:09.443239 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-360ff11b-3d15-46ee-9424-f6a8f92ee0c7 None None] PATH_INFO: `/` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:41:09.444617 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 74/139] 10.222.0.25 () {54 vars in 927 bytes} [Sun Aug 7 10:41:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 07 10:41:09.454899 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b716dcb7-e951-4892-8a7b-1849ad553457 None None] REQUEST_METHOD: `POST` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:41:09.455204 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b716dcb7-e951-4892-8a7b-1849ad553457 None None] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:41:09.455485 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b716dcb7-e951-4892-8a7b-1849ad553457 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:41:09.489859 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 66/140] 10.222.0.25 () {58 vars in 1024 bytes} [Sun Aug 7 10:41:09 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 193 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 07 10:41:09.490686 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.auth.core [None req-b716dcb7-e951-4892-8a7b-1849ad553457 None None] MFA Rules not processed for user `bac4e6b76fe64217ac8fa44e71cffe6a`. Rule list: `[]` (Enabled: `True`). {{(pid=80144) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 10:41:09.491935 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.fernet_utils [None req-b716dcb7-e951-4892-8a7b-1849ad553457 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80144) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 10:41:09.540163 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e86039d-c669-4fc4-87c9-88c80de8fcc2 None None] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 10:41:09.604014 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 75/141] 10.222.0.25 () {58 vars in 1024 bytes} [Sun Aug 7 10:41:09 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 150 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 07 10:41:09.627282 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 10:41:09.631859 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b716dcb7-e951-4892-8a7b-1849ad553457 None None] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 10:41:09.635921 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 10:41:09.688291 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e86039d-c669-4fc4-87c9-88c80de8fcc2 None None] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 10:41:09.746626 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d24b2384-7924-4ec4-9b9b-ba80264147c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 10:41:09.747301 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d24b2384-7924-4ec4-9b9b-ba80264147c2 None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:41:09.747580 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d24b2384-7924-4ec4-9b9b-ba80264147c2 None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:41:09.750947 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d24b2384-7924-4ec4-9b9b-ba80264147c2 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:41:09.750947 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d24b2384-7924-4ec4-9b9b-ba80264147c2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 10:41:09.751142 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d24b2384-7924-4ec4-9b9b-ba80264147c2 None admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 10:41:09.752770 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 67/142] 10.222.0.25 () {56 vars in 1108 bytes} [Sun Aug 7 10:41:09 2022] GET /identity/v3/domains/default => generated 199 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 10:41:09.760164 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d24b2384-7924-4ec4-9b9b-ba80264147c2 None admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 10:41:09.802557 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b716dcb7-e951-4892-8a7b-1849ad553457 None None] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 10:41:09.880222 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b2b329d-e631-415b-ba17-2d2521ed61cd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 10:41:09.880860 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b2b329d-e631-415b-ba17-2d2521ed61cd None admin] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:41:09.881140 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b2b329d-e631-415b-ba17-2d2521ed61cd None admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:41:09.881416 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b2b329d-e631-415b-ba17-2d2521ed61cd None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:41:09.904816 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b2b329d-e631-415b-ba17-2d2521ed61cd None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 07 10:41:09.910101 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application [None req-0b2b329d-e631-415b-ba17-2d2521ed61cd None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Aug 07 10:41:09.910101 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 07 10:41:09.910101 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 07 10:41:09.910101 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 07 10:41:09.910101 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 07 10:41:09.910101 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 07 10:41:09.910101 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 07 10:41:09.910101 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 07 10:41:09.910101 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 07 10:41:09.910101 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 07 10:41:09.910101 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 07 10:41:09.910101 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 07 10:41:09.910101 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 07 10:41:09.910101 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 07 10:41:09.910101 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 07 10:41:09.910101 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 07 10:41:09.910954 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 07 10:41:09.910954 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 07 10:41:09.910954 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 07 10:41:09.910954 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 07 10:41:09.910954 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 07 10:41:09.910954 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 07 10:41:09.910954 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 07 10:41:09.910954 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 07 10:41:09.910954 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 07 10:41:09.910954 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self.get_or_create( Aug 07 10:41:09.910954 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 07 10:41:09.910954 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application with Lock( Aug 07 10:41:09.910954 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 07 10:41:09.910954 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self._enter() Aug 07 10:41:09.910954 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 07 10:41:09.910954 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 07 10:41:09.912143 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 07 10:41:09.912143 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self.creator() Aug 07 10:41:09.912143 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 07 10:41:09.912143 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application created_value = creator( Aug 07 10:41:09.912143 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 07 10:41:09.912143 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 07 10:41:09.912143 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 07 10:41:09.912143 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 07 10:41:09.912143 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 07 10:41:09.912143 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 07 10:41:09.912143 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Aug 07 10:41:09.912143 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application Aug 07 10:41:09.912143 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 76/143] 10.222.0.25 () {56 vars in 1099 bytes} [Sun Aug 7 10:41:09 2022] GET /identity/v3/roles/reader => generated 84 bytes in 280 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 07 10:41:09.919805 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b2b329d-e631-415b-ba17-2d2521ed61cd None admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 10:41:09.956770 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d24b2384-7924-4ec4-9b9b-ba80264147c2 None admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 10:41:10.011067 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=100 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 10:41:10.017486 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36e5b450-a9f1-4af0-87d6-1ff68dac2eee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 10:41:10.018158 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36e5b450-a9f1-4af0-87d6-1ff68dac2eee None admin] REQUEST_METHOD: `POST` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:41:10.018444 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36e5b450-a9f1-4af0-87d6-1ff68dac2eee None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:41:10.018719 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36e5b450-a9f1-4af0-87d6-1ff68dac2eee None admin] PATH_INFO: `/v3/users` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:41:10.019398 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36e5b450-a9f1-4af0-87d6-1ff68dac2eee None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 10:41:10.020846 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36e5b450-a9f1-4af0-87d6-1ff68dac2eee None admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 10:41:10.032249 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 10:41:10.053307 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 68/144] 10.222.0.25 () {60 vars in 1132 bytes} [Sun Aug 7 10:41:09 2022] POST /identity/v3/users => generated 283 bytes in 293 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 07 10:41:10.077676 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b2b329d-e631-415b-ba17-2d2521ed61cd None admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 10:41:10.126731 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23313f81-99e3-469d-9113-4fde9fd717c0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 10:41:10.128178 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23313f81-99e3-469d-9113-4fde9fd717c0 None admin] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:41:10.128681 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23313f81-99e3-469d-9113-4fde9fd717c0 None admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:41:10.129155 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23313f81-99e3-469d-9113-4fde9fd717c0 None admin] PATH_INFO: `/v3/roles` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:41:10.130023 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23313f81-99e3-469d-9113-4fde9fd717c0 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80144) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 10:41:10.130613 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23313f81-99e3-469d-9113-4fde9fd717c0 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 10:41:10.132235 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23313f81-99e3-469d-9113-4fde9fd717c0 None admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 10:41:10.142175 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 77/145] 10.222.0.25 () {56 vars in 1101 bytes} [Sun Aug 7 10:41:09 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 223 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 10:41:10.152206 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36e5b450-a9f1-4af0-87d6-1ff68dac2eee None admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 10:41:10.305964 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36e5b450-a9f1-4af0-87d6-1ff68dac2eee None admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 10:41:10.353930 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d248ae56-6345-4fd5-b15f-928e154a7ff9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 10:41:10.354568 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d248ae56-6345-4fd5-b15f-928e154a7ff9 None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:41:10.354849 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d248ae56-6345-4fd5-b15f-928e154a7ff9 None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:41:10.355221 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d248ae56-6345-4fd5-b15f-928e154a7ff9 None admin] PATH_INFO: `/v3/users/b1626c83857e4f2091030226ccd3ca7e` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:41:10.359665 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d248ae56-6345-4fd5-b15f-928e154a7ff9 None admin] RBAC: Authorizing `identity:get_user(user_id=b1626c83857e4f2091030226ccd3ca7e)` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 10:41:10.361207 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d248ae56-6345-4fd5-b15f-928e154a7ff9 None admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 10:41:10.363264 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 69/146] 10.222.0.25 () {56 vars in 1177 bytes} [Sun Aug 7 10:41:10 2022] GET /identity/v3/users/b1626c83857e4f2091030226ccd3ca7e => generated 289 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 10:41:10.369174 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23313f81-99e3-469d-9113-4fde9fd717c0 None admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 10:41:10.529198 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23313f81-99e3-469d-9113-4fde9fd717c0 None admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 10:41:10.561276 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=100 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 10:41:10.570306 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bbd7dcd-999e-4f9e-9e8b-2b1161c6cc31 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 10:41:10.570969 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bbd7dcd-999e-4f9e-9e8b-2b1161c6cc31 None admin] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:41:10.571257 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bbd7dcd-999e-4f9e-9e8b-2b1161c6cc31 None admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:41:10.571540 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bbd7dcd-999e-4f9e-9e8b-2b1161c6cc31 None admin] PATH_INFO: `/v3/projects/90718391b09e493bb296767dc8a3c0ce` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:41:10.572699 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bbd7dcd-999e-4f9e-9e8b-2b1161c6cc31 None admin] RBAC: Authorizing `identity:get_project(project_id=90718391b09e493bb296767dc8a3c0ce)` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 10:41:10.574423 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bbd7dcd-999e-4f9e-9e8b-2b1161c6cc31 None admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 10:41:10.576435 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 78/147] 10.222.0.25 () {56 vars in 1186 bytes} [Sun Aug 7 10:41:10 2022] GET /identity/v3/projects/90718391b09e493bb296767dc8a3c0ce => generated 298 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 10:41:10.582393 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d248ae56-6345-4fd5-b15f-928e154a7ff9 None admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 10:41:10.690654 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:INSERT=1,keystone:SELECT=99 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 10:41:10.739733 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d248ae56-6345-4fd5-b15f-928e154a7ff9 None admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 10:41:10.787936 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e7689ba-74fd-4fb7-a85d-4da7fa937e61 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 10:41:10.788617 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e7689ba-74fd-4fb7-a85d-4da7fa937e61 None admin] REQUEST_METHOD: `PUT` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:41:10.788897 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e7689ba-74fd-4fb7-a85d-4da7fa937e61 None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:41:10.789174 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e7689ba-74fd-4fb7-a85d-4da7fa937e61 None admin] PATH_INFO: `/v3/projects/90718391b09e493bb296767dc8a3c0ce/users/b1626c83857e4f2091030226ccd3ca7e/roles/567b9b8669874c9986d68642dc9ef845` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:41:10.791156 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e7689ba-74fd-4fb7-a85d-4da7fa937e61 None admin] RBAC: Authorizing `identity:create_grant(project_id=90718391b09e493bb296767dc8a3c0ce, user_id=b1626c83857e4f2091030226ccd3ca7e, role_id=567b9b8669874c9986d68642dc9ef845)` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 10:41:10.793495 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e7689ba-74fd-4fb7-a85d-4da7fa937e61 None admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 10:41:10.811114 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 70/148] 10.222.0.25 () {58 vars in 1439 bytes} [Sun Aug 7 10:41:10 2022] PUT /identity/v3/projects/90718391b09e493bb296767dc8a3c0ce/users/b1626c83857e4f2091030226ccd3ca7e/roles/567b9b8669874c9986d68642dc9ef845 => generated 0 bytes in 229 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 07 10:41:11.295781 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff724ef7-d05a-4cf8-857c-21a12ca87c76 None None] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:41:11.296283 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff724ef7-d05a-4cf8-857c-21a12ca87c76 None None] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:41:11.296357 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff724ef7-d05a-4cf8-857c-21a12ca87c76 None None] PATH_INFO: `/` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:41:11.298034 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 79/149] 10.222.0.25 () {54 vars in 927 bytes} [Sun Aug 7 10:41:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 07 10:41:11.302974 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20a96596-0a1d-4bc7-ba2f-5f0dea44c054 None None] REQUEST_METHOD: `POST` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:41:11.303273 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20a96596-0a1d-4bc7-ba2f-5f0dea44c054 None None] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:41:11.303557 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20a96596-0a1d-4bc7-ba2f-5f0dea44c054 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:41:11.327965 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.auth.core [None req-20a96596-0a1d-4bc7-ba2f-5f0dea44c054 None None] MFA Rules not processed for user `bac4e6b76fe64217ac8fa44e71cffe6a`. Rule list: `[]` (Enabled: `True`). {{(pid=80143) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 10:41:11.329238 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-20a96596-0a1d-4bc7-ba2f-5f0dea44c054 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 10:41:11.418141 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 71/150] 10.222.0.25 () {58 vars in 1024 bytes} [Sun Aug 7 10:41:11 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 07 10:41:11.452825 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff724ef7-d05a-4cf8-857c-21a12ca87c76 None None] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 10:41:11.583033 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff724ef7-d05a-4cf8-857c-21a12ca87c76 None None] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 10:41:11.626828 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a3a830d-a639-4ab1-8247-03cc00c01766 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 10:41:11.627492 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a3a830d-a639-4ab1-8247-03cc00c01766 None admin] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:41:11.627779 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a3a830d-a639-4ab1-8247-03cc00c01766 None admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:41:11.628117 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a3a830d-a639-4ab1-8247-03cc00c01766 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:41:11.642961 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a3a830d-a639-4ab1-8247-03cc00c01766 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 07 10:41:11.649369 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application [None req-2a3a830d-a639-4ab1-8247-03cc00c01766 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 07 10:41:11.649369 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 07 10:41:11.649369 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 07 10:41:11.649369 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 07 10:41:11.649369 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 07 10:41:11.649369 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 07 10:41:11.649369 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 07 10:41:11.649369 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 07 10:41:11.649369 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 07 10:41:11.649369 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 07 10:41:11.649369 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 07 10:41:11.649369 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 07 10:41:11.649369 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 07 10:41:11.649369 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 07 10:41:11.649369 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 07 10:41:11.649369 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 07 10:41:11.650355 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 07 10:41:11.650355 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 07 10:41:11.650355 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 07 10:41:11.650355 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 07 10:41:11.650355 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 07 10:41:11.650355 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 07 10:41:11.650355 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 07 10:41:11.650355 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 07 10:41:11.650355 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 07 10:41:11.650355 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self.get_or_create( Aug 07 10:41:11.650355 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 07 10:41:11.650355 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application with Lock( Aug 07 10:41:11.650355 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 07 10:41:11.650355 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self._enter() Aug 07 10:41:11.650355 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 07 10:41:11.650355 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 07 10:41:11.651491 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 07 10:41:11.651491 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self.creator() Aug 07 10:41:11.651491 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 07 10:41:11.651491 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application created_value = creator( Aug 07 10:41:11.651491 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 07 10:41:11.651491 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 07 10:41:11.651491 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 07 10:41:11.651491 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 07 10:41:11.651491 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 07 10:41:11.651491 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 07 10:41:11.651491 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 07 10:41:11.651491 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application Aug 07 10:41:11.651491 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 80/151] 10.222.0.25 () {56 vars in 1096 bytes} [Sun Aug 7 10:41:11 2022] GET /identity/v3/roles/admin => generated 83 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 07 10:41:11.657784 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20a96596-0a1d-4bc7-ba2f-5f0dea44c054 None None] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 10:41:11.753329 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 10:41:11.772819 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20a96596-0a1d-4bc7-ba2f-5f0dea44c054 None None] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 10:41:11.810217 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be01deb9-dbd2-44f5-bbc7-3bc334288b5b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 10:41:11.810847 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be01deb9-dbd2-44f5-bbc7-3bc334288b5b None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:41:11.811162 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be01deb9-dbd2-44f5-bbc7-3bc334288b5b None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:41:11.811442 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be01deb9-dbd2-44f5-bbc7-3bc334288b5b None admin] PATH_INFO: `/v3/roles` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:41:11.812127 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be01deb9-dbd2-44f5-bbc7-3bc334288b5b None admin] RBAC: Adding query filter params (name=admin) {{(pid=80143) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 10:41:11.812438 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be01deb9-dbd2-44f5-bbc7-3bc334288b5b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 10:41:11.813838 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be01deb9-dbd2-44f5-bbc7-3bc334288b5b None admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 10:41:11.822844 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 72/152] 10.222.0.25 () {56 vars in 1099 bytes} [Sun Aug 7 10:41:11 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 10:41:11.828533 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a3a830d-a639-4ab1-8247-03cc00c01766 None admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 10:41:11.892245 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=100 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 10:41:11.945905 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a3a830d-a639-4ab1-8247-03cc00c01766 None admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 10:41:11.982167 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ffdf480-5148-4202-9e28-0caa3d2d861a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 10:41:11.982814 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ffdf480-5148-4202-9e28-0caa3d2d861a None admin] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:41:11.983125 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ffdf480-5148-4202-9e28-0caa3d2d861a None admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:41:11.983408 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ffdf480-5148-4202-9e28-0caa3d2d861a None admin] PATH_INFO: `/v3/users/bac4e6b76fe64217ac8fa44e71cffe6a` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:41:11.987628 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ffdf480-5148-4202-9e28-0caa3d2d861a None admin] RBAC: Authorizing `identity:get_user(user_id=bac4e6b76fe64217ac8fa44e71cffe6a)` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 10:41:11.989100 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ffdf480-5148-4202-9e28-0caa3d2d861a None admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 10:41:11.990870 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 81/153] 10.222.0.25 () {56 vars in 1177 bytes} [Sun Aug 7 10:41:11 2022] GET /identity/v3/users/bac4e6b76fe64217ac8fa44e71cffe6a => generated 247 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 10:41:11.997173 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be01deb9-dbd2-44f5-bbc7-3bc334288b5b None admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 10:41:12.108918 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be01deb9-dbd2-44f5-bbc7-3bc334288b5b None admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 10:41:12.124423 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=100 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 10:41:12.129618 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c979a461-f42c-44cd-95cf-2ecf8776013f None None] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:41:12.129946 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c979a461-f42c-44cd-95cf-2ecf8776013f None None] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:41:12.130178 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c979a461-f42c-44cd-95cf-2ecf8776013f None None] PATH_INFO: `/` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:41:12.131622 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 82/154] 10.222.0.25 () {54 vars in 927 bytes} [Sun Aug 7 10:41:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 07 10:41:12.143289 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10a7d67d-69d6-4c6a-930f-6a31f328ae4c None None] REQUEST_METHOD: `POST` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:41:12.143531 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10a7d67d-69d6-4c6a-930f-6a31f328ae4c None None] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:41:12.143775 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10a7d67d-69d6-4c6a-930f-6a31f328ae4c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:41:12.168229 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.auth.core [None req-10a7d67d-69d6-4c6a-930f-6a31f328ae4c None None] MFA Rules not processed for user `bac4e6b76fe64217ac8fa44e71cffe6a`. Rule list: `[]` (Enabled: `True`). {{(pid=80144) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 10:41:12.169430 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.fernet_utils [None req-10a7d67d-69d6-4c6a-930f-6a31f328ae4c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80144) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 10:41:12.173366 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fe6d4ec-0e59-444a-99f8-7cde64a4b1ea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 10:41:12.174004 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fe6d4ec-0e59-444a-99f8-7cde64a4b1ea None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:41:12.174287 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fe6d4ec-0e59-444a-99f8-7cde64a4b1ea None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:41:12.174561 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fe6d4ec-0e59-444a-99f8-7cde64a4b1ea None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:41:12.248483 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fe6d4ec-0e59-444a-99f8-7cde64a4b1ea None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 10:41:12.250000 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fe6d4ec-0e59-444a-99f8-7cde64a4b1ea None admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 10:41:12.259083 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 83/155] 10.222.0.25 () {58 vars in 1024 bytes} [Sun Aug 7 10:41:12 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 07 10:41:12.321378 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10a7d67d-69d6-4c6a-930f-6a31f328ae4c None None] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 10:41:12.367547 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 73/156] 10.222.0.25 () {58 vars in 1282 bytes} [Sun Aug 7 10:41:11 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 372 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 07 10:41:12.373922 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fe6d4ec-0e59-444a-99f8-7cde64a4b1ea None admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 10:41:12.400001 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=100 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 10:41:12.406869 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=100 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 10:41:12.508830 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10a7d67d-69d6-4c6a-930f-6a31f328ae4c None None] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 10:41:12.578874 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09ffeff6-20ce-4f91-bab0-453d286257e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 10:41:12.579543 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09ffeff6-20ce-4f91-bab0-453d286257e5 None admin] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:41:12.579825 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09ffeff6-20ce-4f91-bab0-453d286257e5 None admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:41:12.580101 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09ffeff6-20ce-4f91-bab0-453d286257e5 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:41:12.597938 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fe6d4ec-0e59-444a-99f8-7cde64a4b1ea None admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 10:41:12.604210 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09ffeff6-20ce-4f91-bab0-453d286257e5 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 07 10:41:12.614284 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application [None req-09ffeff6-20ce-4f91-bab0-453d286257e5 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 07 10:41:12.614284 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 07 10:41:12.614284 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 07 10:41:12.614284 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 07 10:41:12.614284 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 07 10:41:12.614284 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 07 10:41:12.614284 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 07 10:41:12.614284 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 07 10:41:12.614284 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 07 10:41:12.614284 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 07 10:41:12.614284 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 07 10:41:12.614284 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 07 10:41:12.614284 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 07 10:41:12.614284 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 07 10:41:12.614284 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 07 10:41:12.614284 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 07 10:41:12.615459 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 07 10:41:12.615459 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 07 10:41:12.615459 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 07 10:41:12.615459 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 07 10:41:12.615459 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 07 10:41:12.615459 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 07 10:41:12.615459 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 07 10:41:12.615459 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 07 10:41:12.615459 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 07 10:41:12.615459 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self.get_or_create( Aug 07 10:41:12.615459 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 07 10:41:12.615459 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application with Lock( Aug 07 10:41:12.615459 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 07 10:41:12.615459 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self._enter() Aug 07 10:41:12.615459 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 07 10:41:12.615459 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 07 10:41:12.616327 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 07 10:41:12.616327 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self.creator() Aug 07 10:41:12.616327 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 07 10:41:12.616327 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application created_value = creator( Aug 07 10:41:12.616327 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 07 10:41:12.616327 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 07 10:41:12.616327 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 07 10:41:12.616327 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 07 10:41:12.616327 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 07 10:41:12.616327 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 07 10:41:12.616327 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 07 10:41:12.616327 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application Aug 07 10:41:12.616327 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 84/157] 10.222.0.25 () {56 vars in 1099 bytes} [Sun Aug 7 10:41:12 2022] GET /identity/v3/roles/member => generated 84 bytes in 295 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 07 10:41:12.628222 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09ffeff6-20ce-4f91-bab0-453d286257e5 None admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 10:41:12.681588 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e389ac99-576b-4caf-be73-dcc0febf98b6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 10:41:12.684138 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e389ac99-576b-4caf-be73-dcc0febf98b6 None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:41:12.684402 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e389ac99-576b-4caf-be73-dcc0febf98b6 None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:41:12.684681 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e389ac99-576b-4caf-be73-dcc0febf98b6 None admin] PATH_INFO: `/v3/projects/90718391b09e493bb296767dc8a3c0ce` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:41:12.689517 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e389ac99-576b-4caf-be73-dcc0febf98b6 None admin] RBAC: Authorizing `identity:get_project(project_id=90718391b09e493bb296767dc8a3c0ce)` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 10:41:12.691012 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e389ac99-576b-4caf-be73-dcc0febf98b6 None admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 10:41:12.696258 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 74/158] 10.222.0.25 () {56 vars in 1186 bytes} [Sun Aug 7 10:41:12 2022] GET /identity/v3/projects/90718391b09e493bb296767dc8a3c0ce => generated 298 bytes in 323 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 07 10:41:12.702979 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e389ac99-576b-4caf-be73-dcc0febf98b6 None admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 10:41:12.786104 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09ffeff6-20ce-4f91-bab0-453d286257e5 None admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 10:41:12.801201 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=100 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 10:41:12.816186 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=100 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 10:41:12.828364 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd23bf32-0c48-4206-be58-2484be7b7a61 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 10:41:12.829689 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd23bf32-0c48-4206-be58-2484be7b7a61 None admin] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:41:12.830229 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd23bf32-0c48-4206-be58-2484be7b7a61 None admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:41:12.830710 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd23bf32-0c48-4206-be58-2484be7b7a61 None admin] PATH_INFO: `/v3/roles` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:41:12.831712 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd23bf32-0c48-4206-be58-2484be7b7a61 None admin] RBAC: Adding query filter params (name=member) {{(pid=80144) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 10:41:12.832310 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd23bf32-0c48-4206-be58-2484be7b7a61 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 10:41:12.833962 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd23bf32-0c48-4206-be58-2484be7b7a61 None admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 10:41:12.835046 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e389ac99-576b-4caf-be73-dcc0febf98b6 None admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 10:41:12.844779 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 85/159] 10.222.0.25 () {56 vars in 1101 bytes} [Sun Aug 7 10:41:12 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 217 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 10:41:12.846437 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-517845cb-860b-4427-8eff-95c5f0f6a60f None None] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:41:12.846956 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-517845cb-860b-4427-8eff-95c5f0f6a60f None None] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:41:12.847497 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-517845cb-860b-4427-8eff-95c5f0f6a60f None None] PATH_INFO: `/` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:41:12.848907 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 86/160] 10.222.0.25 () {54 vars in 927 bytes} [Sun Aug 7 10:41:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 07 10:41:12.850066 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-517845cb-860b-4427-8eff-95c5f0f6a60f None None] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 10:41:12.888075 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60585f82-d9d7-4469-958e-a2d40235a629 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 10:41:12.888698 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60585f82-d9d7-4469-958e-a2d40235a629 None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:41:12.889036 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60585f82-d9d7-4469-958e-a2d40235a629 None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:41:12.889316 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60585f82-d9d7-4469-958e-a2d40235a629 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:41:12.890105 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60585f82-d9d7-4469-958e-a2d40235a629 None admin] RBAC: Adding query filter params (role.id=81b1f1bcee2a473088fbbd327c1c5b23, scope.project.id=90718391b09e493bb296767dc8a3c0ce, user.id=bac4e6b76fe64217ac8fa44e71cffe6a) {{(pid=80143) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 10:41:12.890421 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60585f82-d9d7-4469-958e-a2d40235a629 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 10:41:12.892017 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60585f82-d9d7-4469-958e-a2d40235a629 None admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 10:41:12.903442 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 75/161] 10.222.0.25 () {56 vars in 1374 bytes} [Sun Aug 7 10:41:12 2022] GET /identity/v3/role_assignments?user.id=bac4e6b76fe64217ac8fa44e71cffe6a&scope.project.id=90718391b09e493bb296767dc8a3c0ce&role.id=81b1f1bcee2a473088fbbd327c1c5b23 => generated 608 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 10:41:12.904769 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c21c09b0-11ef-4a59-89d0-bca3a2755ecb None None] REQUEST_METHOD: `POST` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:41:12.905115 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c21c09b0-11ef-4a59-89d0-bca3a2755ecb None None] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:41:12.905402 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c21c09b0-11ef-4a59-89d0-bca3a2755ecb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:41:12.932430 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.auth.core [None req-c21c09b0-11ef-4a59-89d0-bca3a2755ecb None None] MFA Rules not processed for user `bac4e6b76fe64217ac8fa44e71cffe6a`. Rule list: `[]` (Enabled: `True`). {{(pid=80143) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 10:41:12.933988 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-c21c09b0-11ef-4a59-89d0-bca3a2755ecb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 10:41:12.969789 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-517845cb-860b-4427-8eff-95c5f0f6a60f None None] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 10:41:13.013651 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cbf7d2f-8dd8-467a-a065-967d3813392d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 10:41:13.014758 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cbf7d2f-8dd8-467a-a065-967d3813392d None admin] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:41:13.015295 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cbf7d2f-8dd8-467a-a065-967d3813392d None admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:41:13.015764 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cbf7d2f-8dd8-467a-a065-967d3813392d None admin] PATH_INFO: `/v3/users/cfdc7e8b78b84c6aa44d115a396152bd` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:41:13.019658 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cbf7d2f-8dd8-467a-a065-967d3813392d None admin] RBAC: Authorizing `identity:get_user(user_id=cfdc7e8b78b84c6aa44d115a396152bd)` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 10:41:13.021544 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cbf7d2f-8dd8-467a-a065-967d3813392d None admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 10:41:13.023477 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 87/162] 10.222.0.25 () {56 vars in 1177 bytes} [Sun Aug 7 10:41:12 2022] GET /identity/v3/users/cfdc7e8b78b84c6aa44d115a396152bd => generated 275 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 07 10:41:13.027033 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-763201f9-4c62-4e0a-ac66-4d959606b9e9 None None] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:41:13.027546 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-763201f9-4c62-4e0a-ac66-4d959606b9e9 None None] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:41:13.028035 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-763201f9-4c62-4e0a-ac66-4d959606b9e9 None None] PATH_INFO: `/` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:41:13.028840 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 76/163] 10.222.0.25 () {58 vars in 1024 bytes} [Sun Aug 7 10:41:12 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 07 10:41:13.029676 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 88/164] 10.222.0.25 () {54 vars in 927 bytes} [Sun Aug 7 10:41:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 07 10:41:13.035246 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-642e70c8-f42a-462b-ac65-b163e4de0280 None None] REQUEST_METHOD: `POST` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:41:13.035536 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-642e70c8-f42a-462b-ac65-b163e4de0280 None None] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:41:13.035863 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-642e70c8-f42a-462b-ac65-b163e4de0280 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:41:13.040412 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-763201f9-4c62-4e0a-ac66-4d959606b9e9 None None] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 10:41:13.070177 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.auth.core [None req-642e70c8-f42a-462b-ac65-b163e4de0280 None None] MFA Rules not processed for user `bac4e6b76fe64217ac8fa44e71cffe6a`. Rule list: `[]` (Enabled: `True`). {{(pid=80143) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 10:41:13.074744 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-642e70c8-f42a-462b-ac65-b163e4de0280 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 10:41:13.158418 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=100 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 10:41:13.177153 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 77/165] 10.222.0.25 () {58 vars in 1024 bytes} [Sun Aug 7 10:41:13 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 143 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 07 10:41:13.178632 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-642e70c8-f42a-462b-ac65-b163e4de0280 None None] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 10:41:13.193882 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=100 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 10:41:13.285016 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-763201f9-4c62-4e0a-ac66-4d959606b9e9 None None] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 10:41:13.348694 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9e7457f-a905-47ae-a5a0-53607fbee423 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 10:41:13.349305 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9e7457f-a905-47ae-a5a0-53607fbee423 None admin] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:41:13.349576 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9e7457f-a905-47ae-a5a0-53607fbee423 None admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:41:13.349850 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9e7457f-a905-47ae-a5a0-53607fbee423 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:41:13.377586 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-642e70c8-f42a-462b-ac65-b163e4de0280 None None] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 10:41:13.444276 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4861f33a-383a-4672-b7d1-60b1f17746e8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 10:41:13.444848 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4861f33a-383a-4672-b7d1-60b1f17746e8 None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:41:13.445130 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4861f33a-383a-4672-b7d1-60b1f17746e8 None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:41:13.445402 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4861f33a-383a-4672-b7d1-60b1f17746e8 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:41:13.469464 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9e7457f-a905-47ae-a5a0-53607fbee423 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 10:41:13.470622 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4861f33a-383a-4672-b7d1-60b1f17746e8 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 07 10:41:13.471102 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9e7457f-a905-47ae-a5a0-53607fbee423 None admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 10:41:13.480071 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application [None req-4861f33a-383a-4672-b7d1-60b1f17746e8 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 07 10:41:13.480071 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 07 10:41:13.480071 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 07 10:41:13.480071 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 07 10:41:13.480071 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 07 10:41:13.480071 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 07 10:41:13.480071 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 07 10:41:13.480071 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 07 10:41:13.480071 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 07 10:41:13.480071 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 07 10:41:13.480071 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 07 10:41:13.480071 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 07 10:41:13.480071 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 07 10:41:13.480071 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 07 10:41:13.480071 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 07 10:41:13.480071 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 07 10:41:13.481369 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 07 10:41:13.481369 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 07 10:41:13.481369 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 07 10:41:13.481369 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 07 10:41:13.481369 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 07 10:41:13.481369 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 07 10:41:13.481369 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 07 10:41:13.481369 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 07 10:41:13.481369 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 07 10:41:13.481369 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.get_or_create( Aug 07 10:41:13.481369 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 07 10:41:13.481369 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application with Lock( Aug 07 10:41:13.481369 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 07 10:41:13.481369 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self._enter() Aug 07 10:41:13.481369 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 07 10:41:13.481369 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 07 10:41:13.482392 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 07 10:41:13.482392 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.creator() Aug 07 10:41:13.482392 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 07 10:41:13.482392 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application created_value = creator( Aug 07 10:41:13.482392 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 07 10:41:13.482392 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 07 10:41:13.482392 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 07 10:41:13.482392 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 07 10:41:13.482392 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 07 10:41:13.482392 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 07 10:41:13.482392 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 07 10:41:13.482392 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application Aug 07 10:41:13.484837 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 78/166] 10.222.0.25 () {56 vars in 1114 bytes} [Sun Aug 7 10:41:13 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 307 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 07 10:41:13.485831 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4861f33a-383a-4672-b7d1-60b1f17746e8 None admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 10:41:13.534907 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=100 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 10:41:13.568312 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=100 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 10:41:13.607328 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4861f33a-383a-4672-b7d1-60b1f17746e8 None admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 10:41:13.644058 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d52243a8-779b-4f1f-bd97-c582b12c434f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 10:41:13.645043 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d52243a8-779b-4f1f-bd97-c582b12c434f None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:41:13.645509 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d52243a8-779b-4f1f-bd97-c582b12c434f None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:41:13.645966 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d52243a8-779b-4f1f-bd97-c582b12c434f None admin] PATH_INFO: `/v3/roles/member` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:41:13.653765 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 89/167] 10.222.0.25 () {58 vars in 1282 bytes} [Sun Aug 7 10:41:13 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 614 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 07 10:41:13.654540 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9e7457f-a905-47ae-a5a0-53607fbee423 None admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 10:41:13.661256 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d52243a8-779b-4f1f-bd97-c582b12c434f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 07 10:41:13.666300 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application [None req-d52243a8-779b-4f1f-bd97-c582b12c434f None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 07 10:41:13.666300 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 07 10:41:13.666300 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 07 10:41:13.666300 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 07 10:41:13.666300 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 07 10:41:13.666300 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 07 10:41:13.666300 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 07 10:41:13.666300 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 07 10:41:13.666300 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 07 10:41:13.666300 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 07 10:41:13.666300 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 07 10:41:13.666300 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 07 10:41:13.666300 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 07 10:41:13.666300 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 07 10:41:13.666300 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 07 10:41:13.666300 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 07 10:41:13.667174 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 07 10:41:13.667174 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 07 10:41:13.667174 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 07 10:41:13.667174 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 07 10:41:13.667174 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 07 10:41:13.667174 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 07 10:41:13.667174 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 07 10:41:13.667174 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 07 10:41:13.667174 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 07 10:41:13.667174 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.get_or_create( Aug 07 10:41:13.667174 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 07 10:41:13.667174 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application with Lock( Aug 07 10:41:13.667174 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 07 10:41:13.667174 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self._enter() Aug 07 10:41:13.667174 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 07 10:41:13.667174 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 07 10:41:13.668795 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 07 10:41:13.668795 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.creator() Aug 07 10:41:13.668795 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 07 10:41:13.668795 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application created_value = creator( Aug 07 10:41:13.668795 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 07 10:41:13.668795 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 07 10:41:13.668795 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 07 10:41:13.668795 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 07 10:41:13.668795 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 07 10:41:13.668795 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 07 10:41:13.668795 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 07 10:41:13.668795 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application Aug 07 10:41:13.675682 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 79/168] 10.222.0.25 () {56 vars in 1099 bytes} [Sun Aug 7 10:41:13 2022] GET /identity/v3/roles/member => generated 84 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 07 10:41:13.676718 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d52243a8-779b-4f1f-bd97-c582b12c434f None admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 10:41:13.790208 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d52243a8-779b-4f1f-bd97-c582b12c434f None admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 10:41:13.823542 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=100 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 10:41:13.828534 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db1f0baa-5206-40ce-8e88-f7eb532c2a6a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 10:41:13.829403 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db1f0baa-5206-40ce-8e88-f7eb532c2a6a None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:41:13.829865 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db1f0baa-5206-40ce-8e88-f7eb532c2a6a None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:41:13.830323 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db1f0baa-5206-40ce-8e88-f7eb532c2a6a None admin] PATH_INFO: `/v3/projects/90718391b09e493bb296767dc8a3c0ce` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:41:13.831800 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db1f0baa-5206-40ce-8e88-f7eb532c2a6a None admin] RBAC: Authorizing `identity:get_project(project_id=90718391b09e493bb296767dc8a3c0ce)` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 10:41:13.833454 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db1f0baa-5206-40ce-8e88-f7eb532c2a6a None admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 10:41:13.835311 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 80/169] 10.222.0.25 () {56 vars in 1186 bytes} [Sun Aug 7 10:41:13 2022] GET /identity/v3/projects/90718391b09e493bb296767dc8a3c0ce => generated 298 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 10:41:13.836402 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db1f0baa-5206-40ce-8e88-f7eb532c2a6a None admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 10:41:13.855869 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9e7457f-a905-47ae-a5a0-53607fbee423 None admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 10:41:13.910772 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c9f52a6-c579-4e38-8b9a-b3babcbe8c8c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 10:41:13.911425 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c9f52a6-c579-4e38-8b9a-b3babcbe8c8c None admin] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:41:13.911713 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c9f52a6-c579-4e38-8b9a-b3babcbe8c8c None admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:41:13.912022 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c9f52a6-c579-4e38-8b9a-b3babcbe8c8c None admin] PATH_INFO: `/v3/roles` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:41:13.912656 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c9f52a6-c579-4e38-8b9a-b3babcbe8c8c None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80144) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 10:41:13.912964 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c9f52a6-c579-4e38-8b9a-b3babcbe8c8c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 10:41:13.914340 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c9f52a6-c579-4e38-8b9a-b3babcbe8c8c None admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 10:41:13.931791 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 90/170] 10.222.0.25 () {56 vars in 1111 bytes} [Sun Aug 7 10:41:13 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 277 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 10:41:13.932638 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c9f52a6-c579-4e38-8b9a-b3babcbe8c8c None admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 10:41:13.939256 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=100 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 10:41:13.952255 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db1f0baa-5206-40ce-8e88-f7eb532c2a6a None admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 10:41:13.991519 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b992cf0-87db-489f-833b-e557c7137827 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 10:41:13.992545 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b992cf0-87db-489f-833b-e557c7137827 None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:41:13.993000 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b992cf0-87db-489f-833b-e557c7137827 None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:41:13.993612 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b992cf0-87db-489f-833b-e557c7137827 None admin] PATH_INFO: `/v3/roles` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:41:13.994477 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b992cf0-87db-489f-833b-e557c7137827 None admin] RBAC: Adding query filter params (name=member) {{(pid=80143) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 10:41:13.994971 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b992cf0-87db-489f-833b-e557c7137827 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 10:41:13.996974 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b992cf0-87db-489f-833b-e557c7137827 None admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 10:41:14.006201 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 81/171] 10.222.0.25 () {56 vars in 1101 bytes} [Sun Aug 7 10:41:13 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 10:41:14.007392 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b992cf0-87db-489f-833b-e557c7137827 None admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 10:41:14.081356 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=100 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 10:41:14.093798 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c9f52a6-c579-4e38-8b9a-b3babcbe8c8c None admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 10:41:14.130737 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b992cf0-87db-489f-833b-e557c7137827 None admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 10:41:14.168192 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ad0720c-ab78-42de-9170-6d3415293f55 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 10:41:14.168905 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ad0720c-ab78-42de-9170-6d3415293f55 None admin] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:41:14.169203 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ad0720c-ab78-42de-9170-6d3415293f55 None admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:41:14.169480 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ad0720c-ab78-42de-9170-6d3415293f55 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:41:14.170365 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ad0720c-ab78-42de-9170-6d3415293f55 None admin] RBAC: Adding query filter params (role.id=b98dfb3b9fee4644af2887a3ac3fd2e6, scope.project.id=90718391b09e493bb296767dc8a3c0ce, user.id=cfdc7e8b78b84c6aa44d115a396152bd) {{(pid=80144) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 10:41:14.170747 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ad0720c-ab78-42de-9170-6d3415293f55 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 10:41:14.171446 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5d9ddf6-a9c3-479a-a29c-053d191bf8bd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 10:41:14.172331 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ad0720c-ab78-42de-9170-6d3415293f55 None admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 10:41:14.172543 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5d9ddf6-a9c3-479a-a29c-053d191bf8bd None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:41:14.173014 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5d9ddf6-a9c3-479a-a29c-053d191bf8bd None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:41:14.173498 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5d9ddf6-a9c3-479a-a29c-053d191bf8bd None admin] PATH_INFO: `/v3/users/cfdc7e8b78b84c6aa44d115a396152bd` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:41:14.175106 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5d9ddf6-a9c3-479a-a29c-053d191bf8bd None admin] RBAC: Authorizing `identity:get_user(user_id=cfdc7e8b78b84c6aa44d115a396152bd)` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 10:41:14.176927 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5d9ddf6-a9c3-479a-a29c-053d191bf8bd None admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 10:41:14.178727 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 82/172] 10.222.0.25 () {56 vars in 1177 bytes} [Sun Aug 7 10:41:14 2022] GET /identity/v3/users/cfdc7e8b78b84c6aa44d115a396152bd => generated 275 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 10:41:14.179290 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 91/173] 10.222.0.25 () {56 vars in 1374 bytes} [Sun Aug 7 10:41:13 2022] GET /identity/v3/role_assignments?user.id=cfdc7e8b78b84c6aa44d115a396152bd&scope.project.id=90718391b09e493bb296767dc8a3c0ce&role.id=b98dfb3b9fee4644af2887a3ac3fd2e6 => generated 608 bytes in 247 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 10:41:14.179941 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5d9ddf6-a9c3-479a-a29c-053d191bf8bd None admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 10:41:14.185872 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c9731c0-61c0-45e0-938e-0b39e8d36449 None None] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:41:14.185872 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c9731c0-61c0-45e0-938e-0b39e8d36449 None None] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:41:14.185872 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c9731c0-61c0-45e0-938e-0b39e8d36449 None None] PATH_INFO: `/` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:41:14.185872 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 92/174] 10.222.0.25 () {54 vars in 927 bytes} [Sun Aug 7 10:41:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 07 10:41:14.187590 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c9731c0-61c0-45e0-938e-0b39e8d36449 None None] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 10:41:14.274197 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=100 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 10:41:14.347342 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c9731c0-61c0-45e0-938e-0b39e8d36449 None None] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 10:41:14.392080 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5d9ddf6-a9c3-479a-a29c-053d191bf8bd None admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 10:41:14.424643 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a59c395-d1eb-4777-9357-14ac9db8a5c0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 10:41:14.425901 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a59c395-d1eb-4777-9357-14ac9db8a5c0 None admin] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:41:14.426381 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a59c395-d1eb-4777-9357-14ac9db8a5c0 None admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:41:14.426876 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a59c395-d1eb-4777-9357-14ac9db8a5c0 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:41:14.446953 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=100 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 10:41:14.469252 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86433d2e-ab28-46e2-809d-cd773052ba7f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 10:41:14.469906 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86433d2e-ab28-46e2-809d-cd773052ba7f None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:41:14.470190 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86433d2e-ab28-46e2-809d-cd773052ba7f None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:41:14.470466 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86433d2e-ab28-46e2-809d-cd773052ba7f None admin] PATH_INFO: `/v3/users/cfdc7e8b78b84c6aa44d115a396152bd` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:41:14.471832 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86433d2e-ab28-46e2-809d-cd773052ba7f None admin] RBAC: Authorizing `identity:get_user(user_id=cfdc7e8b78b84c6aa44d115a396152bd)` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 10:41:14.473293 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86433d2e-ab28-46e2-809d-cd773052ba7f None admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 10:41:14.474875 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 83/175] 10.222.0.25 () {56 vars in 1177 bytes} [Sun Aug 7 10:41:14 2022] GET /identity/v3/users/cfdc7e8b78b84c6aa44d115a396152bd => generated 275 bytes in 295 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 10:41:14.480604 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e220de23-b92e-4856-a8f4-dc71d696a5d5 None None] REQUEST_METHOD: `POST` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:41:14.480943 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e220de23-b92e-4856-a8f4-dc71d696a5d5 None None] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:41:14.481195 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e220de23-b92e-4856-a8f4-dc71d696a5d5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:41:14.524021 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.auth.core [None req-e220de23-b92e-4856-a8f4-dc71d696a5d5 None None] MFA Rules not processed for user `bac4e6b76fe64217ac8fa44e71cffe6a`. Rule list: `[]` (Enabled: `True`). {{(pid=80143) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 10:41:14.525239 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-e220de23-b92e-4856-a8f4-dc71d696a5d5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 10:41:14.551408 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a59c395-d1eb-4777-9357-14ac9db8a5c0 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 10:41:14.552902 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a59c395-d1eb-4777-9357-14ac9db8a5c0 None admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 10:41:14.641804 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 84/176] 10.222.0.25 () {58 vars in 1024 bytes} [Sun Aug 7 10:41:14 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 07 10:41:14.642599 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e220de23-b92e-4856-a8f4-dc71d696a5d5 None None] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 10:41:14.663305 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=100 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 10:41:14.723173 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 93/177] 10.222.0.25 () {58 vars in 1282 bytes} [Sun Aug 7 10:41:14 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 536 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 07 10:41:14.724040 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a59c395-d1eb-4777-9357-14ac9db8a5c0 None admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 10:41:14.831844 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e220de23-b92e-4856-a8f4-dc71d696a5d5 None None] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 10:41:14.876802 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=100 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 10:41:14.901478 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d92c21c-0609-411b-a13f-04ee374cf463 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 10:41:14.902123 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d92c21c-0609-411b-a13f-04ee374cf463 None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:41:14.902427 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d92c21c-0609-411b-a13f-04ee374cf463 None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:41:14.902702 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d92c21c-0609-411b-a13f-04ee374cf463 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:41:14.905567 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a59c395-d1eb-4777-9357-14ac9db8a5c0 None admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 10:41:14.971798 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5878d4c5-63cb-4828-8119-5db4d6a9c48d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 10:41:14.972442 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5878d4c5-63cb-4828-8119-5db4d6a9c48d None admin] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:41:14.972724 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5878d4c5-63cb-4828-8119-5db4d6a9c48d None admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:41:14.973005 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5878d4c5-63cb-4828-8119-5db4d6a9c48d None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:41:14.995014 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5878d4c5-63cb-4828-8119-5db4d6a9c48d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 07 10:41:15.000739 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application [None req-5878d4c5-63cb-4828-8119-5db4d6a9c48d None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Aug 07 10:41:15.000739 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 07 10:41:15.000739 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 07 10:41:15.000739 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 07 10:41:15.000739 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 07 10:41:15.000739 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 07 10:41:15.000739 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 07 10:41:15.000739 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 07 10:41:15.000739 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 07 10:41:15.000739 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 07 10:41:15.000739 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 07 10:41:15.000739 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 07 10:41:15.000739 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 07 10:41:15.000739 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 07 10:41:15.000739 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 07 10:41:15.000739 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 07 10:41:15.001597 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 07 10:41:15.001597 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 07 10:41:15.001597 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 07 10:41:15.001597 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 07 10:41:15.001597 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 07 10:41:15.001597 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 07 10:41:15.001597 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 07 10:41:15.001597 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 07 10:41:15.001597 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 07 10:41:15.001597 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self.get_or_create( Aug 07 10:41:15.001597 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 07 10:41:15.001597 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application with Lock( Aug 07 10:41:15.001597 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 07 10:41:15.001597 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self._enter() Aug 07 10:41:15.001597 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 07 10:41:15.001597 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 07 10:41:15.002368 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 07 10:41:15.002368 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self.creator() Aug 07 10:41:15.002368 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 07 10:41:15.002368 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application created_value = creator( Aug 07 10:41:15.002368 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 07 10:41:15.002368 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 07 10:41:15.002368 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 07 10:41:15.002368 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 07 10:41:15.002368 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 07 10:41:15.002368 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 07 10:41:15.002368 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Aug 07 10:41:15.002368 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application Aug 07 10:41:15.002368 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 94/178] 10.222.0.25 () {56 vars in 1099 bytes} [Sun Aug 7 10:41:14 2022] GET /identity/v3/roles/reader => generated 84 bytes in 278 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 07 10:41:15.003050 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5878d4c5-63cb-4828-8119-5db4d6a9c48d None admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 10:41:15.028447 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d92c21c-0609-411b-a13f-04ee374cf463 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 10:41:15.029924 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d92c21c-0609-411b-a13f-04ee374cf463 None admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 10:41:15.063302 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=100 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 10:41:15.162540 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5878d4c5-63cb-4828-8119-5db4d6a9c48d None admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 10:41:15.163146 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 85/179] 10.222.0.25 () {58 vars in 1282 bytes} [Sun Aug 7 10:41:14 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 521 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 07 10:41:15.164339 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d92c21c-0609-411b-a13f-04ee374cf463 None admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 10:41:15.192156 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=100 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 10:41:15.204518 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80f30aed-6e4b-4224-9e9b-ac5df8646042 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 10:41:15.205161 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80f30aed-6e4b-4224-9e9b-ac5df8646042 None admin] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:41:15.205445 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80f30aed-6e4b-4224-9e9b-ac5df8646042 None admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:41:15.205721 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80f30aed-6e4b-4224-9e9b-ac5df8646042 None admin] PATH_INFO: `/v3/projects/90718391b09e493bb296767dc8a3c0ce` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:41:15.206881 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80f30aed-6e4b-4224-9e9b-ac5df8646042 None admin] RBAC: Authorizing `identity:get_project(project_id=90718391b09e493bb296767dc8a3c0ce)` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 10:41:15.208557 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80f30aed-6e4b-4224-9e9b-ac5df8646042 None admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 10:41:15.210453 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 95/180] 10.222.0.25 () {56 vars in 1186 bytes} [Sun Aug 7 10:41:15 2022] GET /identity/v3/projects/90718391b09e493bb296767dc8a3c0ce => generated 298 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 10:41:15.211867 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d0484bd-2049-4587-a5f7-42fc57930337 None None] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:41:15.212158 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d0484bd-2049-4587-a5f7-42fc57930337 None None] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:41:15.212590 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d0484bd-2049-4587-a5f7-42fc57930337 None None] PATH_INFO: `/` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:41:15.215063 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 96/181] 10.222.0.25 () {54 vars in 927 bytes} [Sun Aug 7 10:41:15 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 07 10:41:15.217866 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d0484bd-2049-4587-a5f7-42fc57930337 None None] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 10:41:15.283909 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d92c21c-0609-411b-a13f-04ee374cf463 None admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 10:41:15.319373 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcecc9cd-45ec-4e5b-bbde-d089ac3dbaaf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 10:41:15.319904 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=100 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 10:41:15.320043 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcecc9cd-45ec-4e5b-bbde-d089ac3dbaaf None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:41:15.320248 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcecc9cd-45ec-4e5b-bbde-d089ac3dbaaf None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:41:15.320526 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcecc9cd-45ec-4e5b-bbde-d089ac3dbaaf None admin] PATH_INFO: `/v3/roles` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:41:15.321264 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcecc9cd-45ec-4e5b-bbde-d089ac3dbaaf None admin] RBAC: Adding query filter params (name=reader) {{(pid=80143) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 10:41:15.321568 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcecc9cd-45ec-4e5b-bbde-d089ac3dbaaf None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 10:41:15.323087 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcecc9cd-45ec-4e5b-bbde-d089ac3dbaaf None admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 10:41:15.326897 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d0484bd-2049-4587-a5f7-42fc57930337 None None] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 10:41:15.331772 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 86/182] 10.222.0.25 () {56 vars in 1101 bytes} [Sun Aug 7 10:41:15 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 10:41:15.331772 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4854ab64-6820-4a93-baaf-6b5af9858d95 None None] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:41:15.331772 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4854ab64-6820-4a93-baaf-6b5af9858d95 None None] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:41:15.331772 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4854ab64-6820-4a93-baaf-6b5af9858d95 None None] PATH_INFO: `/` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:41:15.331772 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 87/183] 10.222.0.25 () {54 vars in 927 bytes} [Sun Aug 7 10:41:15 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 07 10:41:15.331772 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4854ab64-6820-4a93-baaf-6b5af9858d95 None None] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 10:41:15.360625 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb52fa17-b092-4fdd-b597-3d2d5ea8e3d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 10:41:15.361261 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb52fa17-b092-4fdd-b597-3d2d5ea8e3d2 None admin] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:41:15.361536 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb52fa17-b092-4fdd-b597-3d2d5ea8e3d2 None admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:41:15.362060 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb52fa17-b092-4fdd-b597-3d2d5ea8e3d2 None admin] PATH_INFO: `/v3/projects/6974c88728de4be899fbc38155f70bb9` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:41:15.363183 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb52fa17-b092-4fdd-b597-3d2d5ea8e3d2 None admin] RBAC: Authorizing `identity:get_project(project_id=6974c88728de4be899fbc38155f70bb9)` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 10:41:15.364607 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb52fa17-b092-4fdd-b597-3d2d5ea8e3d2 None admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 10:41:15.366196 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 97/184] 10.222.0.25 () {56 vars in 1186 bytes} [Sun Aug 7 10:41:15 2022] GET /identity/v3/projects/6974c88728de4be899fbc38155f70bb9 => generated 312 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 10:41:15.367364 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0950cfd-c757-4006-965b-993676fcde39 None None] REQUEST_METHOD: `POST` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:41:15.367649 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0950cfd-c757-4006-965b-993676fcde39 None None] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:41:15.367924 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0950cfd-c757-4006-965b-993676fcde39 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:41:15.386018 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.auth.core [None req-d0950cfd-c757-4006-965b-993676fcde39 None None] MFA Rules not processed for user `bac4e6b76fe64217ac8fa44e71cffe6a`. Rule list: `[]` (Enabled: `True`). {{(pid=80144) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 10:41:15.387617 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.fernet_utils [None req-d0950cfd-c757-4006-965b-993676fcde39 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80144) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 10:41:15.426402 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=100 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 10:41:15.443514 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4854ab64-6820-4a93-baaf-6b5af9858d95 None None] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 10:41:15.455527 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 98/185] 10.222.0.25 () {58 vars in 1024 bytes} [Sun Aug 7 10:41:15 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 07 10:41:15.456770 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c577bfa3-5f2b-40f7-bbd1-27ab59487141 None None] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:41:15.457060 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c577bfa3-5f2b-40f7-bbd1-27ab59487141 None None] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:41:15.457335 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c577bfa3-5f2b-40f7-bbd1-27ab59487141 None None] PATH_INFO: `/` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:41:15.458553 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 99/186] 10.222.0.25 () {54 vars in 927 bytes} [Sun Aug 7 10:41:15 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 07 10:41:15.459282 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c577bfa3-5f2b-40f7-bbd1-27ab59487141 None None] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 10:41:15.477370 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9d98ca1-0426-4446-8c0a-e1240e09473f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 10:41:15.477979 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9d98ca1-0426-4446-8c0a-e1240e09473f None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:41:15.478336 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9d98ca1-0426-4446-8c0a-e1240e09473f None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:41:15.478613 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9d98ca1-0426-4446-8c0a-e1240e09473f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:41:15.479373 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9d98ca1-0426-4446-8c0a-e1240e09473f None admin] RBAC: Adding query filter params (role.id=1512d531ebe248ea8e2d7d56d754b230, scope.project.id=90718391b09e493bb296767dc8a3c0ce, user.id=cfdc7e8b78b84c6aa44d115a396152bd) {{(pid=80143) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 10:41:15.479680 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9d98ca1-0426-4446-8c0a-e1240e09473f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 10:41:15.481307 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9d98ca1-0426-4446-8c0a-e1240e09473f None admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 10:41:15.487456 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 88/187] 10.222.0.25 () {56 vars in 1374 bytes} [Sun Aug 7 10:41:15 2022] GET /identity/v3/role_assignments?user.id=cfdc7e8b78b84c6aa44d115a396152bd&scope.project.id=90718391b09e493bb296767dc8a3c0ce&role.id=1512d531ebe248ea8e2d7d56d754b230 => generated 608 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 10:41:15.488571 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a445df53-d2a0-4c5c-87c3-cb8c5a7ee945 None None] REQUEST_METHOD: `POST` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:41:15.488854 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a445df53-d2a0-4c5c-87c3-cb8c5a7ee945 None None] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:41:15.489124 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a445df53-d2a0-4c5c-87c3-cb8c5a7ee945 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:41:15.507733 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.auth.core [None req-a445df53-d2a0-4c5c-87c3-cb8c5a7ee945 None None] MFA Rules not processed for user `bac4e6b76fe64217ac8fa44e71cffe6a`. Rule list: `[]` (Enabled: `True`). {{(pid=80143) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 10:41:15.508959 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-a445df53-d2a0-4c5c-87c3-cb8c5a7ee945 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 10:41:15.582223 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=100 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 10:41:15.587646 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 89/188] 10.222.0.25 () {58 vars in 1024 bytes} [Sun Aug 7 10:41:15 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 07 10:41:15.588596 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a445df53-d2a0-4c5c-87c3-cb8c5a7ee945 None None] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 10:41:15.591977 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c577bfa3-5f2b-40f7-bbd1-27ab59487141 None None] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 10:41:15.639316 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd8f5057-5a1e-4e36-a31e-3ad2c24034a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 10:41:15.642738 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd8f5057-5a1e-4e36-a31e-3ad2c24034a7 None admin] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:41:15.642738 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd8f5057-5a1e-4e36-a31e-3ad2c24034a7 None admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:41:15.642738 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd8f5057-5a1e-4e36-a31e-3ad2c24034a7 None admin] PATH_INFO: `/v3/users/b1626c83857e4f2091030226ccd3ca7e` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:41:15.642738 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd8f5057-5a1e-4e36-a31e-3ad2c24034a7 None admin] RBAC: Authorizing `identity:get_user(user_id=b1626c83857e4f2091030226ccd3ca7e)` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 10:41:15.643729 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd8f5057-5a1e-4e36-a31e-3ad2c24034a7 None admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 10:41:15.645307 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 100/189] 10.222.0.25 () {56 vars in 1177 bytes} [Sun Aug 7 10:41:15 2022] GET /identity/v3/users/b1626c83857e4f2091030226ccd3ca7e => generated 289 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 10:41:15.646558 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cb3258b-57ab-4ea0-b26a-2c0acf136cae None None] REQUEST_METHOD: `POST` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:41:15.646843 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cb3258b-57ab-4ea0-b26a-2c0acf136cae None None] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:41:15.647139 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cb3258b-57ab-4ea0-b26a-2c0acf136cae None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:41:15.670538 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.auth.core [None req-5cb3258b-57ab-4ea0-b26a-2c0acf136cae None None] MFA Rules not processed for user `bac4e6b76fe64217ac8fa44e71cffe6a`. Rule list: `[]` (Enabled: `True`). {{(pid=80144) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 10:41:15.672172 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.fernet_utils [None req-5cb3258b-57ab-4ea0-b26a-2c0acf136cae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80144) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 10:41:15.690967 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=100 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 10:41:15.721820 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a445df53-d2a0-4c5c-87c3-cb8c5a7ee945 None None] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 10:41:15.748682 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 101/190] 10.222.0.25 () {58 vars in 1024 bytes} [Sun Aug 7 10:41:15 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 07 10:41:15.749359 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cb3258b-57ab-4ea0-b26a-2c0acf136cae None None] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 10:41:15.758371 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-616ce0d3-932c-4608-a82a-b08e2e786ab8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 10:41:15.759006 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-616ce0d3-932c-4608-a82a-b08e2e786ab8 None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:41:15.759283 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-616ce0d3-932c-4608-a82a-b08e2e786ab8 None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:41:15.759552 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-616ce0d3-932c-4608-a82a-b08e2e786ab8 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:41:15.760281 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-616ce0d3-932c-4608-a82a-b08e2e786ab8 None admin] RBAC: Adding query filter params (role.id=b98dfb3b9fee4644af2887a3ac3fd2e6, scope.project.id=6974c88728de4be899fbc38155f70bb9, user.id=cfdc7e8b78b84c6aa44d115a396152bd) {{(pid=80143) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 10:41:15.760589 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-616ce0d3-932c-4608-a82a-b08e2e786ab8 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 10:41:15.762762 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-616ce0d3-932c-4608-a82a-b08e2e786ab8 None admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 10:41:15.778955 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 90/191] 10.222.0.25 () {56 vars in 1374 bytes} [Sun Aug 7 10:41:15 2022] GET /identity/v3/role_assignments?user.id=cfdc7e8b78b84c6aa44d115a396152bd&scope.project.id=6974c88728de4be899fbc38155f70bb9&role.id=b98dfb3b9fee4644af2887a3ac3fd2e6 => generated 608 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 10:41:15.778955 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-616ce0d3-932c-4608-a82a-b08e2e786ab8 None admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 10:41:15.887351 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-616ce0d3-932c-4608-a82a-b08e2e786ab8 None admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 10:41:15.889772 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=100 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 10:41:15.899502 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cb3258b-57ab-4ea0-b26a-2c0acf136cae None None] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 10:41:15.930256 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-758b05e8-3f68-4a53-ab6a-8f896582838c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 10:41:15.930785 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-758b05e8-3f68-4a53-ab6a-8f896582838c None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:41:15.931115 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-758b05e8-3f68-4a53-ab6a-8f896582838c None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:41:15.931434 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-758b05e8-3f68-4a53-ab6a-8f896582838c None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:41:15.938826 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0159e4c-9deb-4853-8e55-0767d9f802b6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 10:41:15.939520 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0159e4c-9deb-4853-8e55-0767d9f802b6 None admin] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:41:15.939840 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0159e4c-9deb-4853-8e55-0767d9f802b6 None admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:41:15.940553 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0159e4c-9deb-4853-8e55-0767d9f802b6 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:41:15.946116 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-758b05e8-3f68-4a53-ab6a-8f896582838c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 07 10:41:15.951317 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application [None req-758b05e8-3f68-4a53-ab6a-8f896582838c None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 07 10:41:15.951317 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 07 10:41:15.951317 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 07 10:41:15.951317 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 07 10:41:15.951317 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 07 10:41:15.951317 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 07 10:41:15.951317 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 07 10:41:15.951317 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 07 10:41:15.951317 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 07 10:41:15.951317 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 07 10:41:15.951317 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 07 10:41:15.951317 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 07 10:41:15.951317 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 07 10:41:15.951317 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 07 10:41:15.951317 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 07 10:41:15.951317 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 07 10:41:15.952379 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 07 10:41:15.952379 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 07 10:41:15.952379 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 07 10:41:15.952379 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 07 10:41:15.952379 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 07 10:41:15.952379 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 07 10:41:15.952379 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 07 10:41:15.952379 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 07 10:41:15.952379 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 07 10:41:15.952379 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.get_or_create( Aug 07 10:41:15.952379 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 07 10:41:15.952379 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application with Lock( Aug 07 10:41:15.952379 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 07 10:41:15.952379 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self._enter() Aug 07 10:41:15.952379 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 07 10:41:15.952379 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 07 10:41:15.953202 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 07 10:41:15.953202 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.creator() Aug 07 10:41:15.953202 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 07 10:41:15.953202 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application created_value = creator( Aug 07 10:41:15.953202 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 07 10:41:15.953202 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 07 10:41:15.953202 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 07 10:41:15.953202 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 07 10:41:15.953202 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 07 10:41:15.953202 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 07 10:41:15.953202 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 07 10:41:15.953202 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application Aug 07 10:41:15.953202 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 91/192] 10.222.0.25 () {56 vars in 1114 bytes} [Sun Aug 7 10:41:15 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 07 10:41:15.953833 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-758b05e8-3f68-4a53-ab6a-8f896582838c None admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 10:41:15.956164 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=100 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 10:41:15.957447 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0159e4c-9deb-4853-8e55-0767d9f802b6 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 07 10:41:15.963767 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application [None req-f0159e4c-9deb-4853-8e55-0767d9f802b6 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 07 10:41:15.963767 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 07 10:41:15.963767 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 07 10:41:15.963767 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 07 10:41:15.963767 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 07 10:41:15.963767 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 07 10:41:15.963767 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 07 10:41:15.963767 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 07 10:41:15.963767 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 07 10:41:15.963767 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 07 10:41:15.963767 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 07 10:41:15.963767 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 07 10:41:15.963767 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 07 10:41:15.963767 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 07 10:41:15.963767 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 07 10:41:15.963767 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 07 10:41:15.964959 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 07 10:41:15.964959 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 07 10:41:15.964959 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 07 10:41:15.964959 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 07 10:41:15.964959 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 07 10:41:15.964959 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 07 10:41:15.964959 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 07 10:41:15.964959 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 07 10:41:15.964959 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 07 10:41:15.964959 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self.get_or_create( Aug 07 10:41:15.964959 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 07 10:41:15.964959 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application with Lock( Aug 07 10:41:15.964959 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 07 10:41:15.964959 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self._enter() Aug 07 10:41:15.964959 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 07 10:41:15.964959 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 07 10:41:15.965814 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 07 10:41:15.965814 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self.creator() Aug 07 10:41:15.965814 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 07 10:41:15.965814 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application created_value = creator( Aug 07 10:41:15.965814 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 07 10:41:15.965814 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 07 10:41:15.965814 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 07 10:41:15.965814 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 07 10:41:15.965814 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 07 10:41:15.965814 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 07 10:41:15.965814 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 07 10:41:15.965814 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application Aug 07 10:41:15.965814 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 102/193] 10.222.0.25 () {56 vars in 1096 bytes} [Sun Aug 7 10:41:15 2022] GET /identity/v3/roles/admin => generated 83 bytes in 216 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 07 10:41:15.966445 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0159e4c-9deb-4853-8e55-0767d9f802b6 None admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 10:41:16.064887 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-758b05e8-3f68-4a53-ab6a-8f896582838c None admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 10:41:16.072627 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0159e4c-9deb-4853-8e55-0767d9f802b6 None admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 10:41:16.098494 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-501511e6-bc23-4895-b417-a8af20378ef8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 10:41:16.099132 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-501511e6-bc23-4895-b417-a8af20378ef8 None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:41:16.099408 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-501511e6-bc23-4895-b417-a8af20378ef8 None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:41:16.099680 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-501511e6-bc23-4895-b417-a8af20378ef8 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:41:16.107170 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b80d3be8-10fd-4f27-b3e9-ec09363ae684 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 10:41:16.107814 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b80d3be8-10fd-4f27-b3e9-ec09363ae684 None admin] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:41:16.108136 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b80d3be8-10fd-4f27-b3e9-ec09363ae684 None admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:41:16.108446 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b80d3be8-10fd-4f27-b3e9-ec09363ae684 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:41:16.109590 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b80d3be8-10fd-4f27-b3e9-ec09363ae684 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 10:41:16.111105 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b80d3be8-10fd-4f27-b3e9-ec09363ae684 None admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 10:41:16.112990 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 103/194] 10.222.0.25 () {56 vars in 1108 bytes} [Sun Aug 7 10:41:15 2022] GET /identity/v3/domains/default => generated 199 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 10:41:16.115214 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b80d3be8-10fd-4f27-b3e9-ec09363ae684 None admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 10:41:16.137648 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=100 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 10:41:16.169512 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-501511e6-bc23-4895-b417-a8af20378ef8 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 10:41:16.171005 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-501511e6-bc23-4895-b417-a8af20378ef8 None admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 10:41:16.172824 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=100 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 10:41:16.218060 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b80d3be8-10fd-4f27-b3e9-ec09363ae684 None admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 10:41:16.250533 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc7ba4c2-1b8a-4f46-8ab8-a42a5a1eaf8f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 10:41:16.251173 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc7ba4c2-1b8a-4f46-8ab8-a42a5a1eaf8f None admin] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:41:16.251459 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc7ba4c2-1b8a-4f46-8ab8-a42a5a1eaf8f None admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:41:16.251739 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc7ba4c2-1b8a-4f46-8ab8-a42a5a1eaf8f None admin] PATH_INFO: `/v3/roles` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:41:16.252405 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc7ba4c2-1b8a-4f46-8ab8-a42a5a1eaf8f None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80144) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 10:41:16.252748 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc7ba4c2-1b8a-4f46-8ab8-a42a5a1eaf8f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 10:41:16.254125 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc7ba4c2-1b8a-4f46-8ab8-a42a5a1eaf8f None admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 10:41:16.262449 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 104/195] 10.222.0.25 () {56 vars in 1111 bytes} [Sun Aug 7 10:41:16 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 10:41:16.263273 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc7ba4c2-1b8a-4f46-8ab8-a42a5a1eaf8f None admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 10:41:16.280316 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 92/196] 10.222.0.25 () {58 vars in 1282 bytes} [Sun Aug 7 10:41:15 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 322 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 07 10:41:16.280316 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-501511e6-bc23-4895-b417-a8af20378ef8 None admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 10:41:16.346178 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=100 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 10:41:16.368949 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc7ba4c2-1b8a-4f46-8ab8-a42a5a1eaf8f None admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 10:41:16.373465 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-501511e6-bc23-4895-b417-a8af20378ef8 None admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 10:41:16.374985 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=100 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 10:41:16.402523 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bed0a1e0-ffae-41ae-b637-d1d667f5a26c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 10:41:16.403364 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bed0a1e0-ffae-41ae-b637-d1d667f5a26c None admin] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:41:16.403641 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bed0a1e0-ffae-41ae-b637-d1d667f5a26c None admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:41:16.403918 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bed0a1e0-ffae-41ae-b637-d1d667f5a26c None admin] PATH_INFO: `/v3/roles` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:41:16.404574 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bed0a1e0-ffae-41ae-b637-d1d667f5a26c None admin] RBAC: Adding query filter params (name=admin) {{(pid=80144) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 10:41:16.404906 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bed0a1e0-ffae-41ae-b637-d1d667f5a26c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 10:41:16.406272 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bed0a1e0-ffae-41ae-b637-d1d667f5a26c None admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 10:41:16.408025 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c3f873c-d066-46b5-850f-471c53066c0f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 10:41:16.408639 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c3f873c-d066-46b5-850f-471c53066c0f None admin] REQUEST_METHOD: `POST` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:41:16.408921 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c3f873c-d066-46b5-850f-471c53066c0f None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:41:16.409198 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c3f873c-d066-46b5-850f-471c53066c0f None admin] PATH_INFO: `/v3/users` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:41:16.409846 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c3f873c-d066-46b5-850f-471c53066c0f None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 10:41:16.411300 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c3f873c-d066-46b5-850f-471c53066c0f None admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 10:41:16.422125 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 105/197] 10.222.0.25 () {56 vars in 1099 bytes} [Sun Aug 7 10:41:16 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 10:41:16.422125 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bed0a1e0-ffae-41ae-b637-d1d667f5a26c None admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 10:41:16.437046 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 93/198] 10.222.0.25 () {60 vars in 1132 bytes} [Sun Aug 7 10:41:16 2022] POST /identity/v3/users => generated 297 bytes in 161 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 07 10:41:16.437872 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c3f873c-d066-46b5-850f-471c53066c0f None admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 10:41:16.535173 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bed0a1e0-ffae-41ae-b637-d1d667f5a26c None admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 10:41:16.559824 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c3f873c-d066-46b5-850f-471c53066c0f None admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 10:41:16.571109 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd77cc18-f8f7-4d5f-8e4d-9c44fb2f43a8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 10:41:16.571835 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd77cc18-f8f7-4d5f-8e4d-9c44fb2f43a8 None admin] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:41:16.572116 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd77cc18-f8f7-4d5f-8e4d-9c44fb2f43a8 None admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:41:16.572416 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd77cc18-f8f7-4d5f-8e4d-9c44fb2f43a8 None admin] PATH_INFO: `/v3/users/fccf85a112eb4361a125a2f618f5c929` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:41:16.588215 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd77cc18-f8f7-4d5f-8e4d-9c44fb2f43a8 None admin] RBAC: Authorizing `identity:get_user(user_id=fccf85a112eb4361a125a2f618f5c929)` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 10:41:16.591237 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd77cc18-f8f7-4d5f-8e4d-9c44fb2f43a8 None admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 10:41:16.592018 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 106/199] 10.222.0.25 () {56 vars in 1177 bytes} [Sun Aug 7 10:41:16 2022] GET /identity/v3/users/fccf85a112eb4361a125a2f618f5c929 => generated 283 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 10:41:16.592873 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd77cc18-f8f7-4d5f-8e4d-9c44fb2f43a8 None admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 10:41:16.596806 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=100 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 10:41:16.600886 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b0e94df-a121-45ee-aa5a-8700533ac9e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 10:41:16.601534 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b0e94df-a121-45ee-aa5a-8700533ac9e6 None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:41:16.601813 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b0e94df-a121-45ee-aa5a-8700533ac9e6 None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:41:16.602828 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b0e94df-a121-45ee-aa5a-8700533ac9e6 None admin] PATH_INFO: `/v3/projects/90718391b09e493bb296767dc8a3c0ce` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:41:16.603266 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b0e94df-a121-45ee-aa5a-8700533ac9e6 None admin] RBAC: Authorizing `identity:get_project(project_id=90718391b09e493bb296767dc8a3c0ce)` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 10:41:16.605681 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b0e94df-a121-45ee-aa5a-8700533ac9e6 None admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 10:41:16.611021 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 94/200] 10.222.0.25 () {56 vars in 1186 bytes} [Sun Aug 7 10:41:16 2022] GET /identity/v3/projects/90718391b09e493bb296767dc8a3c0ce => generated 298 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 10:41:16.611021 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b0e94df-a121-45ee-aa5a-8700533ac9e6 None admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 10:41:16.641026 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 10:41:16.718523 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd77cc18-f8f7-4d5f-8e4d-9c44fb2f43a8 None admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 10:41:16.739753 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b0e94df-a121-45ee-aa5a-8700533ac9e6 None admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 10:41:16.763097 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fddc450d-3921-4034-9db5-650201746f4d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 10:41:16.763831 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fddc450d-3921-4034-9db5-650201746f4d None admin] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:41:16.764179 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fddc450d-3921-4034-9db5-650201746f4d None admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:41:16.764521 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fddc450d-3921-4034-9db5-650201746f4d None admin] PATH_INFO: `/v3/users/fccf85a112eb4361a125a2f618f5c929` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:41:16.766227 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fddc450d-3921-4034-9db5-650201746f4d None admin] RBAC: Authorizing `identity:get_user(user_id=fccf85a112eb4361a125a2f618f5c929)` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 10:41:16.769074 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fddc450d-3921-4034-9db5-650201746f4d None admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 10:41:16.771906 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 107/201] 10.222.0.25 () {56 vars in 1177 bytes} [Sun Aug 7 10:41:16 2022] GET /identity/v3/users/fccf85a112eb4361a125a2f618f5c929 => generated 283 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 10:41:16.773351 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fddc450d-3921-4034-9db5-650201746f4d None admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 10:41:16.777269 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42bb2311-4e28-4d52-8e4b-deffc07e7be6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 10:41:16.777903 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42bb2311-4e28-4d52-8e4b-deffc07e7be6 None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:41:16.778181 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42bb2311-4e28-4d52-8e4b-deffc07e7be6 None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:41:16.778456 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42bb2311-4e28-4d52-8e4b-deffc07e7be6 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:41:16.859230 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=100 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 10:41:16.876355 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42bb2311-4e28-4d52-8e4b-deffc07e7be6 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 10:41:16.878506 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42bb2311-4e28-4d52-8e4b-deffc07e7be6 None admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 10:41:16.893469 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fddc450d-3921-4034-9db5-650201746f4d None admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 10:41:16.912586 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=100 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 10:41:16.934541 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-359d7dad-1931-4f41-9d92-dfa92f70147f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 10:41:16.935949 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-359d7dad-1931-4f41-9d92-dfa92f70147f None admin] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:41:16.936404 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-359d7dad-1931-4f41-9d92-dfa92f70147f None admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:41:16.937271 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-359d7dad-1931-4f41-9d92-dfa92f70147f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:41:16.938469 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-359d7dad-1931-4f41-9d92-dfa92f70147f None admin] RBAC: Adding query filter params (role.id=567b9b8669874c9986d68642dc9ef845, scope.project.id=90718391b09e493bb296767dc8a3c0ce, user.id=b1626c83857e4f2091030226ccd3ca7e) {{(pid=80144) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 10:41:16.938982 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-359d7dad-1931-4f41-9d92-dfa92f70147f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 10:41:16.941382 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-359d7dad-1931-4f41-9d92-dfa92f70147f None admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 10:41:16.950161 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 108/202] 10.222.0.25 () {56 vars in 1374 bytes} [Sun Aug 7 10:41:16 2022] GET /identity/v3/role_assignments?user.id=b1626c83857e4f2091030226ccd3ca7e&scope.project.id=90718391b09e493bb296767dc8a3c0ce&role.id=567b9b8669874c9986d68642dc9ef845 => generated 608 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 10:41:16.951633 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-359d7dad-1931-4f41-9d92-dfa92f70147f None admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 10:41:17.013094 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 95/203] 10.222.0.25 () {58 vars in 1282 bytes} [Sun Aug 7 10:41:16 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 406 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 07 10:41:17.018858 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42bb2311-4e28-4d52-8e4b-deffc07e7be6 None admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 10:41:17.119206 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-359d7dad-1931-4f41-9d92-dfa92f70147f None admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 10:41:17.165141 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb4e43fd-e8e1-4dd8-8df1-21ca77f868b4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 10:41:17.165141 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb4e43fd-e8e1-4dd8-8df1-21ca77f868b4 None admin] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:41:17.165423 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb4e43fd-e8e1-4dd8-8df1-21ca77f868b4 None admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:41:17.165650 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb4e43fd-e8e1-4dd8-8df1-21ca77f868b4 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:41:17.179857 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=100 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 10:41:17.183863 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42bb2311-4e28-4d52-8e4b-deffc07e7be6 None admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 10:41:17.225835 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=100 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 10:41:17.236285 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02061710-dd2b-46de-bc98-b9e3976aee8e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 10:41:17.237534 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02061710-dd2b-46de-bc98-b9e3976aee8e None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:41:17.237882 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02061710-dd2b-46de-bc98-b9e3976aee8e None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:41:17.238224 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02061710-dd2b-46de-bc98-b9e3976aee8e None admin] PATH_INFO: `/v3/projects/d88586f68b734382abb9836b290aeb54` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:41:17.240879 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02061710-dd2b-46de-bc98-b9e3976aee8e None admin] RBAC: Authorizing `identity:get_project(project_id=d88586f68b734382abb9836b290aeb54)` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 10:41:17.241106 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02061710-dd2b-46de-bc98-b9e3976aee8e None admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 10:41:17.247008 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 96/204] 10.222.0.25 () {56 vars in 1186 bytes} [Sun Aug 7 10:41:17 2022] GET /identity/v3/projects/d88586f68b734382abb9836b290aeb54 => generated 302 bytes in 225 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 10:41:17.251210 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02061710-dd2b-46de-bc98-b9e3976aee8e None admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 10:41:17.270483 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb4e43fd-e8e1-4dd8-8df1-21ca77f868b4 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 10:41:17.273075 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb4e43fd-e8e1-4dd8-8df1-21ca77f868b4 None admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 10:41:17.376227 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02061710-dd2b-46de-bc98-b9e3976aee8e None admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 10:41:17.410029 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 109/205] 10.222.0.25 () {58 vars in 1282 bytes} [Sun Aug 7 10:41:16 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 454 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 07 10:41:17.411902 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb4e43fd-e8e1-4dd8-8df1-21ca77f868b4 None admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 10:41:17.429530 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2d557a6-7c46-425b-b643-73e6391131c0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 10:41:17.430195 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2d557a6-7c46-425b-b643-73e6391131c0 None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:41:17.430479 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2d557a6-7c46-425b-b643-73e6391131c0 None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:41:17.430754 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2d557a6-7c46-425b-b643-73e6391131c0 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:41:17.431568 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2d557a6-7c46-425b-b643-73e6391131c0 None admin] RBAC: Adding query filter params (role.id=1512d531ebe248ea8e2d7d56d754b230, scope.project.id=d88586f68b734382abb9836b290aeb54, user.id=fccf85a112eb4361a125a2f618f5c929) {{(pid=80143) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 10:41:17.431881 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2d557a6-7c46-425b-b643-73e6391131c0 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 10:41:17.433544 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2d557a6-7c46-425b-b643-73e6391131c0 None admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 10:41:17.439923 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 97/206] 10.222.0.25 () {56 vars in 1374 bytes} [Sun Aug 7 10:41:17 2022] GET /identity/v3/role_assignments?user.id=fccf85a112eb4361a125a2f618f5c929&scope.project.id=d88586f68b734382abb9836b290aeb54&role.id=1512d531ebe248ea8e2d7d56d754b230 => generated 259 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 10:41:17.473249 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=100 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 10:41:17.531508 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb4e43fd-e8e1-4dd8-8df1-21ca77f868b4 None admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 10:41:17.570379 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b7338cf-64f0-4730-b320-cf6fb3131399 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 10:41:17.571057 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b7338cf-64f0-4730-b320-cf6fb3131399 None admin] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:41:17.571331 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b7338cf-64f0-4730-b320-cf6fb3131399 None admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:41:17.571603 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b7338cf-64f0-4730-b320-cf6fb3131399 None admin] PATH_INFO: `/v3/projects/d88586f68b734382abb9836b290aeb54` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:41:17.572779 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b7338cf-64f0-4730-b320-cf6fb3131399 None admin] RBAC: Authorizing `identity:get_project(project_id=d88586f68b734382abb9836b290aeb54)` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 10:41:17.575081 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b7338cf-64f0-4730-b320-cf6fb3131399 None admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 10:41:17.577206 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 110/207] 10.222.0.25 () {56 vars in 1186 bytes} [Sun Aug 7 10:41:17 2022] GET /identity/v3/projects/d88586f68b734382abb9836b290aeb54 => generated 302 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 10:41:17.582842 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2d557a6-7c46-425b-b643-73e6391131c0 None admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 10:41:17.651986 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=100 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 10:41:17.696403 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2d557a6-7c46-425b-b643-73e6391131c0 None admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 10:41:17.732943 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7eda47d5-9081-4678-b9e6-f54139422419 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 10:41:17.733578 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eda47d5-9081-4678-b9e6-f54139422419 None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:41:17.733861 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eda47d5-9081-4678-b9e6-f54139422419 None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:41:17.734138 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eda47d5-9081-4678-b9e6-f54139422419 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:41:17.734893 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7eda47d5-9081-4678-b9e6-f54139422419 None admin] RBAC: Adding query filter params (role.id=81b1f1bcee2a473088fbbd327c1c5b23, scope.project.id=d88586f68b734382abb9836b290aeb54, user.id=fccf85a112eb4361a125a2f618f5c929) {{(pid=80143) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 10:41:17.735253 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7eda47d5-9081-4678-b9e6-f54139422419 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 10:41:17.737016 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7eda47d5-9081-4678-b9e6-f54139422419 None admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 10:41:17.743607 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 98/208] 10.222.0.25 () {56 vars in 1374 bytes} [Sun Aug 7 10:41:17 2022] GET /identity/v3/role_assignments?user.id=fccf85a112eb4361a125a2f618f5c929&scope.project.id=d88586f68b734382abb9836b290aeb54&role.id=81b1f1bcee2a473088fbbd327c1c5b23 => generated 259 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 10:41:18.647787 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-552c6009-d880-4e20-81b6-3e865179c02c None None] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:41:18.648291 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-552c6009-d880-4e20-81b6-3e865179c02c None None] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:41:18.648362 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-552c6009-d880-4e20-81b6-3e865179c02c None None] PATH_INFO: `/` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:41:18.649912 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 111/209] 10.222.0.25 () {54 vars in 927 bytes} [Sun Aug 7 10:41:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 07 10:41:18.656015 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e86c9a1b-c4d8-4551-be13-c54ffa586aa6 None None] REQUEST_METHOD: `POST` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:41:18.656419 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e86c9a1b-c4d8-4551-be13-c54ffa586aa6 None None] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:41:18.656586 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e86c9a1b-c4d8-4551-be13-c54ffa586aa6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:41:18.674476 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.auth.core [None req-e86c9a1b-c4d8-4551-be13-c54ffa586aa6 None None] MFA Rules not processed for user `bac4e6b76fe64217ac8fa44e71cffe6a`. Rule list: `[]` (Enabled: `True`). {{(pid=80143) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 10:41:18.675612 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-e86c9a1b-c4d8-4551-be13-c54ffa586aa6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 10:41:18.746458 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 99/210] 10.222.0.25 () {58 vars in 1024 bytes} [Sun Aug 7 10:41:18 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 07 10:41:18.768904 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-552c6009-d880-4e20-81b6-3e865179c02c None None] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 10:41:18.799260 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-957b64d9-5da6-4be8-9dfb-089569c71100 None None] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:41:18.799556 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-957b64d9-5da6-4be8-9dfb-089569c71100 None None] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:41:18.799822 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-957b64d9-5da6-4be8-9dfb-089569c71100 None None] PATH_INFO: `/` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:41:18.801462 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 100/211] 10.222.0.25 () {54 vars in 927 bytes} [Sun Aug 7 10:41:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 07 10:41:18.806396 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0a48b67-2570-401e-a465-715467fe5fbb None None] REQUEST_METHOD: `POST` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:41:18.806681 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0a48b67-2570-401e-a465-715467fe5fbb None None] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:41:18.806977 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0a48b67-2570-401e-a465-715467fe5fbb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:41:18.826452 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.auth.core [None req-c0a48b67-2570-401e-a465-715467fe5fbb None None] MFA Rules not processed for user `bac4e6b76fe64217ac8fa44e71cffe6a`. Rule list: `[]` (Enabled: `True`). {{(pid=80143) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 10:41:18.827733 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-c0a48b67-2570-401e-a465-715467fe5fbb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 10:41:18.883986 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=100 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 10:41:18.891108 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-552c6009-d880-4e20-81b6-3e865179c02c None None] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 10:41:18.892742 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=100 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 10:41:18.906745 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 101/212] 10.222.0.25 () {58 vars in 1024 bytes} [Sun Aug 7 10:41:18 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 07 10:41:18.927905 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81c52a20-9aff-496d-badb-7edeec25da43 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 10:41:18.928469 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81c52a20-9aff-496d-badb-7edeec25da43 None admin] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:41:18.928745 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81c52a20-9aff-496d-badb-7edeec25da43 None admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:41:18.929013 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81c52a20-9aff-496d-badb-7edeec25da43 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:41:18.932852 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0a48b67-2570-401e-a465-715467fe5fbb None None] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 10:41:18.939654 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81c52a20-9aff-496d-badb-7edeec25da43 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 07 10:41:18.944393 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application [None req-81c52a20-9aff-496d-badb-7edeec25da43 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 07 10:41:18.944393 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 07 10:41:18.944393 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 07 10:41:18.944393 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 07 10:41:18.944393 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 07 10:41:18.944393 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 07 10:41:18.944393 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 07 10:41:18.944393 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 07 10:41:18.944393 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 07 10:41:18.944393 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 07 10:41:18.944393 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 07 10:41:18.944393 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 07 10:41:18.944393 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 07 10:41:18.944393 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 07 10:41:18.944393 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 07 10:41:18.944393 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 07 10:41:18.945245 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 07 10:41:18.945245 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 07 10:41:18.945245 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 07 10:41:18.945245 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 07 10:41:18.945245 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 07 10:41:18.945245 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 07 10:41:18.945245 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 07 10:41:18.945245 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 07 10:41:18.945245 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 07 10:41:18.945245 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self.get_or_create( Aug 07 10:41:18.945245 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 07 10:41:18.945245 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application with Lock( Aug 07 10:41:18.945245 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 07 10:41:18.945245 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self._enter() Aug 07 10:41:18.945245 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 07 10:41:18.945245 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 07 10:41:18.946019 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 07 10:41:18.946019 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self.creator() Aug 07 10:41:18.946019 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 07 10:41:18.946019 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application created_value = creator( Aug 07 10:41:18.946019 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 07 10:41:18.946019 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 07 10:41:18.946019 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 07 10:41:18.946019 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 07 10:41:18.946019 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 07 10:41:18.946019 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 07 10:41:18.946019 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 07 10:41:18.946019 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application Aug 07 10:41:18.946019 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 112/213] 10.222.0.25 () {56 vars in 1099 bytes} [Sun Aug 7 10:41:18 2022] GET /identity/v3/roles/member => generated 84 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 07 10:41:18.953426 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81c52a20-9aff-496d-badb-7edeec25da43 None admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 10:41:19.060110 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0a48b67-2570-401e-a465-715467fe5fbb None None] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 10:41:19.079442 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81c52a20-9aff-496d-badb-7edeec25da43 None admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 10:41:19.104412 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9475330d-bc8e-4881-a259-ce910d358f54 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 10:41:19.106392 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9475330d-bc8e-4881-a259-ce910d358f54 None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:41:19.106687 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9475330d-bc8e-4881-a259-ce910d358f54 None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:41:19.107065 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9475330d-bc8e-4881-a259-ce910d358f54 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:41:19.108875 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9475330d-bc8e-4881-a259-ce910d358f54 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 10:41:19.110872 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9475330d-bc8e-4881-a259-ce910d358f54 None admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 10:41:19.113565 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 102/214] 10.222.0.25 () {56 vars in 1108 bytes} [Sun Aug 7 10:41:18 2022] GET /identity/v3/domains/default => generated 199 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 07 10:41:19.116265 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fcee57c-c9b3-486f-a4ee-44442a6c5fe2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 10:41:19.116875 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fcee57c-c9b3-486f-a4ee-44442a6c5fe2 None admin] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:41:19.117153 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fcee57c-c9b3-486f-a4ee-44442a6c5fe2 None admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:41:19.117428 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fcee57c-c9b3-486f-a4ee-44442a6c5fe2 None admin] PATH_INFO: `/v3/roles` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:41:19.118262 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fcee57c-c9b3-486f-a4ee-44442a6c5fe2 None admin] RBAC: Adding query filter params (name=member) {{(pid=80144) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 10:41:19.118563 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fcee57c-c9b3-486f-a4ee-44442a6c5fe2 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 10:41:19.118792 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9475330d-bc8e-4881-a259-ce910d358f54 None admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 10:41:19.119933 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fcee57c-c9b3-486f-a4ee-44442a6c5fe2 None admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 10:41:19.129313 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 113/215] 10.222.0.25 () {56 vars in 1101 bytes} [Sun Aug 7 10:41:18 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 07 10:41:19.136982 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fcee57c-c9b3-486f-a4ee-44442a6c5fe2 None admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 10:41:19.170480 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=100 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 10:41:19.178502 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=100 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 10:41:19.231783 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9475330d-bc8e-4881-a259-ce910d358f54 None admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 10:41:19.268420 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fcee57c-c9b3-486f-a4ee-44442a6c5fe2 None admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 10:41:19.269194 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-047ac69e-93d1-4ee3-8a13-87b2a8bdedc4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 10:41:19.270027 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-047ac69e-93d1-4ee3-8a13-87b2a8bdedc4 None admin] REQUEST_METHOD: `POST` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:41:19.270310 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-047ac69e-93d1-4ee3-8a13-87b2a8bdedc4 None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:41:19.270577 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-047ac69e-93d1-4ee3-8a13-87b2a8bdedc4 None admin] PATH_INFO: `/v3/users` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:41:19.271636 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-047ac69e-93d1-4ee3-8a13-87b2a8bdedc4 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 10:41:19.273095 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-047ac69e-93d1-4ee3-8a13-87b2a8bdedc4 None admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 10:41:19.303427 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 103/216] 10.222.0.25 () {60 vars in 1132 bytes} [Sun Aug 7 10:41:19 2022] POST /identity/v3/users => generated 297 bytes in 185 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 07 10:41:19.323452 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6834c0e8-8fc6-4920-9437-1351aee992d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 10:41:19.324343 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6834c0e8-8fc6-4920-9437-1351aee992d4 None admin] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:41:19.324714 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6834c0e8-8fc6-4920-9437-1351aee992d4 None admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:41:19.324988 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6834c0e8-8fc6-4920-9437-1351aee992d4 None admin] PATH_INFO: `/v3/users/e01f8bc675bc4dd4985cd9f3b1bac86d` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:41:19.342257 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6834c0e8-8fc6-4920-9437-1351aee992d4 None admin] RBAC: Authorizing `identity:get_user(user_id=e01f8bc675bc4dd4985cd9f3b1bac86d)` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 10:41:19.343880 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6834c0e8-8fc6-4920-9437-1351aee992d4 None admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 10:41:19.345737 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 114/217] 10.222.0.25 () {56 vars in 1177 bytes} [Sun Aug 7 10:41:19 2022] GET /identity/v3/users/e01f8bc675bc4dd4985cd9f3b1bac86d => generated 297 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 07 10:41:19.351872 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-047ac69e-93d1-4ee3-8a13-87b2a8bdedc4 None admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 10:41:19.469358 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-047ac69e-93d1-4ee3-8a13-87b2a8bdedc4 None admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 10:41:19.477344 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 10:41:19.511718 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8601181-bcac-4277-986f-c79873e8ade9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 10:41:19.512415 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8601181-bcac-4277-986f-c79873e8ade9 None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:41:19.512701 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8601181-bcac-4277-986f-c79873e8ade9 None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:41:19.512979 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8601181-bcac-4277-986f-c79873e8ade9 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:41:19.576918 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69f4d8ca-b986-4291-98f6-4e4013f4b905 None None] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:41:19.577252 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69f4d8ca-b986-4291-98f6-4e4013f4b905 None None] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:41:19.577474 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69f4d8ca-b986-4291-98f6-4e4013f4b905 None None] PATH_INFO: `/` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:41:19.578995 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 115/218] 10.222.0.25 () {54 vars in 927 bytes} [Sun Aug 7 10:41:19 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 07 10:41:19.586832 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c69b0adc-0ad2-4833-9c91-d1512d9acec8 None None] REQUEST_METHOD: `POST` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:41:19.588044 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c69b0adc-0ad2-4833-9c91-d1512d9acec8 None None] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:41:19.588044 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c69b0adc-0ad2-4833-9c91-d1512d9acec8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:41:19.599727 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8601181-bcac-4277-986f-c79873e8ade9 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 10:41:19.602167 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8601181-bcac-4277-986f-c79873e8ade9 None admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 10:41:19.613884 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.auth.core [None req-c69b0adc-0ad2-4833-9c91-d1512d9acec8 None None] MFA Rules not processed for user `bac4e6b76fe64217ac8fa44e71cffe6a`. Rule list: `[]` (Enabled: `True`). {{(pid=80144) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 10:41:19.615585 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.fernet_utils [None req-c69b0adc-0ad2-4833-9c91-d1512d9acec8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80144) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 10:41:19.702397 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 116/219] 10.222.0.25 () {58 vars in 1024 bytes} [Sun Aug 7 10:41:19 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 07 10:41:19.724839 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c69b0adc-0ad2-4833-9c91-d1512d9acec8 None None] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 10:41:19.736601 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 104/220] 10.222.0.25 () {58 vars in 1282 bytes} [Sun Aug 7 10:41:19 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 385 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 07 10:41:19.740524 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=100 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 10:41:19.743319 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8601181-bcac-4277-986f-c79873e8ade9 None admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 10:41:19.753495 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=100 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 10:41:19.850623 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c69b0adc-0ad2-4833-9c91-d1512d9acec8 None None] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 10:41:19.880553 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8601181-bcac-4277-986f-c79873e8ade9 None admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 10:41:19.894283 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acb42ee6-fd76-46f7-9960-87669d5059b6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 10:41:19.894958 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acb42ee6-fd76-46f7-9960-87669d5059b6 None admin] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:41:19.895235 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acb42ee6-fd76-46f7-9960-87669d5059b6 None admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:41:19.895510 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acb42ee6-fd76-46f7-9960-87669d5059b6 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:41:19.907227 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acb42ee6-fd76-46f7-9960-87669d5059b6 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 07 10:41:19.912065 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application [None req-acb42ee6-fd76-46f7-9960-87669d5059b6 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 07 10:41:19.912065 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 07 10:41:19.912065 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 07 10:41:19.912065 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 07 10:41:19.912065 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 07 10:41:19.912065 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 07 10:41:19.912065 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 07 10:41:19.912065 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 07 10:41:19.912065 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 07 10:41:19.912065 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 07 10:41:19.912065 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 07 10:41:19.912065 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 07 10:41:19.912065 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 07 10:41:19.912065 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 07 10:41:19.912065 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 07 10:41:19.912065 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 07 10:41:19.914075 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 07 10:41:19.914075 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 07 10:41:19.914075 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 07 10:41:19.914075 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 07 10:41:19.914075 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 07 10:41:19.914075 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 07 10:41:19.914075 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 07 10:41:19.914075 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 07 10:41:19.914075 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 07 10:41:19.914075 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self.get_or_create( Aug 07 10:41:19.914075 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 07 10:41:19.914075 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application with Lock( Aug 07 10:41:19.914075 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 07 10:41:19.914075 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self._enter() Aug 07 10:41:19.914075 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 07 10:41:19.914075 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 07 10:41:19.922277 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 07 10:41:19.922277 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self.creator() Aug 07 10:41:19.922277 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 07 10:41:19.922277 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application created_value = creator( Aug 07 10:41:19.922277 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 07 10:41:19.922277 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 07 10:41:19.922277 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 07 10:41:19.922277 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 07 10:41:19.922277 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 07 10:41:19.922277 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 07 10:41:19.922277 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 07 10:41:19.922277 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application Aug 07 10:41:19.922277 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 117/221] 10.222.0.25 () {56 vars in 1114 bytes} [Sun Aug 7 10:41:19 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 07 10:41:19.922277 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5daa8ca9-d73b-4e76-923d-8cb027d6f518 None None] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:41:19.924109 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5daa8ca9-d73b-4e76-923d-8cb027d6f518 None None] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:41:19.924109 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5daa8ca9-d73b-4e76-923d-8cb027d6f518 None None] PATH_INFO: `/` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:41:19.924109 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 118/222] 10.222.0.25 () {54 vars in 927 bytes} [Sun Aug 7 10:41:19 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 07 10:41:19.924109 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5daa8ca9-d73b-4e76-923d-8cb027d6f518 None None] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 10:41:19.924609 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcf4f9a7-9dec-4c57-ae19-ed40566f11d1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 10:41:19.924609 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcf4f9a7-9dec-4c57-ae19-ed40566f11d1 None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:41:19.924917 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcf4f9a7-9dec-4c57-ae19-ed40566f11d1 None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:41:19.925164 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcf4f9a7-9dec-4c57-ae19-ed40566f11d1 None admin] PATH_INFO: `/v3/projects/d88586f68b734382abb9836b290aeb54` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:41:19.926761 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcf4f9a7-9dec-4c57-ae19-ed40566f11d1 None admin] RBAC: Authorizing `identity:get_project(project_id=d88586f68b734382abb9836b290aeb54)` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 10:41:19.929494 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcf4f9a7-9dec-4c57-ae19-ed40566f11d1 None admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 10:41:19.931469 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 105/223] 10.222.0.25 () {56 vars in 1186 bytes} [Sun Aug 7 10:41:19 2022] GET /identity/v3/projects/d88586f68b734382abb9836b290aeb54 => generated 302 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 07 10:41:19.935236 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f3fcf5d-326a-4795-a140-55a6c1767599 None None] REQUEST_METHOD: `POST` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:41:19.935538 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f3fcf5d-326a-4795-a140-55a6c1767599 None None] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:41:19.935817 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f3fcf5d-326a-4795-a140-55a6c1767599 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:41:19.962970 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.auth.core [None req-5f3fcf5d-326a-4795-a140-55a6c1767599 None None] MFA Rules not processed for user `bac4e6b76fe64217ac8fa44e71cffe6a`. Rule list: `[]` (Enabled: `True`). {{(pid=80143) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 10:41:19.965502 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-5f3fcf5d-326a-4795-a140-55a6c1767599 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 10:41:20.015630 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=100 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 10:41:20.049700 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5daa8ca9-d73b-4e76-923d-8cb027d6f518 None None] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 10:41:20.067869 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 106/224] 10.222.0.25 () {58 vars in 1024 bytes} [Sun Aug 7 10:41:19 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 135 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 07 10:41:20.069446 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f3fcf5d-326a-4795-a140-55a6c1767599 None None] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 10:41:20.073908 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=100 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 10:41:20.088386 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc99e941-f4fb-474d-8518-8fc8fca01ad1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 10:41:20.089870 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc99e941-f4fb-474d-8518-8fc8fca01ad1 None admin] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:41:20.090401 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc99e941-f4fb-474d-8518-8fc8fca01ad1 None admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:41:20.090753 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc99e941-f4fb-474d-8518-8fc8fca01ad1 None admin] PATH_INFO: `/v3/roles` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:41:20.091521 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc99e941-f4fb-474d-8518-8fc8fca01ad1 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80144) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 10:41:20.091828 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc99e941-f4fb-474d-8518-8fc8fca01ad1 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 10:41:20.093251 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc99e941-f4fb-474d-8518-8fc8fca01ad1 None admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 10:41:20.102030 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 119/225] 10.222.0.25 () {56 vars in 1111 bytes} [Sun Aug 7 10:41:19 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 10:41:20.108039 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc99e941-f4fb-474d-8518-8fc8fca01ad1 None admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 10:41:20.199546 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f3fcf5d-326a-4795-a140-55a6c1767599 None None] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 10:41:20.232919 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc99e941-f4fb-474d-8518-8fc8fca01ad1 None admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 10:41:20.235234 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a11c755b-147d-461d-aaf5-9eaf72433227 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 10:41:20.235918 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a11c755b-147d-461d-aaf5-9eaf72433227 None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:41:20.236283 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a11c755b-147d-461d-aaf5-9eaf72433227 None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:41:20.237551 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a11c755b-147d-461d-aaf5-9eaf72433227 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:41:20.237673 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a11c755b-147d-461d-aaf5-9eaf72433227 None admin] RBAC: Adding query filter params (role.id=b98dfb3b9fee4644af2887a3ac3fd2e6, scope.project.id=d88586f68b734382abb9836b290aeb54, user.id=e01f8bc675bc4dd4985cd9f3b1bac86d) {{(pid=80143) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 10:41:20.237991 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a11c755b-147d-461d-aaf5-9eaf72433227 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 10:41:20.239547 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a11c755b-147d-461d-aaf5-9eaf72433227 None admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 10:41:20.250959 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 107/226] 10.222.0.25 () {56 vars in 1374 bytes} [Sun Aug 7 10:41:20 2022] GET /identity/v3/role_assignments?user.id=e01f8bc675bc4dd4985cd9f3b1bac86d&scope.project.id=d88586f68b734382abb9836b290aeb54&role.id=b98dfb3b9fee4644af2887a3ac3fd2e6 => generated 259 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 10:41:20.250959 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a11c755b-147d-461d-aaf5-9eaf72433227 None admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 10:41:20.303640 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bd60119-6233-45a5-bd23-6c8024c4bb09 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 10:41:20.304210 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bd60119-6233-45a5-bd23-6c8024c4bb09 None admin] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:41:20.304485 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bd60119-6233-45a5-bd23-6c8024c4bb09 None admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:41:20.304756 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bd60119-6233-45a5-bd23-6c8024c4bb09 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:41:20.307263 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=100 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 10:41:20.342658 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bd60119-6233-45a5-bd23-6c8024c4bb09 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 07 10:41:20.353057 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application [None req-0bd60119-6233-45a5-bd23-6c8024c4bb09 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 07 10:41:20.353057 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 07 10:41:20.353057 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 07 10:41:20.353057 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 07 10:41:20.353057 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 07 10:41:20.353057 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 07 10:41:20.353057 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 07 10:41:20.353057 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 07 10:41:20.353057 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 07 10:41:20.353057 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 07 10:41:20.353057 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 07 10:41:20.353057 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 07 10:41:20.353057 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 07 10:41:20.353057 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 07 10:41:20.353057 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 07 10:41:20.353057 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 07 10:41:20.354000 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 07 10:41:20.354000 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 07 10:41:20.354000 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 07 10:41:20.354000 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 07 10:41:20.354000 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 07 10:41:20.354000 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 07 10:41:20.354000 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 07 10:41:20.354000 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 07 10:41:20.354000 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 07 10:41:20.354000 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self.get_or_create( Aug 07 10:41:20.354000 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 07 10:41:20.354000 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application with Lock( Aug 07 10:41:20.354000 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 07 10:41:20.354000 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self._enter() Aug 07 10:41:20.354000 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 07 10:41:20.354000 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 07 10:41:20.354863 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 07 10:41:20.354863 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self.creator() Aug 07 10:41:20.354863 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 07 10:41:20.354863 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application created_value = creator( Aug 07 10:41:20.354863 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 07 10:41:20.354863 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 07 10:41:20.354863 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 07 10:41:20.354863 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 07 10:41:20.354863 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 07 10:41:20.354863 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 07 10:41:20.354863 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 07 10:41:20.354863 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application Aug 07 10:41:20.358947 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 120/227] 10.222.0.25 () {56 vars in 1096 bytes} [Sun Aug 7 10:41:20 2022] GET /identity/v3/roles/admin => generated 83 bytes in 249 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 07 10:41:20.376011 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bd60119-6233-45a5-bd23-6c8024c4bb09 None admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 10:41:20.418173 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=100 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 10:41:20.462736 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a11c755b-147d-461d-aaf5-9eaf72433227 None admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 10:41:20.509964 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57a706b4-d402-4cd0-a19b-724e8824459c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 10:41:20.510602 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57a706b4-d402-4cd0-a19b-724e8824459c None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:41:20.510876 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57a706b4-d402-4cd0-a19b-724e8824459c None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:41:20.511851 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57a706b4-d402-4cd0-a19b-724e8824459c None admin] PATH_INFO: `/v3/users/fccf85a112eb4361a125a2f618f5c929` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:41:20.514950 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57a706b4-d402-4cd0-a19b-724e8824459c None admin] RBAC: Authorizing `identity:get_user(user_id=fccf85a112eb4361a125a2f618f5c929)` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 10:41:20.516392 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57a706b4-d402-4cd0-a19b-724e8824459c None admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 10:41:20.518117 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 108/228] 10.222.0.25 () {56 vars in 1177 bytes} [Sun Aug 7 10:41:20 2022] GET /identity/v3/users/fccf85a112eb4361a125a2f618f5c929 => generated 283 bytes in 272 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 10:41:20.528060 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57a706b4-d402-4cd0-a19b-724e8824459c None admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 10:41:20.567252 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bd60119-6233-45a5-bd23-6c8024c4bb09 None admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 10:41:20.632829 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0260c561-e54e-4116-869b-175698fed22f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 10:41:20.633840 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0260c561-e54e-4116-869b-175698fed22f None admin] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:41:20.634527 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0260c561-e54e-4116-869b-175698fed22f None admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:41:20.635659 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0260c561-e54e-4116-869b-175698fed22f None admin] PATH_INFO: `/v3/roles` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:41:20.637792 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0260c561-e54e-4116-869b-175698fed22f None admin] RBAC: Adding query filter params (name=admin) {{(pid=80144) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 10:41:20.639307 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0260c561-e54e-4116-869b-175698fed22f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 10:41:20.641148 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0260c561-e54e-4116-869b-175698fed22f None admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 10:41:20.657464 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 121/229] 10.222.0.25 () {56 vars in 1099 bytes} [Sun Aug 7 10:41:20 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 282 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 07 10:41:20.668841 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0260c561-e54e-4116-869b-175698fed22f None admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 10:41:20.711672 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57a706b4-d402-4cd0-a19b-724e8824459c None admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 10:41:20.748641 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=100 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 10:41:20.798720 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0260c561-e54e-4116-869b-175698fed22f None admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 10:41:20.800248 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e98e219-54be-4f9d-b3fd-b8681265dd70 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 10:41:20.800911 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e98e219-54be-4f9d-b3fd-b8681265dd70 None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:41:20.801198 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e98e219-54be-4f9d-b3fd-b8681265dd70 None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:41:20.801479 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e98e219-54be-4f9d-b3fd-b8681265dd70 None admin] PATH_INFO: `/v3/projects/d88586f68b734382abb9836b290aeb54` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:41:20.803690 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e98e219-54be-4f9d-b3fd-b8681265dd70 None admin] RBAC: Authorizing `identity:get_project(project_id=d88586f68b734382abb9836b290aeb54)` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 10:41:20.805163 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e98e219-54be-4f9d-b3fd-b8681265dd70 None admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 10:41:20.810945 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 109/230] 10.222.0.25 () {56 vars in 1186 bytes} [Sun Aug 7 10:41:20 2022] GET /identity/v3/projects/d88586f68b734382abb9836b290aeb54 => generated 302 bytes in 279 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 10:41:20.824165 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e98e219-54be-4f9d-b3fd-b8681265dd70 None admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 10:41:20.829974 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=100 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 10:41:20.850121 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd5cf142-bd51-4490-b591-3adec37097be None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 10:41:20.850758 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd5cf142-bd51-4490-b591-3adec37097be None admin] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:41:20.851083 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd5cf142-bd51-4490-b591-3adec37097be None admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:41:20.851366 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd5cf142-bd51-4490-b591-3adec37097be None admin] PATH_INFO: `/v3/users/fccf85a112eb4361a125a2f618f5c929` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:41:20.853669 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd5cf142-bd51-4490-b591-3adec37097be None admin] RBAC: Authorizing `identity:get_user(user_id=fccf85a112eb4361a125a2f618f5c929)` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 10:41:20.855219 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd5cf142-bd51-4490-b591-3adec37097be None admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 10:41:20.856985 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 122/231] 10.222.0.25 () {56 vars in 1177 bytes} [Sun Aug 7 10:41:20 2022] GET /identity/v3/users/fccf85a112eb4361a125a2f618f5c929 => generated 283 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 10:41:20.863962 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd5cf142-bd51-4490-b591-3adec37097be None admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 10:41:21.035623 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd5cf142-bd51-4490-b591-3adec37097be None admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 10:41:21.046719 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e98e219-54be-4f9d-b3fd-b8681265dd70 None admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 10:41:21.061360 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=100 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 10:41:21.072852 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88ea661e-ca1c-44ea-b020-bd8ed85f4cb6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 10:41:21.073682 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88ea661e-ca1c-44ea-b020-bd8ed85f4cb6 None admin] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:41:21.074115 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88ea661e-ca1c-44ea-b020-bd8ed85f4cb6 None admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:41:21.074474 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88ea661e-ca1c-44ea-b020-bd8ed85f4cb6 None admin] PATH_INFO: `/v3/projects/d88586f68b734382abb9836b290aeb54` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:41:21.075696 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88ea661e-ca1c-44ea-b020-bd8ed85f4cb6 None admin] RBAC: Authorizing `identity:get_project(project_id=d88586f68b734382abb9836b290aeb54)` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 10:41:21.077151 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88ea661e-ca1c-44ea-b020-bd8ed85f4cb6 None admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 10:41:21.078841 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 123/232] 10.222.0.25 () {56 vars in 1186 bytes} [Sun Aug 7 10:41:20 2022] GET /identity/v3/projects/d88586f68b734382abb9836b290aeb54 => generated 302 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 10:41:21.084866 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88ea661e-ca1c-44ea-b020-bd8ed85f4cb6 None admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 10:41:21.152295 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47a79601-026e-426e-bf46-09041b75844c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 10:41:21.152975 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47a79601-026e-426e-bf46-09041b75844c None admin] REQUEST_METHOD: `PUT` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:41:21.153257 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47a79601-026e-426e-bf46-09041b75844c None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:41:21.153529 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47a79601-026e-426e-bf46-09041b75844c None admin] PATH_INFO: `/v3/projects/d88586f68b734382abb9836b290aeb54/users/fccf85a112eb4361a125a2f618f5c929/roles/1512d531ebe248ea8e2d7d56d754b230` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:41:21.163714 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47a79601-026e-426e-bf46-09041b75844c None admin] RBAC: Authorizing `identity:create_grant(project_id=d88586f68b734382abb9836b290aeb54, user_id=fccf85a112eb4361a125a2f618f5c929, role_id=1512d531ebe248ea8e2d7d56d754b230)` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 10:41:21.166944 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47a79601-026e-426e-bf46-09041b75844c None admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 10:41:21.190775 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88ea661e-ca1c-44ea-b020-bd8ed85f4cb6 None admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 10:41:21.197358 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 110/233] 10.222.0.25 () {58 vars in 1439 bytes} [Sun Aug 7 10:41:20 2022] PUT /identity/v3/projects/d88586f68b734382abb9836b290aeb54/users/fccf85a112eb4361a125a2f618f5c929/roles/1512d531ebe248ea8e2d7d56d754b230 => generated 0 bytes in 374 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 07 10:41:21.228222 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-977e9413-8458-42cb-8264-b35a26ec750c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 10:41:21.228896 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-977e9413-8458-42cb-8264-b35a26ec750c None admin] REQUEST_METHOD: `PUT` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:41:21.229186 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-977e9413-8458-42cb-8264-b35a26ec750c None admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:41:21.229488 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-977e9413-8458-42cb-8264-b35a26ec750c None admin] PATH_INFO: `/v3/projects/d88586f68b734382abb9836b290aeb54/users/fccf85a112eb4361a125a2f618f5c929/roles/81b1f1bcee2a473088fbbd327c1c5b23` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:41:21.232442 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-977e9413-8458-42cb-8264-b35a26ec750c None admin] RBAC: Authorizing `identity:create_grant(project_id=d88586f68b734382abb9836b290aeb54, user_id=fccf85a112eb4361a125a2f618f5c929, role_id=81b1f1bcee2a473088fbbd327c1c5b23)` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 10:41:21.233933 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-977e9413-8458-42cb-8264-b35a26ec750c None admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 10:41:21.253011 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 124/234] 10.222.0.25 () {58 vars in 1439 bytes} [Sun Aug 7 10:41:21 2022] PUT /identity/v3/projects/d88586f68b734382abb9836b290aeb54/users/fccf85a112eb4361a125a2f618f5c929/roles/81b1f1bcee2a473088fbbd327c1c5b23 => generated 0 bytes in 168 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 07 10:41:21.757084 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbfc8f38-f663-4325-973d-59ac3aa3f208 None None] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:41:21.757439 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbfc8f38-f663-4325-973d-59ac3aa3f208 None None] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:41:21.757646 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbfc8f38-f663-4325-973d-59ac3aa3f208 None None] PATH_INFO: `/` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:41:21.759026 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 111/235] 10.222.0.25 () {54 vars in 927 bytes} [Sun Aug 7 10:41:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 07 10:41:21.764505 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa106b74-6dfb-4124-8a93-01c3da7d3dc6 None None] REQUEST_METHOD: `POST` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:41:21.764943 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa106b74-6dfb-4124-8a93-01c3da7d3dc6 None None] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:41:21.765129 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa106b74-6dfb-4124-8a93-01c3da7d3dc6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:41:21.783852 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.auth.core [None req-aa106b74-6dfb-4124-8a93-01c3da7d3dc6 None None] MFA Rules not processed for user `bac4e6b76fe64217ac8fa44e71cffe6a`. Rule list: `[]` (Enabled: `True`). {{(pid=80144) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 10:41:21.785098 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.fernet_utils [None req-aa106b74-6dfb-4124-8a93-01c3da7d3dc6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80144) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 10:41:21.834307 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 10:41:21.861489 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 125/236] 10.222.0.25 () {58 vars in 1024 bytes} [Sun Aug 7 10:41:21 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 07 10:41:21.906164 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbfc8f38-f663-4325-973d-59ac3aa3f208 None None] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 10:41:21.998536 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 10:41:22.051204 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbfc8f38-f663-4325-973d-59ac3aa3f208 None None] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 10:41:22.097864 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-002d6236-e556-4cb4-af1f-c4dbbb67e58b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 10:41:22.098512 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-002d6236-e556-4cb4-af1f-c4dbbb67e58b None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:41:22.098797 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-002d6236-e556-4cb4-af1f-c4dbbb67e58b None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:41:22.099100 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-002d6236-e556-4cb4-af1f-c4dbbb67e58b None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:41:22.119804 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-002d6236-e556-4cb4-af1f-c4dbbb67e58b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 07 10:41:22.125787 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application [None req-002d6236-e556-4cb4-af1f-c4dbbb67e58b None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Aug 07 10:41:22.125787 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 07 10:41:22.125787 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 07 10:41:22.125787 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 07 10:41:22.125787 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 07 10:41:22.125787 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 07 10:41:22.125787 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 07 10:41:22.125787 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 07 10:41:22.125787 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 07 10:41:22.125787 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 07 10:41:22.125787 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 07 10:41:22.125787 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 07 10:41:22.125787 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 07 10:41:22.125787 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 07 10:41:22.125787 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 07 10:41:22.125787 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 07 10:41:22.126638 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 07 10:41:22.126638 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 07 10:41:22.126638 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 07 10:41:22.126638 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 07 10:41:22.126638 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 07 10:41:22.126638 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 07 10:41:22.126638 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 07 10:41:22.126638 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 07 10:41:22.126638 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 07 10:41:22.126638 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.get_or_create( Aug 07 10:41:22.126638 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 07 10:41:22.126638 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application with Lock( Aug 07 10:41:22.126638 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 07 10:41:22.126638 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self._enter() Aug 07 10:41:22.126638 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 07 10:41:22.126638 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 07 10:41:22.127469 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 07 10:41:22.127469 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.creator() Aug 07 10:41:22.127469 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 07 10:41:22.127469 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application created_value = creator( Aug 07 10:41:22.127469 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 07 10:41:22.127469 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 07 10:41:22.127469 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 07 10:41:22.127469 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 07 10:41:22.127469 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 07 10:41:22.127469 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 07 10:41:22.127469 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Aug 07 10:41:22.127469 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application Aug 07 10:41:22.127469 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 112/237] 10.222.0.25 () {56 vars in 1099 bytes} [Sun Aug 7 10:41:21 2022] GET /identity/v3/roles/reader => generated 84 bytes in 221 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 07 10:41:22.134111 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa106b74-6dfb-4124-8a93-01c3da7d3dc6 None None] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 10:41:22.251676 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa106b74-6dfb-4124-8a93-01c3da7d3dc6 None None] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 10:41:22.297949 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c966b391-0896-4abb-9b7b-d709389fb590 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 10:41:22.298573 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c966b391-0896-4abb-9b7b-d709389fb590 None admin] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:41:22.298855 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c966b391-0896-4abb-9b7b-d709389fb590 None admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:41:22.299159 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c966b391-0896-4abb-9b7b-d709389fb590 None admin] PATH_INFO: `/v3/roles` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:41:22.299834 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c966b391-0896-4abb-9b7b-d709389fb590 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80144) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 10:41:22.300140 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c966b391-0896-4abb-9b7b-d709389fb590 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 10:41:22.301558 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c966b391-0896-4abb-9b7b-d709389fb590 None admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 10:41:22.310673 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 126/238] 10.222.0.25 () {56 vars in 1101 bytes} [Sun Aug 7 10:41:22 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 10:41:22.316529 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-befcce90-401d-4e25-b4dd-c95de9fc6b9f None None] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:41:22.316819 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-befcce90-401d-4e25-b4dd-c95de9fc6b9f None None] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:41:22.317096 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-befcce90-401d-4e25-b4dd-c95de9fc6b9f None None] PATH_INFO: `/` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:41:22.318425 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 113/239] 10.222.0.25 () {54 vars in 927 bytes} [Sun Aug 7 10:41:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 07 10:41:22.319363 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c966b391-0896-4abb-9b7b-d709389fb590 None admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 10:41:22.323642 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e0bfacf-23c8-4382-bd64-a16eb4cb66e8 None None] REQUEST_METHOD: `POST` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:41:22.324128 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e0bfacf-23c8-4382-bd64-a16eb4cb66e8 None None] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:41:22.324528 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e0bfacf-23c8-4382-bd64-a16eb4cb66e8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:41:22.353902 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.auth.core [None req-0e0bfacf-23c8-4382-bd64-a16eb4cb66e8 None None] MFA Rules not processed for user `bac4e6b76fe64217ac8fa44e71cffe6a`. Rule list: `[]` (Enabled: `True`). {{(pid=80143) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 10:41:22.355198 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-0e0bfacf-23c8-4382-bd64-a16eb4cb66e8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 10:41:22.411202 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=100 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 10:41:22.465586 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 114/240] 10.222.0.25 () {58 vars in 1024 bytes} [Sun Aug 7 10:41:22 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 143 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 07 10:41:22.499987 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c966b391-0896-4abb-9b7b-d709389fb590 None admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 10:41:22.504450 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e0bfacf-23c8-4382-bd64-a16eb4cb66e8 None None] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 10:41:22.597910 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f6b56e6-9914-4d76-89ee-92c6a2351efd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 10:41:22.598474 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f6b56e6-9914-4d76-89ee-92c6a2351efd None admin] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:41:22.598756 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f6b56e6-9914-4d76-89ee-92c6a2351efd None admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:41:22.599057 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f6b56e6-9914-4d76-89ee-92c6a2351efd None admin] PATH_INFO: `/v3/users/efa5629181584c3c8f0385d3afb56e64` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:41:22.632282 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=100 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 10:41:22.633740 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f6b56e6-9914-4d76-89ee-92c6a2351efd None admin] RBAC: Authorizing `identity:get_user(user_id=efa5629181584c3c8f0385d3afb56e64)` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 10:41:22.636862 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f6b56e6-9914-4d76-89ee-92c6a2351efd None admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 10:41:22.640588 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 127/241] 10.222.0.25 () {56 vars in 1177 bytes} [Sun Aug 7 10:41:22 2022] GET /identity/v3/users/efa5629181584c3c8f0385d3afb56e64 => generated 297 bytes in 322 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 10:41:22.643890 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07ebcb27-f042-4642-a1e2-861cad4f1156 None None] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:41:22.645287 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07ebcb27-f042-4642-a1e2-861cad4f1156 None None] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:41:22.645808 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07ebcb27-f042-4642-a1e2-861cad4f1156 None None] PATH_INFO: `/` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:41:22.647472 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 128/242] 10.222.0.25 () {54 vars in 927 bytes} [Sun Aug 7 10:41:22 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 07 10:41:22.653322 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07ebcb27-f042-4642-a1e2-861cad4f1156 None None] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 10:41:22.744547 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e0bfacf-23c8-4382-bd64-a16eb4cb66e8 None None] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 10:41:22.810024 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cbe010b-c5ba-47c7-92c2-8ab583643ba0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 10:41:22.811785 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cbe010b-c5ba-47c7-92c2-8ab583643ba0 None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:41:22.812069 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cbe010b-c5ba-47c7-92c2-8ab583643ba0 None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:41:22.812341 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cbe010b-c5ba-47c7-92c2-8ab583643ba0 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:41:22.813453 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07ebcb27-f042-4642-a1e2-861cad4f1156 None None] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 10:41:22.815618 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=100 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 10:41:22.815723 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cbe010b-c5ba-47c7-92c2-8ab583643ba0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 10:41:22.817108 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cbe010b-c5ba-47c7-92c2-8ab583643ba0 None admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 10:41:22.823275 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 115/243] 10.222.0.25 () {56 vars in 1108 bytes} [Sun Aug 7 10:41:22 2022] GET /identity/v3/domains/default => generated 199 bytes in 319 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 10:41:22.826732 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fa58ec5-c02d-4402-87a7-950467a58f4d None None] REQUEST_METHOD: `POST` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:41:22.830958 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fa58ec5-c02d-4402-87a7-950467a58f4d None None] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:41:22.830958 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fa58ec5-c02d-4402-87a7-950467a58f4d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:41:22.860396 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.auth.core [None req-0fa58ec5-c02d-4402-87a7-950467a58f4d None None] MFA Rules not processed for user `bac4e6b76fe64217ac8fa44e71cffe6a`. Rule list: `[]` (Enabled: `True`). {{(pid=80143) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 10:41:22.861582 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-0fa58ec5-c02d-4402-87a7-950467a58f4d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 10:41:22.871205 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-109eb8b2-42b9-4989-8dd0-15340c21824f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 10:41:22.871970 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-109eb8b2-42b9-4989-8dd0-15340c21824f None admin] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:41:22.872249 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-109eb8b2-42b9-4989-8dd0-15340c21824f None admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:41:22.872520 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-109eb8b2-42b9-4989-8dd0-15340c21824f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:41:22.931820 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 116/244] 10.222.0.25 () {58 vars in 1024 bytes} [Sun Aug 7 10:41:22 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 07 10:41:22.939764 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fa58ec5-c02d-4402-87a7-950467a58f4d None None] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 10:41:23.023291 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=100 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 10:41:23.062116 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-109eb8b2-42b9-4989-8dd0-15340c21824f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 10:41:23.063612 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-109eb8b2-42b9-4989-8dd0-15340c21824f None admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 10:41:23.079897 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fa58ec5-c02d-4402-87a7-950467a58f4d None None] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 10:41:23.118890 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01cb9b44-8e08-4c12-9495-0648e63a550f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 10:41:23.119459 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01cb9b44-8e08-4c12-9495-0648e63a550f None admin] REQUEST_METHOD: `POST` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:41:23.119739 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01cb9b44-8e08-4c12-9495-0648e63a550f None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:41:23.120014 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01cb9b44-8e08-4c12-9495-0648e63a550f None admin] PATH_INFO: `/v3/users` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:41:23.121293 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01cb9b44-8e08-4c12-9495-0648e63a550f None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 10:41:23.122862 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01cb9b44-8e08-4c12-9495-0648e63a550f None admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 10:41:23.148130 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 117/245] 10.222.0.25 () {60 vars in 1132 bytes} [Sun Aug 7 10:41:22 2022] POST /identity/v3/users => generated 293 bytes in 216 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 07 10:41:23.152592 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01cb9b44-8e08-4c12-9495-0648e63a550f None admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 10:41:23.225908 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 129/246] 10.222.0.25 () {58 vars in 1282 bytes} [Sun Aug 7 10:41:22 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 573 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 07 10:41:23.231894 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-109eb8b2-42b9-4989-8dd0-15340c21824f None admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 10:41:23.234172 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=100 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 10:41:23.285259 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01cb9b44-8e08-4c12-9495-0648e63a550f None admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 10:41:23.312342 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 10:41:23.372288 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d4c92a3-3199-4bd6-89d9-72baface79e7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 10:41:23.372868 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d4c92a3-3199-4bd6-89d9-72baface79e7 None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:41:23.373147 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d4c92a3-3199-4bd6-89d9-72baface79e7 None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:41:23.373421 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d4c92a3-3199-4bd6-89d9-72baface79e7 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:41:23.386362 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d4c92a3-3199-4bd6-89d9-72baface79e7 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 07 10:41:23.392500 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application [None req-3d4c92a3-3199-4bd6-89d9-72baface79e7 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 07 10:41:23.392500 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 07 10:41:23.392500 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 07 10:41:23.392500 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 07 10:41:23.392500 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 07 10:41:23.392500 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 07 10:41:23.392500 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 07 10:41:23.392500 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 07 10:41:23.392500 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 07 10:41:23.392500 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 07 10:41:23.392500 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 07 10:41:23.392500 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 07 10:41:23.392500 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 07 10:41:23.392500 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 07 10:41:23.392500 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 07 10:41:23.392500 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 07 10:41:23.393368 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 07 10:41:23.393368 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 07 10:41:23.393368 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 07 10:41:23.393368 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 07 10:41:23.393368 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 07 10:41:23.393368 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 07 10:41:23.393368 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 07 10:41:23.393368 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 07 10:41:23.393368 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 07 10:41:23.393368 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.get_or_create( Aug 07 10:41:23.393368 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 07 10:41:23.393368 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application with Lock( Aug 07 10:41:23.393368 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 07 10:41:23.393368 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self._enter() Aug 07 10:41:23.393368 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 07 10:41:23.393368 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 07 10:41:23.394221 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 07 10:41:23.394221 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.creator() Aug 07 10:41:23.394221 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 07 10:41:23.394221 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application created_value = creator( Aug 07 10:41:23.394221 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 07 10:41:23.394221 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 07 10:41:23.394221 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 07 10:41:23.394221 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 07 10:41:23.394221 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 07 10:41:23.394221 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 07 10:41:23.394221 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 07 10:41:23.394221 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application Aug 07 10:41:23.394221 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 118/247] 10.222.0.25 () {56 vars in 1099 bytes} [Sun Aug 7 10:41:23 2022] GET /identity/v3/roles/member => generated 84 bytes in 245 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 07 10:41:23.403612 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d4c92a3-3199-4bd6-89d9-72baface79e7 None admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 10:41:23.414961 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-109eb8b2-42b9-4989-8dd0-15340c21824f None admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 10:41:23.454820 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1b1d3bd-a640-4eb4-af53-dd87ae9edb03 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 10:41:23.455497 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1b1d3bd-a640-4eb4-af53-dd87ae9edb03 None admin] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:41:23.455875 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1b1d3bd-a640-4eb4-af53-dd87ae9edb03 None admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:41:23.456151 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1b1d3bd-a640-4eb4-af53-dd87ae9edb03 None admin] PATH_INFO: `/v3/projects/d88586f68b734382abb9836b290aeb54` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:41:23.457363 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1b1d3bd-a640-4eb4-af53-dd87ae9edb03 None admin] RBAC: Authorizing `identity:get_project(project_id=d88586f68b734382abb9836b290aeb54)` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 10:41:23.458884 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1b1d3bd-a640-4eb4-af53-dd87ae9edb03 None admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 10:41:23.461214 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 130/248] 10.222.0.25 () {56 vars in 1186 bytes} [Sun Aug 7 10:41:23 2022] GET /identity/v3/projects/d88586f68b734382abb9836b290aeb54 => generated 302 bytes in 230 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 07 10:41:23.466866 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1b1d3bd-a640-4eb4-af53-dd87ae9edb03 None admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 10:41:23.531517 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d4c92a3-3199-4bd6-89d9-72baface79e7 None admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 10:41:23.551459 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=100 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 10:41:23.577356 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02472588-a3f3-4d11-ac56-74102bbdc66c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 10:41:23.577977 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02472588-a3f3-4d11-ac56-74102bbdc66c None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:41:23.578306 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02472588-a3f3-4d11-ac56-74102bbdc66c None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:41:23.578583 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02472588-a3f3-4d11-ac56-74102bbdc66c None admin] PATH_INFO: `/v3/roles` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:41:23.579755 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02472588-a3f3-4d11-ac56-74102bbdc66c None admin] RBAC: Adding query filter params (name=member) {{(pid=80143) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 10:41:23.580068 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02472588-a3f3-4d11-ac56-74102bbdc66c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 10:41:23.581983 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02472588-a3f3-4d11-ac56-74102bbdc66c None admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 10:41:23.592214 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1b1d3bd-a640-4eb4-af53-dd87ae9edb03 None admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 10:41:23.593103 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 119/249] 10.222.0.25 () {56 vars in 1101 bytes} [Sun Aug 7 10:41:23 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 10:41:23.601466 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02472588-a3f3-4d11-ac56-74102bbdc66c None admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 10:41:23.639788 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=100 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 10:41:23.640188 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d015d85-81a0-4808-aa19-d858e12799fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 10:41:23.641630 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d015d85-81a0-4808-aa19-d858e12799fd None admin] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:41:23.641914 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d015d85-81a0-4808-aa19-d858e12799fd None admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:41:23.642198 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d015d85-81a0-4808-aa19-d858e12799fd None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:41:23.642956 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d015d85-81a0-4808-aa19-d858e12799fd None admin] RBAC: Adding query filter params (role.id=567b9b8669874c9986d68642dc9ef845, scope.project.id=d88586f68b734382abb9836b290aeb54, user.id=efa5629181584c3c8f0385d3afb56e64) {{(pid=80144) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 10:41:23.643298 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d015d85-81a0-4808-aa19-d858e12799fd None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 10:41:23.644754 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d015d85-81a0-4808-aa19-d858e12799fd None admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 10:41:23.652234 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 131/250] 10.222.0.25 () {56 vars in 1374 bytes} [Sun Aug 7 10:41:23 2022] GET /identity/v3/role_assignments?user.id=efa5629181584c3c8f0385d3afb56e64&scope.project.id=d88586f68b734382abb9836b290aeb54&role.id=567b9b8669874c9986d68642dc9ef845 => generated 259 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 10:41:23.653709 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2fb560b-5b1c-48c7-9651-62ee4b909ef9 None None] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:41:23.654050 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2fb560b-5b1c-48c7-9651-62ee4b909ef9 None None] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:41:23.654328 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2fb560b-5b1c-48c7-9651-62ee4b909ef9 None None] PATH_INFO: `/` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:41:23.655795 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 132/251] 10.222.0.25 () {54 vars in 927 bytes} [Sun Aug 7 10:41:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 07 10:41:23.661838 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2287b8b1-0b9b-4ec4-abca-815f03e33295 None None] REQUEST_METHOD: `POST` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:41:23.662137 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2287b8b1-0b9b-4ec4-abca-815f03e33295 None None] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:41:23.662400 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2287b8b1-0b9b-4ec4-abca-815f03e33295 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:41:23.686369 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.auth.core [None req-2287b8b1-0b9b-4ec4-abca-815f03e33295 None None] MFA Rules not processed for user `bac4e6b76fe64217ac8fa44e71cffe6a`. Rule list: `[]` (Enabled: `True`). {{(pid=80144) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 10:41:23.688054 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.fernet_utils [None req-2287b8b1-0b9b-4ec4-abca-815f03e33295 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80144) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 10:41:23.752615 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02472588-a3f3-4d11-ac56-74102bbdc66c None admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 10:41:23.784979 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 133/252] 10.222.0.25 () {58 vars in 1024 bytes} [Sun Aug 7 10:41:23 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 07 10:41:23.786167 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-766c2a03-ea07-43d1-8194-309906720125 None None] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:41:23.786457 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-766c2a03-ea07-43d1-8194-309906720125 None None] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:41:23.786731 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-766c2a03-ea07-43d1-8194-309906720125 None None] PATH_INFO: `/` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:41:23.788337 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 134/253] 10.222.0.25 () {54 vars in 927 bytes} [Sun Aug 7 10:41:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 07 10:41:23.794551 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-415f3bf6-64ba-4be2-8ac1-85c11a95b069 None None] REQUEST_METHOD: `POST` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:41:23.794876 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-415f3bf6-64ba-4be2-8ac1-85c11a95b069 None None] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:41:23.795132 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-415f3bf6-64ba-4be2-8ac1-85c11a95b069 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:41:23.807667 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d5ef69f-565e-4d1d-8cae-bad10a8808de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 10:41:23.808850 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d5ef69f-565e-4d1d-8cae-bad10a8808de None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:41:23.809132 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d5ef69f-565e-4d1d-8cae-bad10a8808de None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:41:23.809408 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d5ef69f-565e-4d1d-8cae-bad10a8808de None admin] PATH_INFO: `/v3/users/e01f8bc675bc4dd4985cd9f3b1bac86d` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:41:23.811652 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d5ef69f-565e-4d1d-8cae-bad10a8808de None admin] RBAC: Authorizing `identity:get_user(user_id=e01f8bc675bc4dd4985cd9f3b1bac86d)` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 10:41:23.814463 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d5ef69f-565e-4d1d-8cae-bad10a8808de None admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 10:41:23.816361 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 120/254] 10.222.0.25 () {56 vars in 1177 bytes} [Sun Aug 7 10:41:23 2022] GET /identity/v3/users/e01f8bc675bc4dd4985cd9f3b1bac86d => generated 297 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 10:41:23.822118 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d5ef69f-565e-4d1d-8cae-bad10a8808de None admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 10:41:23.822537 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.auth.core [None req-415f3bf6-64ba-4be2-8ac1-85c11a95b069 None None] MFA Rules not processed for user `bac4e6b76fe64217ac8fa44e71cffe6a`. Rule list: `[]` (Enabled: `True`). {{(pid=80144) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 10:41:23.824507 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.fernet_utils [None req-415f3bf6-64ba-4be2-8ac1-85c11a95b069 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80144) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 10:41:23.918943 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=100 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 10:41:23.934318 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 135/255] 10.222.0.25 () {58 vars in 1024 bytes} [Sun Aug 7 10:41:23 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 140 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 07 10:41:23.936733 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-415f3bf6-64ba-4be2-8ac1-85c11a95b069 None None] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 10:41:23.994180 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=100 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 10:41:24.002270 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d5ef69f-565e-4d1d-8cae-bad10a8808de None admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 10:41:24.071747 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a47d1bf-dba5-4c8f-8401-5fefeec04206 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 10:41:24.072408 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a47d1bf-dba5-4c8f-8401-5fefeec04206 None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:41:24.072680 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a47d1bf-dba5-4c8f-8401-5fefeec04206 None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:41:24.072950 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a47d1bf-dba5-4c8f-8401-5fefeec04206 None admin] PATH_INFO: `/v3/projects/d88586f68b734382abb9836b290aeb54` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:41:24.074709 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a47d1bf-dba5-4c8f-8401-5fefeec04206 None admin] RBAC: Authorizing `identity:get_project(project_id=d88586f68b734382abb9836b290aeb54)` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 10:41:24.077392 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a47d1bf-dba5-4c8f-8401-5fefeec04206 None admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 10:41:24.081930 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 121/256] 10.222.0.25 () {56 vars in 1186 bytes} [Sun Aug 7 10:41:23 2022] GET /identity/v3/projects/d88586f68b734382abb9836b290aeb54 => generated 302 bytes in 260 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 07 10:41:24.082763 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a47d1bf-dba5-4c8f-8401-5fefeec04206 None admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 10:41:24.090565 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-415f3bf6-64ba-4be2-8ac1-85c11a95b069 None None] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 10:41:24.162611 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ea229d4-f4ea-4c21-bd27-0549be1ec8c4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 10:41:24.163286 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ea229d4-f4ea-4c21-bd27-0549be1ec8c4 None admin] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:41:24.163565 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ea229d4-f4ea-4c21-bd27-0549be1ec8c4 None admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:41:24.163842 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ea229d4-f4ea-4c21-bd27-0549be1ec8c4 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:41:24.184958 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ea229d4-f4ea-4c21-bd27-0549be1ec8c4 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 07 10:41:24.189724 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application [None req-8ea229d4-f4ea-4c21-bd27-0549be1ec8c4 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 07 10:41:24.189724 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 07 10:41:24.189724 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 07 10:41:24.189724 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 07 10:41:24.189724 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 07 10:41:24.189724 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 07 10:41:24.189724 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 07 10:41:24.189724 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 07 10:41:24.189724 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 07 10:41:24.189724 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 07 10:41:24.189724 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 07 10:41:24.189724 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 07 10:41:24.189724 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 07 10:41:24.189724 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 07 10:41:24.189724 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 07 10:41:24.189724 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 07 10:41:24.190590 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 07 10:41:24.190590 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 07 10:41:24.190590 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 07 10:41:24.190590 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 07 10:41:24.190590 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 07 10:41:24.190590 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 07 10:41:24.190590 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 07 10:41:24.190590 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 07 10:41:24.190590 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 07 10:41:24.190590 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self.get_or_create( Aug 07 10:41:24.190590 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 07 10:41:24.190590 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application with Lock( Aug 07 10:41:24.190590 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 07 10:41:24.190590 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self._enter() Aug 07 10:41:24.190590 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 07 10:41:24.190590 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 07 10:41:24.191730 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 07 10:41:24.191730 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self.creator() Aug 07 10:41:24.191730 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 07 10:41:24.191730 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application created_value = creator( Aug 07 10:41:24.191730 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 07 10:41:24.191730 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 07 10:41:24.191730 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 07 10:41:24.191730 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 07 10:41:24.191730 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 07 10:41:24.191730 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 07 10:41:24.191730 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 07 10:41:24.191730 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application Aug 07 10:41:24.193887 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 136/257] 10.222.0.25 () {56 vars in 1114 bytes} [Sun Aug 7 10:41:23 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 259 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 07 10:41:24.194728 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ea229d4-f4ea-4c21-bd27-0549be1ec8c4 None admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 10:41:24.220346 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a47d1bf-dba5-4c8f-8401-5fefeec04206 None admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 10:41:24.267136 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4996b869-5e91-4bc4-8281-6cf34dde2b6d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 10:41:24.268126 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4996b869-5e91-4bc4-8281-6cf34dde2b6d None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:41:24.268621 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4996b869-5e91-4bc4-8281-6cf34dde2b6d None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:41:24.269083 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4996b869-5e91-4bc4-8281-6cf34dde2b6d None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:41:24.290105 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4996b869-5e91-4bc4-8281-6cf34dde2b6d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 07 10:41:24.297352 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application [None req-4996b869-5e91-4bc4-8281-6cf34dde2b6d None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 07 10:41:24.297352 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 07 10:41:24.297352 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 07 10:41:24.297352 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 07 10:41:24.297352 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 07 10:41:24.297352 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 07 10:41:24.297352 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 07 10:41:24.297352 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 07 10:41:24.297352 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 07 10:41:24.297352 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 07 10:41:24.297352 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 07 10:41:24.297352 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 07 10:41:24.297352 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 07 10:41:24.297352 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 07 10:41:24.297352 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 07 10:41:24.297352 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 07 10:41:24.298209 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 07 10:41:24.298209 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 07 10:41:24.298209 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 07 10:41:24.298209 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 07 10:41:24.298209 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 07 10:41:24.298209 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 07 10:41:24.298209 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 07 10:41:24.298209 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 07 10:41:24.298209 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 07 10:41:24.298209 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.get_or_create( Aug 07 10:41:24.298209 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 07 10:41:24.298209 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application with Lock( Aug 07 10:41:24.298209 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 07 10:41:24.298209 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self._enter() Aug 07 10:41:24.298209 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 07 10:41:24.298209 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 07 10:41:24.299004 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 07 10:41:24.299004 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.creator() Aug 07 10:41:24.299004 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 07 10:41:24.299004 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application created_value = creator( Aug 07 10:41:24.299004 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 07 10:41:24.299004 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 07 10:41:24.299004 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 07 10:41:24.299004 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 07 10:41:24.299004 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 07 10:41:24.299004 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 07 10:41:24.299004 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 07 10:41:24.299004 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application Aug 07 10:41:24.301485 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 122/258] 10.222.0.25 () {56 vars in 1096 bytes} [Sun Aug 7 10:41:24 2022] GET /identity/v3/roles/admin => generated 83 bytes in 219 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 07 10:41:24.302454 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4996b869-5e91-4bc4-8281-6cf34dde2b6d None admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 10:41:24.332324 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=100 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 10:41:24.337025 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=100 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 10:41:24.440339 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ea229d4-f4ea-4c21-bd27-0549be1ec8c4 None admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 10:41:24.455380 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4996b869-5e91-4bc4-8281-6cf34dde2b6d None admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 10:41:24.504694 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fba72c1-a2a8-4155-8dbe-b89eb4b9a39b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 10:41:24.505371 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fba72c1-a2a8-4155-8dbe-b89eb4b9a39b None admin] REQUEST_METHOD: `PUT` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:41:24.505654 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fba72c1-a2a8-4155-8dbe-b89eb4b9a39b None admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:41:24.505932 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fba72c1-a2a8-4155-8dbe-b89eb4b9a39b None admin] PATH_INFO: `/v3/projects/d88586f68b734382abb9836b290aeb54/users/e01f8bc675bc4dd4985cd9f3b1bac86d/roles/b98dfb3b9fee4644af2887a3ac3fd2e6` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:41:24.508582 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fba72c1-a2a8-4155-8dbe-b89eb4b9a39b None admin] RBAC: Authorizing `identity:create_grant(project_id=d88586f68b734382abb9836b290aeb54, user_id=e01f8bc675bc4dd4985cd9f3b1bac86d, role_id=b98dfb3b9fee4644af2887a3ac3fd2e6)` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 10:41:24.510035 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fba72c1-a2a8-4155-8dbe-b89eb4b9a39b None admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 10:41:24.518152 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92f85b94-0f68-4ee5-b058-e8abf9db9ca3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 10:41:24.518816 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92f85b94-0f68-4ee5-b058-e8abf9db9ca3 None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:41:24.519150 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92f85b94-0f68-4ee5-b058-e8abf9db9ca3 None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:41:24.519469 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92f85b94-0f68-4ee5-b058-e8abf9db9ca3 None admin] PATH_INFO: `/v3/roles` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:41:24.520152 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92f85b94-0f68-4ee5-b058-e8abf9db9ca3 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80143) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 10:41:24.520513 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92f85b94-0f68-4ee5-b058-e8abf9db9ca3 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 10:41:24.521882 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92f85b94-0f68-4ee5-b058-e8abf9db9ca3 None admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 10:41:24.532502 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 137/259] 10.222.0.25 () {58 vars in 1439 bytes} [Sun Aug 7 10:41:24 2022] PUT /identity/v3/projects/d88586f68b734382abb9836b290aeb54/users/e01f8bc675bc4dd4985cd9f3b1bac86d/roles/b98dfb3b9fee4644af2887a3ac3fd2e6 => generated 0 bytes in 338 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 07 10:41:24.533155 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fba72c1-a2a8-4155-8dbe-b89eb4b9a39b None admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 10:41:24.539894 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 123/260] 10.222.0.25 () {56 vars in 1111 bytes} [Sun Aug 7 10:41:24 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 238 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 10:41:24.546743 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92f85b94-0f68-4ee5-b058-e8abf9db9ca3 None admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 10:41:24.703428 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=100 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 10:41:24.723481 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92f85b94-0f68-4ee5-b058-e8abf9db9ca3 None admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 10:41:24.741169 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fba72c1-a2a8-4155-8dbe-b89eb4b9a39b None admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 10:41:24.768095 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 10:41:24.792186 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75772f45-c094-44ad-ad0e-5c7911bf5639 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 10:41:24.792857 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75772f45-c094-44ad-ad0e-5c7911bf5639 None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:41:24.793139 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75772f45-c094-44ad-ad0e-5c7911bf5639 None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:41:24.793408 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75772f45-c094-44ad-ad0e-5c7911bf5639 None admin] PATH_INFO: `/v3/users/fccf85a112eb4361a125a2f618f5c929` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:41:24.795545 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75772f45-c094-44ad-ad0e-5c7911bf5639 None admin] RBAC: Authorizing `identity:get_user(user_id=fccf85a112eb4361a125a2f618f5c929)` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 10:41:24.797009 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75772f45-c094-44ad-ad0e-5c7911bf5639 None admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 10:41:24.798764 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 124/261] 10.222.0.25 () {56 vars in 1177 bytes} [Sun Aug 7 10:41:24 2022] GET /identity/v3/users/fccf85a112eb4361a125a2f618f5c929 => generated 283 bytes in 252 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 10:41:24.804052 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75772f45-c094-44ad-ad0e-5c7911bf5639 None admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 10:41:24.809940 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-446da78f-91de-4693-853c-91d21a63b213 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 10:41:24.810576 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-446da78f-91de-4693-853c-91d21a63b213 None admin] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:41:24.810858 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-446da78f-91de-4693-853c-91d21a63b213 None admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:41:24.811162 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-446da78f-91de-4693-853c-91d21a63b213 None admin] PATH_INFO: `/v3/roles` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:41:24.811853 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-446da78f-91de-4693-853c-91d21a63b213 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80144) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 10:41:24.812161 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-446da78f-91de-4693-853c-91d21a63b213 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 10:41:24.813550 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-446da78f-91de-4693-853c-91d21a63b213 None admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 10:41:24.833761 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 138/262] 10.222.0.25 () {56 vars in 1099 bytes} [Sun Aug 7 10:41:24 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 301 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 10:41:24.840081 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-446da78f-91de-4693-853c-91d21a63b213 None admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 10:41:24.966819 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-446da78f-91de-4693-853c-91d21a63b213 None admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 10:41:24.971975 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75772f45-c094-44ad-ad0e-5c7911bf5639 None admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 10:41:25.003157 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f2cba47-166d-43a0-8fcf-b59072150b5f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 10:41:25.004155 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f2cba47-166d-43a0-8fcf-b59072150b5f None admin] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:41:25.004701 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f2cba47-166d-43a0-8fcf-b59072150b5f None admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:41:25.005176 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f2cba47-166d-43a0-8fcf-b59072150b5f None admin] PATH_INFO: `/v3/users/fccf85a112eb4361a125a2f618f5c929` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:41:25.006513 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f2cba47-166d-43a0-8fcf-b59072150b5f None admin] RBAC: Authorizing `identity:get_user(user_id=fccf85a112eb4361a125a2f618f5c929)` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 10:41:25.008492 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f2cba47-166d-43a0-8fcf-b59072150b5f None admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 10:41:25.011652 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 139/263] 10.222.0.25 () {56 vars in 1177 bytes} [Sun Aug 7 10:41:24 2022] GET /identity/v3/users/fccf85a112eb4361a125a2f618f5c929 => generated 283 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 07 10:41:25.017476 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f2cba47-166d-43a0-8fcf-b59072150b5f None admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 10:41:25.071891 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39cccdd3-8d8f-4b90-b8ad-f60b083d0a16 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 10:41:25.072426 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39cccdd3-8d8f-4b90-b8ad-f60b083d0a16 None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:41:25.072672 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39cccdd3-8d8f-4b90-b8ad-f60b083d0a16 None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:41:25.072950 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39cccdd3-8d8f-4b90-b8ad-f60b083d0a16 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:41:25.076979 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=100 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 10:41:25.135292 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=100 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 10:41:25.141844 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f2cba47-166d-43a0-8fcf-b59072150b5f None admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 10:41:25.179888 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5752f45-2a9b-4ad0-bc65-38b02f6d7ab1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 10:41:25.180647 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5752f45-2a9b-4ad0-bc65-38b02f6d7ab1 None admin] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:41:25.181029 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5752f45-2a9b-4ad0-bc65-38b02f6d7ab1 None admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:41:25.181400 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5752f45-2a9b-4ad0-bc65-38b02f6d7ab1 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:41:25.240542 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39cccdd3-8d8f-4b90-b8ad-f60b083d0a16 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 10:41:25.241934 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39cccdd3-8d8f-4b90-b8ad-f60b083d0a16 None admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 10:41:25.252768 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5752f45-2a9b-4ad0-bc65-38b02f6d7ab1 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 10:41:25.254334 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5752f45-2a9b-4ad0-bc65-38b02f6d7ab1 None admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 10:41:25.330191 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=100 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 10:41:25.357759 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 125/264] 10.222.0.25 () {58 vars in 1282 bytes} [Sun Aug 7 10:41:24 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 554 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 07 10:41:25.363847 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39cccdd3-8d8f-4b90-b8ad-f60b083d0a16 None admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 10:41:25.429147 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=100 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 10:41:25.480149 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39cccdd3-8d8f-4b90-b8ad-f60b083d0a16 None admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 10:41:25.498294 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 140/265] 10.222.0.25 () {58 vars in 1282 bytes} [Sun Aug 7 10:41:25 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 481 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Aug 07 10:41:25.503957 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5752f45-2a9b-4ad0-bc65-38b02f6d7ab1 None admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 10:41:25.524037 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa5e2b13-c57f-44db-a3ab-67f135a77cec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 10:41:25.525947 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa5e2b13-c57f-44db-a3ab-67f135a77cec None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:41:25.527183 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa5e2b13-c57f-44db-a3ab-67f135a77cec None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:41:25.528407 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa5e2b13-c57f-44db-a3ab-67f135a77cec None admin] PATH_INFO: `/v3/projects/d88586f68b734382abb9836b290aeb54` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:41:25.531172 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa5e2b13-c57f-44db-a3ab-67f135a77cec None admin] RBAC: Authorizing `identity:get_project(project_id=d88586f68b734382abb9836b290aeb54)` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 10:41:25.533044 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa5e2b13-c57f-44db-a3ab-67f135a77cec None admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 10:41:25.540544 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 126/266] 10.222.0.25 () {56 vars in 1186 bytes} [Sun Aug 7 10:41:25 2022] GET /identity/v3/projects/d88586f68b734382abb9836b290aeb54 => generated 302 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 07 10:41:25.547056 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa5e2b13-c57f-44db-a3ab-67f135a77cec None admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 10:41:25.720288 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5752f45-2a9b-4ad0-bc65-38b02f6d7ab1 None admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 10:41:25.757528 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa5e2b13-c57f-44db-a3ab-67f135a77cec None admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 10:41:25.776772 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f7df1c6-4e82-430b-9d34-5e68fb4ec306 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 10:41:25.777711 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f7df1c6-4e82-430b-9d34-5e68fb4ec306 None admin] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:41:25.778199 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f7df1c6-4e82-430b-9d34-5e68fb4ec306 None admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:41:25.778650 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f7df1c6-4e82-430b-9d34-5e68fb4ec306 None admin] PATH_INFO: `/v3/projects/d88586f68b734382abb9836b290aeb54` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:41:25.780000 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f7df1c6-4e82-430b-9d34-5e68fb4ec306 None admin] RBAC: Authorizing `identity:get_project(project_id=d88586f68b734382abb9836b290aeb54)` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 10:41:25.782623 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f7df1c6-4e82-430b-9d34-5e68fb4ec306 None admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 10:41:25.784546 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 141/267] 10.222.0.25 () {56 vars in 1186 bytes} [Sun Aug 7 10:41:25 2022] GET /identity/v3/projects/d88586f68b734382abb9836b290aeb54 => generated 302 bytes in 281 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 07 10:41:25.791930 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f7df1c6-4e82-430b-9d34-5e68fb4ec306 None admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 10:41:25.810991 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=100 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 10:41:25.825253 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=100 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 10:41:25.835614 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31014137-a83e-43f0-a576-603c808ff002 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 10:41:25.836239 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31014137-a83e-43f0-a576-603c808ff002 None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:41:25.836518 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31014137-a83e-43f0-a576-603c808ff002 None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:41:25.836792 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31014137-a83e-43f0-a576-603c808ff002 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:41:25.837526 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31014137-a83e-43f0-a576-603c808ff002 None admin] RBAC: Adding query filter params (role.id=1512d531ebe248ea8e2d7d56d754b230, scope.project.id=d88586f68b734382abb9836b290aeb54, user.id=fccf85a112eb4361a125a2f618f5c929) {{(pid=80143) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 10:41:25.837834 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31014137-a83e-43f0-a576-603c808ff002 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 10:41:25.839623 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31014137-a83e-43f0-a576-603c808ff002 None admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 10:41:25.851492 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 127/268] 10.222.0.25 () {56 vars in 1374 bytes} [Sun Aug 7 10:41:25 2022] GET /identity/v3/role_assignments?user.id=fccf85a112eb4361a125a2f618f5c929&scope.project.id=d88586f68b734382abb9836b290aeb54&role.id=1512d531ebe248ea8e2d7d56d754b230 => generated 608 bytes in 305 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 10:41:25.954578 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f7df1c6-4e82-430b-9d34-5e68fb4ec306 None admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 10:41:25.992166 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b9d986d-5e45-43af-b3c7-604dba80aa1f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 10:41:25.992775 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b9d986d-5e45-43af-b3c7-604dba80aa1f None admin] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:41:25.993058 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b9d986d-5e45-43af-b3c7-604dba80aa1f None admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:41:25.993323 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b9d986d-5e45-43af-b3c7-604dba80aa1f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:41:25.994043 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b9d986d-5e45-43af-b3c7-604dba80aa1f None admin] RBAC: Adding query filter params (role.id=81b1f1bcee2a473088fbbd327c1c5b23, scope.project.id=d88586f68b734382abb9836b290aeb54, user.id=fccf85a112eb4361a125a2f618f5c929) {{(pid=80144) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 10:41:25.994349 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b9d986d-5e45-43af-b3c7-604dba80aa1f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 10:41:25.995886 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b9d986d-5e45-43af-b3c7-604dba80aa1f None admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 10:41:26.006371 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 142/269] 10.222.0.25 () {56 vars in 1374 bytes} [Sun Aug 7 10:41:25 2022] GET /identity/v3/role_assignments?user.id=fccf85a112eb4361a125a2f618f5c929&scope.project.id=d88586f68b734382abb9836b290aeb54&role.id=81b1f1bcee2a473088fbbd327c1c5b23 => generated 608 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 10:41:26.433663 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1849e83-9f42-47ff-a4e8-8a66055ce0c8 None None] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:41:26.434149 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1849e83-9f42-47ff-a4e8-8a66055ce0c8 None None] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:41:26.434442 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1849e83-9f42-47ff-a4e8-8a66055ce0c8 None None] PATH_INFO: `/` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:41:26.436097 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 128/270] 10.222.0.25 () {54 vars in 927 bytes} [Sun Aug 7 10:41:26 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 07 10:41:26.441661 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90aa96e8-b67b-4811-a65e-a32e5d16b708 None None] REQUEST_METHOD: `POST` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:41:26.441953 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90aa96e8-b67b-4811-a65e-a32e5d16b708 None None] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:41:26.442232 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90aa96e8-b67b-4811-a65e-a32e5d16b708 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:41:26.463784 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.auth.core [None req-90aa96e8-b67b-4811-a65e-a32e5d16b708 None None] MFA Rules not processed for user `bac4e6b76fe64217ac8fa44e71cffe6a`. Rule list: `[]` (Enabled: `True`). {{(pid=80144) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 10:41:26.465353 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.fernet_utils [None req-90aa96e8-b67b-4811-a65e-a32e5d16b708 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80144) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 10:41:26.482064 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8f4dc82-c3c2-46a0-a886-23786519c171 None None] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:41:26.482395 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8f4dc82-c3c2-46a0-a886-23786519c171 None None] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:41:26.482619 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8f4dc82-c3c2-46a0-a886-23786519c171 None None] PATH_INFO: `/` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:41:26.484579 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 129/271] 10.222.0.25 () {54 vars in 927 bytes} [Sun Aug 7 10:41:26 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 07 10:41:26.492302 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0352cc58-0ad9-4625-9f3d-2d83a11d22b5 None None] REQUEST_METHOD: `POST` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:41:26.492586 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0352cc58-0ad9-4625-9f3d-2d83a11d22b5 None None] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:41:26.492866 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0352cc58-0ad9-4625-9f3d-2d83a11d22b5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:41:26.517422 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.auth.core [None req-0352cc58-0ad9-4625-9f3d-2d83a11d22b5 None None] MFA Rules not processed for user `bac4e6b76fe64217ac8fa44e71cffe6a`. Rule list: `[]` (Enabled: `True`). {{(pid=80143) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 10:41:26.518633 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-0352cc58-0ad9-4625-9f3d-2d83a11d22b5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 10:41:26.552192 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 143/272] 10.222.0.25 () {58 vars in 1024 bytes} [Sun Aug 7 10:41:26 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 07 10:41:26.553462 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a3fd660-21e3-4abc-b709-422c39ce56d8 None None] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:41:26.553749 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a3fd660-21e3-4abc-b709-422c39ce56d8 None None] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:41:26.554022 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a3fd660-21e3-4abc-b709-422c39ce56d8 None None] PATH_INFO: `/` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:41:26.555350 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 144/273] 10.222.0.25 () {54 vars in 927 bytes} [Sun Aug 7 10:41:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 07 10:41:26.560887 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad37d39d-e859-4fd4-aa2d-d34a95dfb8d3 None None] REQUEST_METHOD: `POST` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:41:26.561266 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad37d39d-e859-4fd4-aa2d-d34a95dfb8d3 None None] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:41:26.561551 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad37d39d-e859-4fd4-aa2d-d34a95dfb8d3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:41:26.575249 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=100 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 10:41:26.583418 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.auth.core [None req-ad37d39d-e859-4fd4-aa2d-d34a95dfb8d3 None None] MFA Rules not processed for user `bac4e6b76fe64217ac8fa44e71cffe6a`. Rule list: `[]` (Enabled: `True`). {{(pid=80144) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 10:41:26.584819 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.fernet_utils [None req-ad37d39d-e859-4fd4-aa2d-d34a95dfb8d3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80144) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 10:41:26.608750 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 130/274] 10.222.0.25 () {58 vars in 1024 bytes} [Sun Aug 7 10:41:26 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 07 10:41:26.610737 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0352cc58-0ad9-4625-9f3d-2d83a11d22b5 None None] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 10:41:26.658168 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 145/275] 10.222.0.25 () {58 vars in 1024 bytes} [Sun Aug 7 10:41:26 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 07 10:41:26.663821 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad37d39d-e859-4fd4-aa2d-d34a95dfb8d3 None None] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 10:41:26.726154 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0352cc58-0ad9-4625-9f3d-2d83a11d22b5 None None] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 10:41:26.744906 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=100 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 10:41:26.764198 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28e01a64-3deb-4661-b242-592749d109ca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 10:41:26.764871 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28e01a64-3deb-4661-b242-592749d109ca None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:41:26.765225 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28e01a64-3deb-4661-b242-592749d109ca None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:41:26.765499 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28e01a64-3deb-4661-b242-592749d109ca None admin] PATH_INFO: `/v3/roles/member` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:41:26.771294 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad37d39d-e859-4fd4-aa2d-d34a95dfb8d3 None None] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 10:41:26.777100 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28e01a64-3deb-4661-b242-592749d109ca None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 07 10:41:26.782770 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application [None req-28e01a64-3deb-4661-b242-592749d109ca None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 07 10:41:26.782770 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 07 10:41:26.782770 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 07 10:41:26.782770 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 07 10:41:26.782770 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 07 10:41:26.782770 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 07 10:41:26.782770 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 07 10:41:26.782770 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 07 10:41:26.782770 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 07 10:41:26.782770 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 07 10:41:26.782770 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 07 10:41:26.782770 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 07 10:41:26.782770 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 07 10:41:26.782770 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 07 10:41:26.782770 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 07 10:41:26.782770 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 07 10:41:26.783646 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 07 10:41:26.783646 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 07 10:41:26.783646 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 07 10:41:26.783646 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 07 10:41:26.783646 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 07 10:41:26.783646 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 07 10:41:26.783646 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 07 10:41:26.783646 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 07 10:41:26.783646 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 07 10:41:26.783646 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.get_or_create( Aug 07 10:41:26.783646 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 07 10:41:26.783646 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application with Lock( Aug 07 10:41:26.783646 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 07 10:41:26.783646 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self._enter() Aug 07 10:41:26.783646 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 07 10:41:26.783646 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 07 10:41:26.784440 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 07 10:41:26.784440 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.creator() Aug 07 10:41:26.784440 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 07 10:41:26.784440 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application created_value = creator( Aug 07 10:41:26.784440 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 07 10:41:26.784440 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 07 10:41:26.784440 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 07 10:41:26.784440 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 07 10:41:26.784440 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 07 10:41:26.784440 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 07 10:41:26.784440 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 07 10:41:26.784440 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application Aug 07 10:41:26.788167 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 131/276] 10.222.0.25 () {56 vars in 1099 bytes} [Sun Aug 7 10:41:26 2022] GET /identity/v3/roles/member => generated 84 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 07 10:41:26.788167 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28e01a64-3deb-4661-b242-592749d109ca None admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 10:41:26.814180 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54723ca8-fbf0-4c32-b9de-dc8e6d3a3fbd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 10:41:26.814821 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54723ca8-fbf0-4c32-b9de-dc8e6d3a3fbd None admin] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:41:26.815147 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54723ca8-fbf0-4c32-b9de-dc8e6d3a3fbd None admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:41:26.815426 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54723ca8-fbf0-4c32-b9de-dc8e6d3a3fbd None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:41:26.818525 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=100 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 10:41:26.828654 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54723ca8-fbf0-4c32-b9de-dc8e6d3a3fbd None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 07 10:41:26.833350 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application [None req-54723ca8-fbf0-4c32-b9de-dc8e6d3a3fbd None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Aug 07 10:41:26.833350 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 07 10:41:26.833350 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 07 10:41:26.833350 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 07 10:41:26.833350 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 07 10:41:26.833350 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 07 10:41:26.833350 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 07 10:41:26.833350 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 07 10:41:26.833350 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 07 10:41:26.833350 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 07 10:41:26.833350 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 07 10:41:26.833350 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 07 10:41:26.833350 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 07 10:41:26.833350 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 07 10:41:26.833350 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 07 10:41:26.833350 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 07 10:41:26.834598 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 07 10:41:26.834598 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 07 10:41:26.834598 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 07 10:41:26.834598 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 07 10:41:26.834598 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 07 10:41:26.834598 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 07 10:41:26.834598 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 07 10:41:26.834598 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 07 10:41:26.834598 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 07 10:41:26.834598 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self.get_or_create( Aug 07 10:41:26.834598 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 07 10:41:26.834598 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application with Lock( Aug 07 10:41:26.834598 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 07 10:41:26.834598 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self._enter() Aug 07 10:41:26.834598 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 07 10:41:26.834598 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 07 10:41:26.836180 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 07 10:41:26.836180 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self.creator() Aug 07 10:41:26.836180 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 07 10:41:26.836180 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application created_value = creator( Aug 07 10:41:26.836180 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 07 10:41:26.836180 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 07 10:41:26.836180 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 07 10:41:26.836180 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 07 10:41:26.836180 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 07 10:41:26.836180 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 07 10:41:26.836180 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Aug 07 10:41:26.836180 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application Aug 07 10:41:26.836180 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 146/277] 10.222.0.25 () {56 vars in 1099 bytes} [Sun Aug 7 10:41:26 2022] GET /identity/v3/roles/reader => generated 84 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 07 10:41:26.836180 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54723ca8-fbf0-4c32-b9de-dc8e6d3a3fbd None admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 10:41:26.914551 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28e01a64-3deb-4661-b242-592749d109ca None admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 10:41:26.945325 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54723ca8-fbf0-4c32-b9de-dc8e6d3a3fbd None admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 10:41:26.954622 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d94048a-4d29-4d14-8ad3-3ccb32d909cd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 10:41:26.955641 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d94048a-4d29-4d14-8ad3-3ccb32d909cd None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:41:26.955946 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d94048a-4d29-4d14-8ad3-3ccb32d909cd None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:41:26.956224 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d94048a-4d29-4d14-8ad3-3ccb32d909cd None admin] PATH_INFO: `/v3/domains/default` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:41:26.957585 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d94048a-4d29-4d14-8ad3-3ccb32d909cd None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 10:41:26.959069 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d94048a-4d29-4d14-8ad3-3ccb32d909cd None admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 10:41:26.960630 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 132/278] 10.222.0.25 () {56 vars in 1108 bytes} [Sun Aug 7 10:41:26 2022] GET /identity/v3/domains/default => generated 199 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 10:41:26.964460 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d94048a-4d29-4d14-8ad3-3ccb32d909cd None admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 10:41:26.982085 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7760f825-13e7-4065-88d9-81998615cedb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 10:41:26.982627 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7760f825-13e7-4065-88d9-81998615cedb None admin] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:41:26.982914 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7760f825-13e7-4065-88d9-81998615cedb None admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:41:26.983232 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7760f825-13e7-4065-88d9-81998615cedb None admin] PATH_INFO: `/v3/roles` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:41:26.983895 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7760f825-13e7-4065-88d9-81998615cedb None admin] RBAC: Adding query filter params (name=member) {{(pid=80144) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 10:41:26.984325 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7760f825-13e7-4065-88d9-81998615cedb None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 10:41:26.985804 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7760f825-13e7-4065-88d9-81998615cedb None admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 10:41:27.000125 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 147/279] 10.222.0.25 () {56 vars in 1101 bytes} [Sun Aug 7 10:41:26 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 10:41:27.000891 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7760f825-13e7-4065-88d9-81998615cedb None admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 10:41:27.013512 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=100 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 10:41:27.064569 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=100 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 10:41:27.072256 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d94048a-4d29-4d14-8ad3-3ccb32d909cd None admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 10:41:27.110202 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7760f825-13e7-4065-88d9-81998615cedb None admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 10:41:27.112766 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b22b0758-4464-4149-9ea3-af2e6c59b133 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 10:41:27.113376 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b22b0758-4464-4149-9ea3-af2e6c59b133 None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:41:27.113663 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b22b0758-4464-4149-9ea3-af2e6c59b133 None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:41:27.113941 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b22b0758-4464-4149-9ea3-af2e6c59b133 None admin] PATH_INFO: `/v3/roles` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:41:27.114615 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b22b0758-4464-4149-9ea3-af2e6c59b133 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80143) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 10:41:27.114935 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b22b0758-4464-4149-9ea3-af2e6c59b133 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 10:41:27.116355 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b22b0758-4464-4149-9ea3-af2e6c59b133 None admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 10:41:27.130027 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 133/280] 10.222.0.25 () {56 vars in 1101 bytes} [Sun Aug 7 10:41:26 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 10:41:27.130027 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b22b0758-4464-4149-9ea3-af2e6c59b133 None admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 10:41:27.152201 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10ac30fb-9732-42fe-ba26-29823f075046 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 10:41:27.153070 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10ac30fb-9732-42fe-ba26-29823f075046 None admin] REQUEST_METHOD: `POST` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:41:27.153351 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10ac30fb-9732-42fe-ba26-29823f075046 None admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:41:27.153620 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10ac30fb-9732-42fe-ba26-29823f075046 None admin] PATH_INFO: `/v3/users` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:41:27.154262 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10ac30fb-9732-42fe-ba26-29823f075046 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 10:41:27.155732 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10ac30fb-9732-42fe-ba26-29823f075046 None admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 10:41:27.185047 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 148/281] 10.222.0.25 () {60 vars in 1132 bytes} [Sun Aug 7 10:41:27 2022] POST /identity/v3/users => generated 293 bytes in 184 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 07 10:41:27.192009 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10ac30fb-9732-42fe-ba26-29823f075046 None admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 10:41:27.238569 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b22b0758-4464-4149-9ea3-af2e6c59b133 None admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 10:41:27.249477 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=100 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 10:41:27.278009 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7eb0bb1-7c76-470e-aa0d-b397c28a8072 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 10:41:27.278680 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7eb0bb1-7c76-470e-aa0d-b397c28a8072 None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:41:27.278976 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7eb0bb1-7c76-470e-aa0d-b397c28a8072 None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:41:27.279271 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7eb0bb1-7c76-470e-aa0d-b397c28a8072 None admin] PATH_INFO: `/v3/users/5a4d08b8bd68447fa7d2f23a95eee88b` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:41:27.295704 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7eb0bb1-7c76-470e-aa0d-b397c28a8072 None admin] RBAC: Authorizing `identity:get_user(user_id=5a4d08b8bd68447fa7d2f23a95eee88b)` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 10:41:27.297194 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7eb0bb1-7c76-470e-aa0d-b397c28a8072 None admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 10:41:27.299137 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 134/282] 10.222.0.25 () {56 vars in 1177 bytes} [Sun Aug 7 10:41:27 2022] GET /identity/v3/users/5a4d08b8bd68447fa7d2f23a95eee88b => generated 293 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 10:41:27.300276 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97f4b966-a0bd-4fe2-94f6-cb045edcef20 None None] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:41:27.300567 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97f4b966-a0bd-4fe2-94f6-cb045edcef20 None None] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:41:27.300844 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97f4b966-a0bd-4fe2-94f6-cb045edcef20 None None] PATH_INFO: `/` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:41:27.302245 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 135/283] 10.222.0.25 () {54 vars in 927 bytes} [Sun Aug 7 10:41:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 07 10:41:27.306244 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97f4b966-a0bd-4fe2-94f6-cb045edcef20 None None] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 10:41:27.311726 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10ac30fb-9732-42fe-ba26-29823f075046 None admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 10:41:27.328691 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 10:41:27.356038 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36e93907-e99c-4342-933c-5c6d7c37a081 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 10:41:27.357112 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36e93907-e99c-4342-933c-5c6d7c37a081 None admin] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:41:27.357112 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36e93907-e99c-4342-933c-5c6d7c37a081 None admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:41:27.357302 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36e93907-e99c-4342-933c-5c6d7c37a081 None admin] PATH_INFO: `/v3/users/efa5629181584c3c8f0385d3afb56e64` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:41:27.359051 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36e93907-e99c-4342-933c-5c6d7c37a081 None admin] RBAC: Authorizing `identity:get_user(user_id=efa5629181584c3c8f0385d3afb56e64)` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 10:41:27.359870 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36e93907-e99c-4342-933c-5c6d7c37a081 None admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 10:41:27.363125 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 149/284] 10.222.0.25 () {56 vars in 1177 bytes} [Sun Aug 7 10:41:27 2022] GET /identity/v3/users/efa5629181584c3c8f0385d3afb56e64 => generated 297 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 10:41:27.367578 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40f6e0c2-b9e5-4a62-84b4-af5eb962d097 None None] REQUEST_METHOD: `POST` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:41:27.367869 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40f6e0c2-b9e5-4a62-84b4-af5eb962d097 None None] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:41:27.368148 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40f6e0c2-b9e5-4a62-84b4-af5eb962d097 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:41:27.387410 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.auth.core [None req-40f6e0c2-b9e5-4a62-84b4-af5eb962d097 None None] MFA Rules not processed for user `bac4e6b76fe64217ac8fa44e71cffe6a`. Rule list: `[]` (Enabled: `True`). {{(pid=80144) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 10:41:27.389113 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.fernet_utils [None req-40f6e0c2-b9e5-4a62-84b4-af5eb962d097 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80144) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 10:41:27.431161 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97f4b966-a0bd-4fe2-94f6-cb045edcef20 None None] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 10:41:27.473426 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 150/285] 10.222.0.25 () {58 vars in 1024 bytes} [Sun Aug 7 10:41:27 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 07 10:41:27.474384 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40f6e0c2-b9e5-4a62-84b4-af5eb962d097 None None] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 10:41:27.476760 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c694c80d-c760-4328-b0dc-87294bd0b48c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 10:41:27.477377 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c694c80d-c760-4328-b0dc-87294bd0b48c None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:41:27.477707 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c694c80d-c760-4328-b0dc-87294bd0b48c None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:41:27.477990 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c694c80d-c760-4328-b0dc-87294bd0b48c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:41:27.478739 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c694c80d-c760-4328-b0dc-87294bd0b48c None admin] RBAC: Adding query filter params (role.id=b98dfb3b9fee4644af2887a3ac3fd2e6, user.id=5a4d08b8bd68447fa7d2f23a95eee88b, scope.system=all) {{(pid=80143) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 10:41:27.479134 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c694c80d-c760-4328-b0dc-87294bd0b48c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 10:41:27.481154 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c694c80d-c760-4328-b0dc-87294bd0b48c None admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 10:41:27.487345 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 136/286] 10.222.0.25 () {56 vars in 1308 bytes} [Sun Aug 7 10:41:27 2022] GET /identity/v3/role_assignments?user.id=5a4d08b8bd68447fa7d2f23a95eee88b&scope.system=all&role.id=b98dfb3b9fee4644af2887a3ac3fd2e6 => generated 226 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 07 10:41:27.503474 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c694c80d-c760-4328-b0dc-87294bd0b48c None admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 10:41:27.544889 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=100 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 10:41:27.640943 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40f6e0c2-b9e5-4a62-84b4-af5eb962d097 None None] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 10:41:27.661919 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c694c80d-c760-4328-b0dc-87294bd0b48c None admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 10:41:27.666565 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=100 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 10:41:27.694097 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68c7a660-c72d-45ae-a0f1-b4ddbf450266 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 10:41:27.695131 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68c7a660-c72d-45ae-a0f1-b4ddbf450266 None admin] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:41:27.695639 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68c7a660-c72d-45ae-a0f1-b4ddbf450266 None admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:41:27.696114 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68c7a660-c72d-45ae-a0f1-b4ddbf450266 None admin] PATH_INFO: `/v3/projects/d88586f68b734382abb9836b290aeb54` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:41:27.698373 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68c7a660-c72d-45ae-a0f1-b4ddbf450266 None admin] RBAC: Authorizing `identity:get_project(project_id=d88586f68b734382abb9836b290aeb54)` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 10:41:27.700851 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68c7a660-c72d-45ae-a0f1-b4ddbf450266 None admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 10:41:27.701689 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f855b2a8-4df0-4541-b819-bbc390e42291 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 10:41:27.702327 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f855b2a8-4df0-4541-b819-bbc390e42291 None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:41:27.702613 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f855b2a8-4df0-4541-b819-bbc390e42291 None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:41:27.702888 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f855b2a8-4df0-4541-b819-bbc390e42291 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:41:27.711062 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 151/287] 10.222.0.25 () {56 vars in 1186 bytes} [Sun Aug 7 10:41:27 2022] GET /identity/v3/projects/d88586f68b734382abb9836b290aeb54 => generated 302 bytes in 234 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 10:41:27.716123 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68c7a660-c72d-45ae-a0f1-b4ddbf450266 None admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 10:41:27.734734 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f855b2a8-4df0-4541-b819-bbc390e42291 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 07 10:41:27.742131 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application [None req-f855b2a8-4df0-4541-b819-bbc390e42291 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 07 10:41:27.742131 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 07 10:41:27.742131 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 07 10:41:27.742131 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 07 10:41:27.742131 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 07 10:41:27.742131 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 07 10:41:27.742131 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 07 10:41:27.742131 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 07 10:41:27.742131 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 07 10:41:27.742131 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 07 10:41:27.742131 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 07 10:41:27.742131 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 07 10:41:27.742131 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 07 10:41:27.742131 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 07 10:41:27.742131 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 07 10:41:27.742131 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 07 10:41:27.743053 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 07 10:41:27.743053 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 07 10:41:27.743053 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 07 10:41:27.743053 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 07 10:41:27.743053 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 07 10:41:27.743053 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 07 10:41:27.743053 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 07 10:41:27.743053 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 07 10:41:27.743053 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 07 10:41:27.743053 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.get_or_create( Aug 07 10:41:27.743053 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 07 10:41:27.743053 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application with Lock( Aug 07 10:41:27.743053 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 07 10:41:27.743053 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self._enter() Aug 07 10:41:27.743053 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 07 10:41:27.743053 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 07 10:41:27.743854 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 07 10:41:27.743854 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.creator() Aug 07 10:41:27.743854 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 07 10:41:27.743854 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application created_value = creator( Aug 07 10:41:27.743854 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 07 10:41:27.743854 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 07 10:41:27.743854 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 07 10:41:27.743854 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 07 10:41:27.743854 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 07 10:41:27.743854 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 07 10:41:27.743854 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 07 10:41:27.743854 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application Aug 07 10:41:27.743854 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 137/288] 10.222.0.25 () {56 vars in 1099 bytes} [Sun Aug 7 10:41:27 2022] GET /identity/v3/roles/member => generated 84 bytes in 241 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 07 10:41:27.750224 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f855b2a8-4df0-4541-b819-bbc390e42291 None admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 10:41:27.852482 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=100 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 10:41:27.863822 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f855b2a8-4df0-4541-b819-bbc390e42291 None admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 10:41:27.901916 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-390d602f-3789-4de5-b948-eccf5d0d0129 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 10:41:27.902970 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-390d602f-3789-4de5-b948-eccf5d0d0129 None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:41:27.903265 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-390d602f-3789-4de5-b948-eccf5d0d0129 None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:41:27.903541 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-390d602f-3789-4de5-b948-eccf5d0d0129 None admin] PATH_INFO: `/v3/roles` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:41:27.904275 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-390d602f-3789-4de5-b948-eccf5d0d0129 None admin] RBAC: Adding query filter params (name=member) {{(pid=80143) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 10:41:27.904583 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-390d602f-3789-4de5-b948-eccf5d0d0129 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 10:41:27.906028 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-390d602f-3789-4de5-b948-eccf5d0d0129 None admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 10:41:27.916352 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 138/289] 10.222.0.25 () {56 vars in 1101 bytes} [Sun Aug 7 10:41:27 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 10:41:27.923049 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-390d602f-3789-4de5-b948-eccf5d0d0129 None admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 10:41:27.953772 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68c7a660-c72d-45ae-a0f1-b4ddbf450266 None admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 10:41:28.032709 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0129247d-bb10-45a5-b872-6bfef99f0065 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 10:41:28.033488 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0129247d-bb10-45a5-b872-6bfef99f0065 None admin] REQUEST_METHOD: `PUT` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:41:28.033668 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0129247d-bb10-45a5-b872-6bfef99f0065 None admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:41:28.033944 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0129247d-bb10-45a5-b872-6bfef99f0065 None admin] PATH_INFO: `/v3/projects/d88586f68b734382abb9836b290aeb54/users/efa5629181584c3c8f0385d3afb56e64/roles/567b9b8669874c9986d68642dc9ef845` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:41:28.036564 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0129247d-bb10-45a5-b872-6bfef99f0065 None admin] RBAC: Authorizing `identity:create_grant(project_id=d88586f68b734382abb9836b290aeb54, user_id=efa5629181584c3c8f0385d3afb56e64, role_id=567b9b8669874c9986d68642dc9ef845)` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 10:41:28.037993 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0129247d-bb10-45a5-b872-6bfef99f0065 None admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 10:41:28.064868 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 152/290] 10.222.0.25 () {58 vars in 1439 bytes} [Sun Aug 7 10:41:27 2022] PUT /identity/v3/projects/d88586f68b734382abb9836b290aeb54/users/efa5629181584c3c8f0385d3afb56e64/roles/567b9b8669874c9986d68642dc9ef845 => generated 0 bytes in 349 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 07 10:41:28.080040 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-390d602f-3789-4de5-b948-eccf5d0d0129 None admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 10:41:28.126963 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55c438f3-5ff9-42ef-becd-769778d5010f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 10:41:28.127557 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55c438f3-5ff9-42ef-becd-769778d5010f None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:41:28.127836 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55c438f3-5ff9-42ef-becd-769778d5010f None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:41:28.128110 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55c438f3-5ff9-42ef-becd-769778d5010f None admin] PATH_INFO: `/v3/users/e01f8bc675bc4dd4985cd9f3b1bac86d` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:41:28.129318 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55c438f3-5ff9-42ef-becd-769778d5010f None admin] RBAC: Authorizing `identity:get_user(user_id=e01f8bc675bc4dd4985cd9f3b1bac86d)` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 10:41:28.130787 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55c438f3-5ff9-42ef-becd-769778d5010f None admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 10:41:28.132539 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 139/291] 10.222.0.25 () {56 vars in 1177 bytes} [Sun Aug 7 10:41:27 2022] GET /identity/v3/users/e01f8bc675bc4dd4985cd9f3b1bac86d => generated 297 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 10:41:28.138479 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0129247d-bb10-45a5-b872-6bfef99f0065 None admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 10:41:28.194082 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 10:41:28.278399 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0129247d-bb10-45a5-b872-6bfef99f0065 None admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 10:41:28.314055 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acce19a9-5e30-43a7-a6d0-baba6dc8ff7a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 10:41:28.314654 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acce19a9-5e30-43a7-a6d0-baba6dc8ff7a None admin] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:41:28.314948 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acce19a9-5e30-43a7-a6d0-baba6dc8ff7a None admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:41:28.315229 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acce19a9-5e30-43a7-a6d0-baba6dc8ff7a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:41:28.388272 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acce19a9-5e30-43a7-a6d0-baba6dc8ff7a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 10:41:28.389666 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acce19a9-5e30-43a7-a6d0-baba6dc8ff7a None admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 10:41:28.432146 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=100 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 10:41:28.492533 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 153/292] 10.222.0.25 () {58 vars in 1282 bytes} [Sun Aug 7 10:41:28 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 354 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 07 10:41:28.498561 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55c438f3-5ff9-42ef-becd-769778d5010f None admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 10:41:28.535531 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=100 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 10:41:28.643161 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55c438f3-5ff9-42ef-becd-769778d5010f None admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 10:41:28.686678 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3abfde07-df8c-472e-9506-feed884dce69 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 10:41:28.687344 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3abfde07-df8c-472e-9506-feed884dce69 None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:41:28.687627 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3abfde07-df8c-472e-9506-feed884dce69 None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:41:28.687904 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3abfde07-df8c-472e-9506-feed884dce69 None admin] PATH_INFO: `/v3/projects/d88586f68b734382abb9836b290aeb54` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:41:28.691598 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3abfde07-df8c-472e-9506-feed884dce69 None admin] RBAC: Authorizing `identity:get_project(project_id=d88586f68b734382abb9836b290aeb54)` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 10:41:28.693059 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3abfde07-df8c-472e-9506-feed884dce69 None admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 10:41:28.694702 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 140/293] 10.222.0.25 () {56 vars in 1186 bytes} [Sun Aug 7 10:41:28 2022] GET /identity/v3/projects/d88586f68b734382abb9836b290aeb54 => generated 302 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 10:41:28.701394 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acce19a9-5e30-43a7-a6d0-baba6dc8ff7a None admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 10:41:28.813873 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acce19a9-5e30-43a7-a6d0-baba6dc8ff7a None admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 10:41:28.848886 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-277e9ceb-7b3c-4a3e-a78c-776a360c2b47 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 10:41:28.849494 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-277e9ceb-7b3c-4a3e-a78c-776a360c2b47 None admin] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:41:28.849771 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-277e9ceb-7b3c-4a3e-a78c-776a360c2b47 None admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:41:28.850041 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-277e9ceb-7b3c-4a3e-a78c-776a360c2b47 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:41:28.850757 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-277e9ceb-7b3c-4a3e-a78c-776a360c2b47 None admin] RBAC: Adding query filter params (role.id=b98dfb3b9fee4644af2887a3ac3fd2e6, scope.project.id=d88586f68b734382abb9836b290aeb54, user.id=e01f8bc675bc4dd4985cd9f3b1bac86d) {{(pid=80144) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 10:41:28.851097 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-277e9ceb-7b3c-4a3e-a78c-776a360c2b47 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 10:41:28.852514 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-277e9ceb-7b3c-4a3e-a78c-776a360c2b47 None admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 10:41:28.859353 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 154/294] 10.222.0.25 () {56 vars in 1374 bytes} [Sun Aug 7 10:41:28 2022] GET /identity/v3/role_assignments?user.id=e01f8bc675bc4dd4985cd9f3b1bac86d&scope.project.id=d88586f68b734382abb9836b290aeb54&role.id=b98dfb3b9fee4644af2887a3ac3fd2e6 => generated 608 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 10:41:29.599366 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c36f34c3-609f-4dc4-a5ce-a713d6135cad None None] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:41:29.599860 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c36f34c3-609f-4dc4-a5ce-a713d6135cad None None] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:41:29.599933 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c36f34c3-609f-4dc4-a5ce-a713d6135cad None None] PATH_INFO: `/` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:41:29.601497 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 141/295] 10.222.0.25 () {54 vars in 927 bytes} [Sun Aug 7 10:41:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 07 10:41:29.607036 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b06a4492-e6ad-48a5-ae79-03103d07f3f6 None None] REQUEST_METHOD: `POST` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:41:29.607431 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b06a4492-e6ad-48a5-ae79-03103d07f3f6 None None] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:41:29.607615 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b06a4492-e6ad-48a5-ae79-03103d07f3f6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:41:29.615184 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=100 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 10:41:29.626908 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.auth.core [None req-b06a4492-e6ad-48a5-ae79-03103d07f3f6 None None] MFA Rules not processed for user `bac4e6b76fe64217ac8fa44e71cffe6a`. Rule list: `[]` (Enabled: `True`). {{(pid=80144) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 10:41:29.628277 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.fernet_utils [None req-b06a4492-e6ad-48a5-ae79-03103d07f3f6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80144) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 10:41:29.697108 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 155/296] 10.222.0.25 () {58 vars in 1024 bytes} [Sun Aug 7 10:41:29 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 07 10:41:29.719707 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c36f34c3-609f-4dc4-a5ce-a713d6135cad None None] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 10:41:29.815312 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=100 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 10:41:29.825582 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c36f34c3-609f-4dc4-a5ce-a713d6135cad None None] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 10:41:29.860364 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89f2c087-7adc-45ae-8c11-534eafc30276 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 10:41:29.861226 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89f2c087-7adc-45ae-8c11-534eafc30276 None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:41:29.861507 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89f2c087-7adc-45ae-8c11-534eafc30276 None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:41:29.861782 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89f2c087-7adc-45ae-8c11-534eafc30276 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:41:29.873334 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89f2c087-7adc-45ae-8c11-534eafc30276 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 07 10:41:29.878504 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application [None req-89f2c087-7adc-45ae-8c11-534eafc30276 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Aug 07 10:41:29.878504 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 07 10:41:29.878504 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 07 10:41:29.878504 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 07 10:41:29.878504 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 07 10:41:29.878504 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 07 10:41:29.878504 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 07 10:41:29.878504 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 07 10:41:29.878504 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 07 10:41:29.878504 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 07 10:41:29.878504 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 07 10:41:29.878504 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 07 10:41:29.878504 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 07 10:41:29.878504 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 07 10:41:29.878504 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 07 10:41:29.878504 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 07 10:41:29.880717 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 07 10:41:29.880717 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 07 10:41:29.880717 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 07 10:41:29.880717 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 07 10:41:29.880717 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 07 10:41:29.880717 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 07 10:41:29.880717 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 07 10:41:29.880717 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 07 10:41:29.880717 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 07 10:41:29.880717 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.get_or_create( Aug 07 10:41:29.880717 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 07 10:41:29.880717 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application with Lock( Aug 07 10:41:29.880717 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 07 10:41:29.880717 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self._enter() Aug 07 10:41:29.880717 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 07 10:41:29.880717 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 07 10:41:29.882850 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 07 10:41:29.882850 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.creator() Aug 07 10:41:29.882850 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 07 10:41:29.882850 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application created_value = creator( Aug 07 10:41:29.882850 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 07 10:41:29.882850 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 07 10:41:29.882850 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 07 10:41:29.882850 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 07 10:41:29.882850 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 07 10:41:29.882850 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 07 10:41:29.882850 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Aug 07 10:41:29.882850 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application Aug 07 10:41:29.882850 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 142/297] 10.222.0.25 () {56 vars in 1099 bytes} [Sun Aug 7 10:41:29 2022] GET /identity/v3/roles/reader => generated 84 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 07 10:41:29.888368 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc33b3eb-9f50-4685-a54c-3cf241ffc355 None None] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:41:29.888368 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc33b3eb-9f50-4685-a54c-3cf241ffc355 None None] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:41:29.888368 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc33b3eb-9f50-4685-a54c-3cf241ffc355 None None] PATH_INFO: `/` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:41:29.889138 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89f2c087-7adc-45ae-8c11-534eafc30276 None admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 10:41:29.889239 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 156/298] 10.222.0.25 () {54 vars in 927 bytes} [Sun Aug 7 10:41:29 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 07 10:41:29.893882 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86e6321f-1858-47fe-8fa7-151d27d99428 None None] REQUEST_METHOD: `POST` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:41:29.894169 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86e6321f-1858-47fe-8fa7-151d27d99428 None None] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:41:29.894451 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86e6321f-1858-47fe-8fa7-151d27d99428 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:41:29.913247 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.auth.core [None req-86e6321f-1858-47fe-8fa7-151d27d99428 None None] MFA Rules not processed for user `bac4e6b76fe64217ac8fa44e71cffe6a`. Rule list: `[]` (Enabled: `True`). {{(pid=80144) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 10:41:29.914423 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.fernet_utils [None req-86e6321f-1858-47fe-8fa7-151d27d99428 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80144) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 10:41:29.985107 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 157/299] 10.222.0.25 () {58 vars in 1024 bytes} [Sun Aug 7 10:41:29 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 07 10:41:29.992305 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f922cb42-a6e3-4357-a4ea-8ad67652411e None None] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:41:29.992526 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89f2c087-7adc-45ae-8c11-534eafc30276 None admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 10:41:29.993035 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f922cb42-a6e3-4357-a4ea-8ad67652411e None None] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:41:29.993512 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f922cb42-a6e3-4357-a4ea-8ad67652411e None None] PATH_INFO: `/` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:41:30.000011 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 158/300] 10.222.0.25 () {54 vars in 927 bytes} [Sun Aug 7 10:41:29 2022] GET /identity => generated 270 bytes in 14 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 07 10:41:30.001689 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42604231-2fe0-4d99-915b-7b15926acf59 None None] REQUEST_METHOD: `POST` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:41:30.002208 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42604231-2fe0-4d99-915b-7b15926acf59 None None] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:41:30.002717 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42604231-2fe0-4d99-915b-7b15926acf59 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:41:30.021294 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.auth.core [None req-42604231-2fe0-4d99-915b-7b15926acf59 None None] MFA Rules not processed for user `bac4e6b76fe64217ac8fa44e71cffe6a`. Rule list: `[]` (Enabled: `True`). {{(pid=80144) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 10:41:30.022705 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.fernet_utils [None req-42604231-2fe0-4d99-915b-7b15926acf59 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80144) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 10:41:30.029506 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bba235d-3d6d-4abe-b0ee-58794b105f54 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 10:41:30.030121 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bba235d-3d6d-4abe-b0ee-58794b105f54 None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:41:30.030398 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bba235d-3d6d-4abe-b0ee-58794b105f54 None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:41:30.030666 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bba235d-3d6d-4abe-b0ee-58794b105f54 None admin] PATH_INFO: `/v3/roles` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:41:30.031700 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bba235d-3d6d-4abe-b0ee-58794b105f54 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80143) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 10:41:30.031700 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bba235d-3d6d-4abe-b0ee-58794b105f54 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 10:41:30.033020 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bba235d-3d6d-4abe-b0ee-58794b105f54 None admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 10:41:30.043202 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 143/301] 10.222.0.25 () {56 vars in 1101 bytes} [Sun Aug 7 10:41:29 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 10:41:30.043849 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bba235d-3d6d-4abe-b0ee-58794b105f54 None admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 10:41:30.056763 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=100 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 10:41:30.092362 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=100 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 10:41:30.101789 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 159/302] 10.222.0.25 () {58 vars in 1024 bytes} [Sun Aug 7 10:41:30 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 07 10:41:30.111329 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42604231-2fe0-4d99-915b-7b15926acf59 None None] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 10:41:30.153023 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bba235d-3d6d-4abe-b0ee-58794b105f54 None admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 10:41:30.190544 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d24a896f-44d3-4d37-9e2c-40c5a4ecb1f6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 10:41:30.191308 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d24a896f-44d3-4d37-9e2c-40c5a4ecb1f6 None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:41:30.191583 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d24a896f-44d3-4d37-9e2c-40c5a4ecb1f6 None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:41:30.191849 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d24a896f-44d3-4d37-9e2c-40c5a4ecb1f6 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:41:30.193077 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d24a896f-44d3-4d37-9e2c-40c5a4ecb1f6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 10:41:30.194634 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d24a896f-44d3-4d37-9e2c-40c5a4ecb1f6 None admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 10:41:30.200807 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 144/303] 10.222.0.25 () {56 vars in 1108 bytes} [Sun Aug 7 10:41:30 2022] GET /identity/v3/domains/default => generated 199 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 10:41:30.200807 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d24a896f-44d3-4d37-9e2c-40c5a4ecb1f6 None admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 10:41:30.220723 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42604231-2fe0-4d99-915b-7b15926acf59 None None] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 10:41:30.256107 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd8d62cc-e332-44b5-97c9-6fafc23d2207 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 10:41:30.256753 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd8d62cc-e332-44b5-97c9-6fafc23d2207 None admin] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:41:30.257036 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd8d62cc-e332-44b5-97c9-6fafc23d2207 None admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:41:30.257317 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd8d62cc-e332-44b5-97c9-6fafc23d2207 None admin] PATH_INFO: `/v3/users/affbb5676bbf4e7d901f9afdb707c399` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:41:30.271338 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd8d62cc-e332-44b5-97c9-6fafc23d2207 None admin] RBAC: Authorizing `identity:get_user(user_id=affbb5676bbf4e7d901f9afdb707c399)` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 10:41:30.273753 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd8d62cc-e332-44b5-97c9-6fafc23d2207 None admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 10:41:30.275584 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 160/304] 10.222.0.25 () {56 vars in 1177 bytes} [Sun Aug 7 10:41:30 2022] GET /identity/v3/users/affbb5676bbf4e7d901f9afdb707c399 => generated 293 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 10:41:30.276662 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd8d62cc-e332-44b5-97c9-6fafc23d2207 None admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 10:41:30.282613 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=100 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 10:41:30.309260 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d24a896f-44d3-4d37-9e2c-40c5a4ecb1f6 None admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 10:41:30.335870 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=100 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 10:41:30.346194 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-420d698e-a018-411e-8681-592ef73342bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 10:41:30.346787 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-420d698e-a018-411e-8681-592ef73342bf None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:41:30.347171 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-420d698e-a018-411e-8681-592ef73342bf None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:41:30.347431 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-420d698e-a018-411e-8681-592ef73342bf None admin] PATH_INFO: `/v3/roles/member` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:41:30.358392 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-420d698e-a018-411e-8681-592ef73342bf None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 07 10:41:30.363959 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application [None req-420d698e-a018-411e-8681-592ef73342bf None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 07 10:41:30.363959 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 07 10:41:30.363959 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 07 10:41:30.363959 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 07 10:41:30.363959 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 07 10:41:30.363959 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 07 10:41:30.363959 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 07 10:41:30.363959 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 07 10:41:30.363959 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 07 10:41:30.363959 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 07 10:41:30.363959 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 07 10:41:30.363959 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 07 10:41:30.363959 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 07 10:41:30.363959 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 07 10:41:30.363959 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 07 10:41:30.363959 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 07 10:41:30.364803 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 07 10:41:30.364803 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 07 10:41:30.364803 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 07 10:41:30.364803 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 07 10:41:30.364803 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 07 10:41:30.364803 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 07 10:41:30.364803 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 07 10:41:30.364803 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 07 10:41:30.364803 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 07 10:41:30.364803 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.get_or_create( Aug 07 10:41:30.364803 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 07 10:41:30.364803 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application with Lock( Aug 07 10:41:30.364803 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 07 10:41:30.364803 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self._enter() Aug 07 10:41:30.364803 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 07 10:41:30.364803 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 07 10:41:30.365590 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 07 10:41:30.365590 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.creator() Aug 07 10:41:30.365590 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 07 10:41:30.365590 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application created_value = creator( Aug 07 10:41:30.365590 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 07 10:41:30.365590 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 07 10:41:30.365590 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 07 10:41:30.365590 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 07 10:41:30.365590 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 07 10:41:30.365590 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 07 10:41:30.365590 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 07 10:41:30.365590 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application Aug 07 10:41:30.365590 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 145/305] 10.222.0.25 () {56 vars in 1099 bytes} [Sun Aug 7 10:41:30 2022] GET /identity/v3/roles/member => generated 84 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 07 10:41:30.366308 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-420d698e-a018-411e-8681-592ef73342bf None admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 10:41:30.394391 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd8d62cc-e332-44b5-97c9-6fafc23d2207 None admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 10:41:30.437541 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bea1da99-fdf3-4d99-b790-8e5e1d7a972e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 10:41:30.438212 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bea1da99-fdf3-4d99-b790-8e5e1d7a972e None admin] REQUEST_METHOD: `POST` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:41:30.438498 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bea1da99-fdf3-4d99-b790-8e5e1d7a972e None admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:41:30.438766 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bea1da99-fdf3-4d99-b790-8e5e1d7a972e None admin] PATH_INFO: `/v3/groups` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:41:30.439420 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bea1da99-fdf3-4d99-b790-8e5e1d7a972e None admin] RBAC: Authorizing `identity:create_group()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 10:41:30.440844 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bea1da99-fdf3-4d99-b790-8e5e1d7a972e None admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 10:41:30.458158 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 161/306] 10.222.0.25 () {60 vars in 1134 bytes} [Sun Aug 7 10:41:30 2022] POST /identity/v3/groups => generated 229 bytes in 182 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 07 10:41:30.459216 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bea1da99-fdf3-4d99-b790-8e5e1d7a972e None admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 10:41:30.477042 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-420d698e-a018-411e-8681-592ef73342bf None admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 10:41:30.521718 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06eaf030-a2cc-4e4c-8e6a-b358e4510004 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 10:41:30.522658 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06eaf030-a2cc-4e4c-8e6a-b358e4510004 None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:41:30.522973 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06eaf030-a2cc-4e4c-8e6a-b358e4510004 None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:41:30.523256 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06eaf030-a2cc-4e4c-8e6a-b358e4510004 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:41:30.524002 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06eaf030-a2cc-4e4c-8e6a-b358e4510004 None admin] RBAC: Adding query filter params (role.id=567b9b8669874c9986d68642dc9ef845, user.id=affbb5676bbf4e7d901f9afdb707c399, scope.system=all) {{(pid=80143) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 10:41:30.524319 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06eaf030-a2cc-4e4c-8e6a-b358e4510004 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 10:41:30.527170 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06eaf030-a2cc-4e4c-8e6a-b358e4510004 None admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 10:41:30.540798 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 146/307] 10.222.0.25 () {56 vars in 1308 bytes} [Sun Aug 7 10:41:30 2022] GET /identity/v3/role_assignments?user.id=affbb5676bbf4e7d901f9afdb707c399&scope.system=all&role.id=567b9b8669874c9986d68642dc9ef845 => generated 226 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 10:41:30.540798 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9b63da3-34f7-4f78-a5f4-4581b1553558 None None] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:41:30.540798 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9b63da3-34f7-4f78-a5f4-4581b1553558 None None] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:41:30.540798 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9b63da3-34f7-4f78-a5f4-4581b1553558 None None] PATH_INFO: `/` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:41:30.540798 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 147/308] 10.222.0.25 () {54 vars in 927 bytes} [Sun Aug 7 10:41:30 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 07 10:41:30.545997 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb53569c-c039-4289-b82c-c28fd98dfc04 None None] REQUEST_METHOD: `POST` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:41:30.548060 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb53569c-c039-4289-b82c-c28fd98dfc04 None None] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:41:30.548060 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb53569c-c039-4289-b82c-c28fd98dfc04 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:41:30.574028 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.auth.core [None req-eb53569c-c039-4289-b82c-c28fd98dfc04 None None] MFA Rules not processed for user `bac4e6b76fe64217ac8fa44e71cffe6a`. Rule list: `[]` (Enabled: `True`). {{(pid=80143) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 10:41:30.575481 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-eb53569c-c039-4289-b82c-c28fd98dfc04 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 10:41:30.580849 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=100 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 10:41:30.592733 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bea1da99-fdf3-4d99-b790-8e5e1d7a972e None admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 10:41:30.614372 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 10:41:30.651670 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92e8ebdd-8628-4930-b62e-67aae1bed515 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 10:41:30.652309 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92e8ebdd-8628-4930-b62e-67aae1bed515 None admin] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:41:30.652592 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92e8ebdd-8628-4930-b62e-67aae1bed515 None admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:41:30.652919 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92e8ebdd-8628-4930-b62e-67aae1bed515 None admin] PATH_INFO: `/v3/roles` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:41:30.654203 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92e8ebdd-8628-4930-b62e-67aae1bed515 None admin] RBAC: Adding query filter params (name=member) {{(pid=80144) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 10:41:30.654511 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92e8ebdd-8628-4930-b62e-67aae1bed515 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 10:41:30.655962 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92e8ebdd-8628-4930-b62e-67aae1bed515 None admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 10:41:30.667864 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 162/309] 10.222.0.25 () {56 vars in 1101 bytes} [Sun Aug 7 10:41:30 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 10:41:30.673724 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92e8ebdd-8628-4930-b62e-67aae1bed515 None admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 10:41:30.675138 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 148/310] 10.222.0.25 () {58 vars in 1024 bytes} [Sun Aug 7 10:41:30 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 07 10:41:30.703727 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb53569c-c039-4289-b82c-c28fd98dfc04 None None] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 10:41:30.796141 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92e8ebdd-8628-4930-b62e-67aae1bed515 None admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 10:41:30.825895 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb53569c-c039-4289-b82c-c28fd98dfc04 None None] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 10:41:30.838630 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3090228-ba8a-48b9-b755-fde765276fab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 10:41:30.839425 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3090228-ba8a-48b9-b755-fde765276fab None admin] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:41:30.839715 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3090228-ba8a-48b9-b755-fde765276fab None admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:41:30.839993 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3090228-ba8a-48b9-b755-fde765276fab None admin] PATH_INFO: `/v3/users/5a4d08b8bd68447fa7d2f23a95eee88b` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:41:30.841713 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3090228-ba8a-48b9-b755-fde765276fab None admin] RBAC: Authorizing `identity:get_user(user_id=5a4d08b8bd68447fa7d2f23a95eee88b)` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 10:41:30.843478 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3090228-ba8a-48b9-b755-fde765276fab None admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 10:41:30.845831 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 163/311] 10.222.0.25 () {56 vars in 1177 bytes} [Sun Aug 7 10:41:30 2022] GET /identity/v3/users/5a4d08b8bd68447fa7d2f23a95eee88b => generated 293 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 07 10:41:30.851744 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3090228-ba8a-48b9-b755-fde765276fab None admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 10:41:30.863212 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=100 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 10:41:30.870657 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c74807f-4f91-41ee-86b4-7d388345f75c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 10:41:30.871881 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c74807f-4f91-41ee-86b4-7d388345f75c None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:41:30.872311 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c74807f-4f91-41ee-86b4-7d388345f75c None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:41:30.872786 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c74807f-4f91-41ee-86b4-7d388345f75c None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:41:30.888852 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c74807f-4f91-41ee-86b4-7d388345f75c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 07 10:41:30.893982 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application [None req-8c74807f-4f91-41ee-86b4-7d388345f75c None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Aug 07 10:41:30.893982 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 07 10:41:30.893982 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 07 10:41:30.893982 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 07 10:41:30.893982 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 07 10:41:30.893982 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 07 10:41:30.893982 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 07 10:41:30.893982 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 07 10:41:30.893982 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 07 10:41:30.893982 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 07 10:41:30.893982 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 07 10:41:30.893982 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 07 10:41:30.893982 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 07 10:41:30.893982 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 07 10:41:30.893982 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 07 10:41:30.893982 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 07 10:41:30.895080 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 07 10:41:30.895080 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 07 10:41:30.895080 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 07 10:41:30.895080 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 07 10:41:30.895080 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 07 10:41:30.895080 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 07 10:41:30.895080 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 07 10:41:30.895080 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 07 10:41:30.895080 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 07 10:41:30.895080 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.get_or_create( Aug 07 10:41:30.895080 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 07 10:41:30.895080 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application with Lock( Aug 07 10:41:30.895080 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 07 10:41:30.895080 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self._enter() Aug 07 10:41:30.895080 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 07 10:41:30.895080 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 07 10:41:30.895966 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 07 10:41:30.895966 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.creator() Aug 07 10:41:30.895966 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 07 10:41:30.895966 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application created_value = creator( Aug 07 10:41:30.895966 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 07 10:41:30.895966 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 07 10:41:30.895966 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 07 10:41:30.895966 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 07 10:41:30.895966 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 07 10:41:30.895966 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 07 10:41:30.895966 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Aug 07 10:41:30.895966 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application Aug 07 10:41:30.898178 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 149/312] 10.222.0.25 () {56 vars in 1099 bytes} [Sun Aug 7 10:41:30 2022] GET /identity/v3/roles/reader => generated 84 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 07 10:41:30.904424 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c74807f-4f91-41ee-86b4-7d388345f75c None admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 10:41:30.907057 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=100 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 10:41:30.978100 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3090228-ba8a-48b9-b755-fde765276fab None admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 10:41:31.014603 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c74807f-4f91-41ee-86b4-7d388345f75c None admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 10:41:31.018172 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2127059-47f2-4691-8232-2d31ceef1adc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 10:41:31.019052 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2127059-47f2-4691-8232-2d31ceef1adc None admin] REQUEST_METHOD: `PUT` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:41:31.019494 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2127059-47f2-4691-8232-2d31ceef1adc None admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:41:31.019913 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2127059-47f2-4691-8232-2d31ceef1adc None admin] PATH_INFO: `/v3/system/users/5a4d08b8bd68447fa7d2f23a95eee88b/roles/b98dfb3b9fee4644af2887a3ac3fd2e6` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:41:31.021476 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2127059-47f2-4691-8232-2d31ceef1adc None admin] RBAC: Authorizing `identity:create_system_grant_for_user(user_id=5a4d08b8bd68447fa7d2f23a95eee88b, role_id=b98dfb3b9fee4644af2887a3ac3fd2e6)` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 10:41:31.023211 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2127059-47f2-4691-8232-2d31ceef1adc None admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 10:41:31.034418 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 164/313] 10.222.0.25 () {58 vars in 1334 bytes} [Sun Aug 7 10:41:30 2022] PUT /identity/v3/system/users/5a4d08b8bd68447fa7d2f23a95eee88b/roles/b98dfb3b9fee4644af2887a3ac3fd2e6 => generated 0 bytes in 183 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 07 10:41:31.077159 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ca011c8-6b1e-4367-88a2-ec82c83126b9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 10:41:31.077910 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ca011c8-6b1e-4367-88a2-ec82c83126b9 None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:41:31.078489 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ca011c8-6b1e-4367-88a2-ec82c83126b9 None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:41:31.078886 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ca011c8-6b1e-4367-88a2-ec82c83126b9 None admin] PATH_INFO: `/v3/roles` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:41:31.079724 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ca011c8-6b1e-4367-88a2-ec82c83126b9 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80143) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 10:41:31.080156 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ca011c8-6b1e-4367-88a2-ec82c83126b9 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 10:41:31.081762 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ca011c8-6b1e-4367-88a2-ec82c83126b9 None admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 10:41:31.093013 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 150/314] 10.222.0.25 () {56 vars in 1101 bytes} [Sun Aug 7 10:41:30 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 07 10:41:31.098568 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2127059-47f2-4691-8232-2d31ceef1adc None admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 10:41:31.219707 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2127059-47f2-4691-8232-2d31ceef1adc None admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 10:41:31.229364 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 10:41:31.256775 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0599a62a-9e35-4a1b-a75f-938ba01d3b15 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 10:41:31.257432 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0599a62a-9e35-4a1b-a75f-938ba01d3b15 None admin] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:41:31.257711 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0599a62a-9e35-4a1b-a75f-938ba01d3b15 None admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:41:31.257987 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0599a62a-9e35-4a1b-a75f-938ba01d3b15 None admin] PATH_INFO: `/v3/users/efa5629181584c3c8f0385d3afb56e64` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:41:31.259194 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0599a62a-9e35-4a1b-a75f-938ba01d3b15 None admin] RBAC: Authorizing `identity:get_user(user_id=efa5629181584c3c8f0385d3afb56e64)` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 10:41:31.260656 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0599a62a-9e35-4a1b-a75f-938ba01d3b15 None admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 10:41:31.263148 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 165/315] 10.222.0.25 () {56 vars in 1177 bytes} [Sun Aug 7 10:41:31 2022] GET /identity/v3/users/efa5629181584c3c8f0385d3afb56e64 => generated 297 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 10:41:31.268622 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ca011c8-6b1e-4367-88a2-ec82c83126b9 None admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 10:41:31.333130 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=100 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 10:41:31.379236 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ca011c8-6b1e-4367-88a2-ec82c83126b9 None admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 10:41:31.424165 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fa027a8-b3dc-4a1f-8376-47ebef86a2cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 10:41:31.424281 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fa027a8-b3dc-4a1f-8376-47ebef86a2cc None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:41:31.424540 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fa027a8-b3dc-4a1f-8376-47ebef86a2cc None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:41:31.424859 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fa027a8-b3dc-4a1f-8376-47ebef86a2cc None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:41:31.493609 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fa027a8-b3dc-4a1f-8376-47ebef86a2cc None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 10:41:31.495163 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fa027a8-b3dc-4a1f-8376-47ebef86a2cc None admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 10:41:31.548521 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=100 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 10:41:31.595042 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 151/316] 10.222.0.25 () {58 vars in 1282 bytes} [Sun Aug 7 10:41:31 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 327 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 07 10:41:31.600676 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0599a62a-9e35-4a1b-a75f-938ba01d3b15 None admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 10:41:31.715372 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0599a62a-9e35-4a1b-a75f-938ba01d3b15 None admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 10:41:31.754291 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ace76984-1c96-41b4-9ab8-0843d283c4e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 10:41:31.754950 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ace76984-1c96-41b4-9ab8-0843d283c4e5 None admin] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:41:31.755246 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ace76984-1c96-41b4-9ab8-0843d283c4e5 None admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:41:31.755523 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ace76984-1c96-41b4-9ab8-0843d283c4e5 None admin] PATH_INFO: `/v3/projects/d88586f68b734382abb9836b290aeb54` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:41:31.756659 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ace76984-1c96-41b4-9ab8-0843d283c4e5 None admin] RBAC: Authorizing `identity:get_project(project_id=d88586f68b734382abb9836b290aeb54)` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 10:41:31.758096 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ace76984-1c96-41b4-9ab8-0843d283c4e5 None admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 10:41:31.760075 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 166/317] 10.222.0.25 () {56 vars in 1186 bytes} [Sun Aug 7 10:41:31 2022] GET /identity/v3/projects/d88586f68b734382abb9836b290aeb54 => generated 302 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 10:41:31.766335 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fa027a8-b3dc-4a1f-8376-47ebef86a2cc None admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 10:41:31.873391 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fa027a8-b3dc-4a1f-8376-47ebef86a2cc None admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 10:41:31.908881 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b72761b-6290-4577-85b3-ce17b3d4fdea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 10:41:31.909496 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b72761b-6290-4577-85b3-ce17b3d4fdea None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:41:31.909778 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b72761b-6290-4577-85b3-ce17b3d4fdea None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:41:31.910057 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b72761b-6290-4577-85b3-ce17b3d4fdea None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:41:31.910786 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b72761b-6290-4577-85b3-ce17b3d4fdea None admin] RBAC: Adding query filter params (role.id=567b9b8669874c9986d68642dc9ef845, scope.project.id=d88586f68b734382abb9836b290aeb54, user.id=efa5629181584c3c8f0385d3afb56e64) {{(pid=80143) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 10:41:31.911127 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b72761b-6290-4577-85b3-ce17b3d4fdea None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 10:41:31.912554 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b72761b-6290-4577-85b3-ce17b3d4fdea None admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 10:41:31.918858 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 152/318] 10.222.0.25 () {56 vars in 1374 bytes} [Sun Aug 7 10:41:31 2022] GET /identity/v3/role_assignments?user.id=efa5629181584c3c8f0385d3afb56e64&scope.project.id=d88586f68b734382abb9836b290aeb54&role.id=567b9b8669874c9986d68642dc9ef845 => generated 608 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 10:41:32.587790 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8b06967-4132-4250-8cbe-b7b611d41e5f None None] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:41:32.588177 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8b06967-4132-4250-8cbe-b7b611d41e5f None None] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:41:32.588353 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8b06967-4132-4250-8cbe-b7b611d41e5f None None] PATH_INFO: `/` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:41:32.590010 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 167/319] 10.222.0.25 () {54 vars in 927 bytes} [Sun Aug 7 10:41:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 07 10:41:32.595063 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f8b918b-0446-489f-aa69-3cc6f441934b None None] REQUEST_METHOD: `POST` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:41:32.595353 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f8b918b-0446-489f-aa69-3cc6f441934b None None] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:41:32.595626 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f8b918b-0446-489f-aa69-3cc6f441934b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:41:32.614140 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.auth.core [None req-0f8b918b-0446-489f-aa69-3cc6f441934b None None] MFA Rules not processed for user `bac4e6b76fe64217ac8fa44e71cffe6a`. Rule list: `[]` (Enabled: `True`). {{(pid=80143) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 10:41:32.615337 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-0f8b918b-0446-489f-aa69-3cc6f441934b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 10:41:32.628573 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=100 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 10:41:32.670595 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8490188d-fc0e-4bd6-8653-54539a4bdc87 None None] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:41:32.670860 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8490188d-fc0e-4bd6-8653-54539a4bdc87 None None] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:41:32.671175 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8490188d-fc0e-4bd6-8653-54539a4bdc87 None None] PATH_INFO: `/` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:41:32.672626 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 168/320] 10.222.0.25 () {54 vars in 927 bytes} [Sun Aug 7 10:41:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 07 10:41:32.677230 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72cfe01e-d82d-4a27-8308-b7f77dd4d7b5 None None] REQUEST_METHOD: `POST` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:41:32.677519 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72cfe01e-d82d-4a27-8308-b7f77dd4d7b5 None None] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:41:32.677796 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72cfe01e-d82d-4a27-8308-b7f77dd4d7b5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:41:32.685242 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 153/321] 10.222.0.25 () {58 vars in 1024 bytes} [Sun Aug 7 10:41:32 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 07 10:41:32.696423 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.auth.core [None req-72cfe01e-d82d-4a27-8308-b7f77dd4d7b5 None None] MFA Rules not processed for user `bac4e6b76fe64217ac8fa44e71cffe6a`. Rule list: `[]` (Enabled: `True`). {{(pid=80144) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 10:41:32.697613 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.fernet_utils [None req-72cfe01e-d82d-4a27-8308-b7f77dd4d7b5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80144) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 10:41:32.706724 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f8b918b-0446-489f-aa69-3cc6f441934b None None] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 10:41:32.736505 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=100 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 10:41:32.771673 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 169/322] 10.222.0.25 () {58 vars in 1024 bytes} [Sun Aug 7 10:41:32 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 07 10:41:32.793488 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72cfe01e-d82d-4a27-8308-b7f77dd4d7b5 None None] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 10:41:32.815535 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f8b918b-0446-489f-aa69-3cc6f441934b None None] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 10:41:32.853146 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-290ec849-4c0a-457e-ba4d-b7ceabbce73a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 10:41:32.854067 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-290ec849-4c0a-457e-ba4d-b7ceabbce73a None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:41:32.854588 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-290ec849-4c0a-457e-ba4d-b7ceabbce73a None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:41:32.855331 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-290ec849-4c0a-457e-ba4d-b7ceabbce73a None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:41:32.866402 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-290ec849-4c0a-457e-ba4d-b7ceabbce73a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 07 10:41:32.872237 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application [None req-290ec849-4c0a-457e-ba4d-b7ceabbce73a None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Aug 07 10:41:32.872237 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 07 10:41:32.872237 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 07 10:41:32.872237 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 07 10:41:32.872237 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 07 10:41:32.872237 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 07 10:41:32.872237 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 07 10:41:32.872237 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 07 10:41:32.872237 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 07 10:41:32.872237 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 07 10:41:32.872237 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 07 10:41:32.872237 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 07 10:41:32.872237 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 07 10:41:32.872237 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 07 10:41:32.872237 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 07 10:41:32.872237 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 07 10:41:32.873305 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 07 10:41:32.873305 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 07 10:41:32.873305 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 07 10:41:32.873305 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 07 10:41:32.873305 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 07 10:41:32.873305 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 07 10:41:32.873305 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 07 10:41:32.873305 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 07 10:41:32.873305 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 07 10:41:32.873305 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.get_or_create( Aug 07 10:41:32.873305 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 07 10:41:32.873305 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application with Lock( Aug 07 10:41:32.873305 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 07 10:41:32.873305 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self._enter() Aug 07 10:41:32.873305 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 07 10:41:32.873305 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 07 10:41:32.874273 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 07 10:41:32.874273 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.creator() Aug 07 10:41:32.874273 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 07 10:41:32.874273 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application created_value = creator( Aug 07 10:41:32.874273 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 07 10:41:32.874273 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 07 10:41:32.874273 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 07 10:41:32.874273 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 07 10:41:32.874273 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 07 10:41:32.874273 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 07 10:41:32.874273 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Aug 07 10:41:32.874273 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application Aug 07 10:41:32.876245 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 154/323] 10.222.0.25 () {56 vars in 1099 bytes} [Sun Aug 7 10:41:32 2022] GET /identity/v3/roles/reader => generated 84 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 07 10:41:32.881854 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-290ec849-4c0a-457e-ba4d-b7ceabbce73a None admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 10:41:32.894145 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=100 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 10:41:32.896803 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72cfe01e-d82d-4a27-8308-b7f77dd4d7b5 None None] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 10:41:32.930346 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-810958af-41bd-4315-9d22-94fd0c05cbba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 10:41:32.930900 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-810958af-41bd-4315-9d22-94fd0c05cbba None admin] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:41:32.931206 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-810958af-41bd-4315-9d22-94fd0c05cbba None admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:41:32.931476 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-810958af-41bd-4315-9d22-94fd0c05cbba None admin] PATH_INFO: `/v3/domains/default` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:41:32.933107 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-810958af-41bd-4315-9d22-94fd0c05cbba None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 10:41:32.934534 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-810958af-41bd-4315-9d22-94fd0c05cbba None admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 10:41:32.936126 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 170/324] 10.222.0.25 () {56 vars in 1108 bytes} [Sun Aug 7 10:41:32 2022] GET /identity/v3/domains/default => generated 199 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 07 10:41:32.942001 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-810958af-41bd-4315-9d22-94fd0c05cbba None admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 10:41:32.985582 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=100 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 10:41:33.000615 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-290ec849-4c0a-457e-ba4d-b7ceabbce73a None admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 10:41:33.034791 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d189745f-3e07-4dde-aa6d-b1d0989e8948 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 10:41:33.035454 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d189745f-3e07-4dde-aa6d-b1d0989e8948 None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:41:33.035738 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d189745f-3e07-4dde-aa6d-b1d0989e8948 None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:41:33.036013 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d189745f-3e07-4dde-aa6d-b1d0989e8948 None admin] PATH_INFO: `/v3/roles` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:41:33.036714 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d189745f-3e07-4dde-aa6d-b1d0989e8948 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80143) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 10:41:33.037130 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d189745f-3e07-4dde-aa6d-b1d0989e8948 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 10:41:33.038526 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d189745f-3e07-4dde-aa6d-b1d0989e8948 None admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 10:41:33.048062 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 155/325] 10.222.0.25 () {56 vars in 1101 bytes} [Sun Aug 7 10:41:32 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 07 10:41:33.049605 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-810958af-41bd-4315-9d22-94fd0c05cbba None admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 10:41:33.053247 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d189745f-3e07-4dde-aa6d-b1d0989e8948 None admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 10:41:33.088729 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa7199bb-878b-4a33-94b4-ab02a5cbf514 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 10:41:33.089876 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa7199bb-878b-4a33-94b4-ab02a5cbf514 None admin] REQUEST_METHOD: `POST` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:41:33.090161 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa7199bb-878b-4a33-94b4-ab02a5cbf514 None admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:41:33.090436 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa7199bb-878b-4a33-94b4-ab02a5cbf514 None admin] PATH_INFO: `/v3/groups` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:41:33.091099 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa7199bb-878b-4a33-94b4-ab02a5cbf514 None admin] RBAC: Authorizing `identity:create_group()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 10:41:33.092529 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa7199bb-878b-4a33-94b4-ab02a5cbf514 None admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 10:41:33.108844 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 171/326] 10.222.0.25 () {60 vars in 1134 bytes} [Sun Aug 7 10:41:32 2022] POST /identity/v3/groups => generated 226 bytes in 167 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 07 10:41:33.139090 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=100 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 10:41:33.175312 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d189745f-3e07-4dde-aa6d-b1d0989e8948 None admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 10:41:33.190134 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e08ee99-124a-4477-93a1-e40f6f4fb1c3 None None] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:41:33.190376 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e08ee99-124a-4477-93a1-e40f6f4fb1c3 None None] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:41:33.190607 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e08ee99-124a-4477-93a1-e40f6f4fb1c3 None None] PATH_INFO: `/` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:41:33.192014 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 172/327] 10.222.0.25 () {54 vars in 927 bytes} [Sun Aug 7 10:41:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 07 10:41:33.197547 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-777aee5d-77c5-4215-8903-a8a93a737c41 None None] REQUEST_METHOD: `POST` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:41:33.197850 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-777aee5d-77c5-4215-8903-a8a93a737c41 None None] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:41:33.198103 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-777aee5d-77c5-4215-8903-a8a93a737c41 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:41:33.216517 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-750c7e53-e33f-4f30-94c8-0c72c6786dd2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 10:41:33.217476 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-750c7e53-e33f-4f30-94c8-0c72c6786dd2 None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:41:33.217848 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.auth.core [None req-777aee5d-77c5-4215-8903-a8a93a737c41 None None] MFA Rules not processed for user `bac4e6b76fe64217ac8fa44e71cffe6a`. Rule list: `[]` (Enabled: `True`). {{(pid=80144) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 10:41:33.218190 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-750c7e53-e33f-4f30-94c8-0c72c6786dd2 None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:41:33.218632 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-750c7e53-e33f-4f30-94c8-0c72c6786dd2 None admin] PATH_INFO: `/v3/users/affbb5676bbf4e7d901f9afdb707c399` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:41:33.219087 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.fernet_utils [None req-777aee5d-77c5-4215-8903-a8a93a737c41 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80144) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 10:41:33.220285 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-750c7e53-e33f-4f30-94c8-0c72c6786dd2 None admin] RBAC: Authorizing `identity:get_user(user_id=affbb5676bbf4e7d901f9afdb707c399)` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 10:41:33.221945 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-750c7e53-e33f-4f30-94c8-0c72c6786dd2 None admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 10:41:33.224339 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 156/328] 10.222.0.25 () {56 vars in 1177 bytes} [Sun Aug 7 10:41:33 2022] GET /identity/v3/users/affbb5676bbf4e7d901f9afdb707c399 => generated 293 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 10:41:33.230084 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-750c7e53-e33f-4f30-94c8-0c72c6786dd2 None admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 10:41:33.298092 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 173/329] 10.222.0.25 () {58 vars in 1024 bytes} [Sun Aug 7 10:41:33 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 07 10:41:33.321398 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-777aee5d-77c5-4215-8903-a8a93a737c41 None None] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 10:41:33.353595 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-750c7e53-e33f-4f30-94c8-0c72c6786dd2 None admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 10:41:33.360696 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 10:41:33.395312 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5265dc78-b992-44a7-811f-7c7b2ba01bd5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 10:41:33.395977 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5265dc78-b992-44a7-811f-7c7b2ba01bd5 None admin] REQUEST_METHOD: `PUT` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:41:33.396253 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5265dc78-b992-44a7-811f-7c7b2ba01bd5 None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:41:33.396525 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5265dc78-b992-44a7-811f-7c7b2ba01bd5 None admin] PATH_INFO: `/v3/system/users/affbb5676bbf4e7d901f9afdb707c399/roles/567b9b8669874c9986d68642dc9ef845` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:41:33.397930 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5265dc78-b992-44a7-811f-7c7b2ba01bd5 None admin] RBAC: Authorizing `identity:create_system_grant_for_user(user_id=affbb5676bbf4e7d901f9afdb707c399, role_id=567b9b8669874c9986d68642dc9ef845)` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 10:41:33.399419 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5265dc78-b992-44a7-811f-7c7b2ba01bd5 None admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 10:41:33.415008 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 157/330] 10.222.0.25 () {58 vars in 1334 bytes} [Sun Aug 7 10:41:33 2022] PUT /identity/v3/system/users/affbb5676bbf4e7d901f9afdb707c399/roles/567b9b8669874c9986d68642dc9ef845 => generated 0 bytes in 185 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 07 10:41:33.461245 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-777aee5d-77c5-4215-8903-a8a93a737c41 None None] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 10:41:33.500258 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2d5bc34-dc23-45e0-b743-8d34540fd722 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 10:41:33.500922 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2d5bc34-dc23-45e0-b743-8d34540fd722 None admin] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:41:33.501206 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2d5bc34-dc23-45e0-b743-8d34540fd722 None admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:41:33.501483 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2d5bc34-dc23-45e0-b743-8d34540fd722 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:41:33.513870 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2d5bc34-dc23-45e0-b743-8d34540fd722 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 07 10:41:33.518865 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application [None req-a2d5bc34-dc23-45e0-b743-8d34540fd722 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 07 10:41:33.518865 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 07 10:41:33.518865 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 07 10:41:33.518865 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 07 10:41:33.518865 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 07 10:41:33.518865 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 07 10:41:33.518865 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 07 10:41:33.518865 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 07 10:41:33.518865 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 07 10:41:33.518865 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 07 10:41:33.518865 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 07 10:41:33.518865 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 07 10:41:33.518865 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 07 10:41:33.518865 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 07 10:41:33.518865 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 07 10:41:33.518865 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 07 10:41:33.519836 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 07 10:41:33.519836 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 07 10:41:33.519836 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 07 10:41:33.519836 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 07 10:41:33.519836 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 07 10:41:33.519836 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 07 10:41:33.519836 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 07 10:41:33.519836 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 07 10:41:33.519836 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 07 10:41:33.519836 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self.get_or_create( Aug 07 10:41:33.519836 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 07 10:41:33.519836 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application with Lock( Aug 07 10:41:33.519836 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 07 10:41:33.519836 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self._enter() Aug 07 10:41:33.519836 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 07 10:41:33.519836 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 07 10:41:33.520660 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 07 10:41:33.520660 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self.creator() Aug 07 10:41:33.520660 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 07 10:41:33.520660 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application created_value = creator( Aug 07 10:41:33.520660 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 07 10:41:33.520660 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 07 10:41:33.520660 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 07 10:41:33.520660 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 07 10:41:33.520660 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 07 10:41:33.520660 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 07 10:41:33.520660 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 07 10:41:33.520660 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application Aug 07 10:41:33.520660 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 174/331] 10.222.0.25 () {56 vars in 1099 bytes} [Sun Aug 7 10:41:33 2022] GET /identity/v3/roles/member => generated 84 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 07 10:41:33.526912 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5265dc78-b992-44a7-811f-7c7b2ba01bd5 None admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 10:41:33.530335 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 10:41:33.648993 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5265dc78-b992-44a7-811f-7c7b2ba01bd5 None admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 10:41:33.685050 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70523a34-6fde-46b0-806b-166eea98d68a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 10:41:33.685674 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70523a34-6fde-46b0-806b-166eea98d68a None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:41:33.685956 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70523a34-6fde-46b0-806b-166eea98d68a None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:41:33.686235 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70523a34-6fde-46b0-806b-166eea98d68a None admin] PATH_INFO: `/v3/roles` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:41:33.686910 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70523a34-6fde-46b0-806b-166eea98d68a None admin] RBAC: Adding query filter params (name=member) {{(pid=80143) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 10:41:33.687244 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70523a34-6fde-46b0-806b-166eea98d68a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 10:41:33.688613 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70523a34-6fde-46b0-806b-166eea98d68a None admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 10:41:33.698846 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 158/332] 10.222.0.25 () {56 vars in 1101 bytes} [Sun Aug 7 10:41:33 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 10:41:33.704436 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2d5bc34-dc23-45e0-b743-8d34540fd722 None admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 10:41:33.804910 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=100 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 10:41:33.817624 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2d5bc34-dc23-45e0-b743-8d34540fd722 None admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 10:41:33.859533 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85c6214b-d254-4c33-a79b-986f06196d3a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 10:41:33.860182 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85c6214b-d254-4c33-a79b-986f06196d3a None admin] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:41:33.860466 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85c6214b-d254-4c33-a79b-986f06196d3a None admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:41:33.860748 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85c6214b-d254-4c33-a79b-986f06196d3a None admin] PATH_INFO: `/v3/users/5a4d08b8bd68447fa7d2f23a95eee88b` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:41:33.862170 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85c6214b-d254-4c33-a79b-986f06196d3a None admin] RBAC: Authorizing `identity:get_user(user_id=5a4d08b8bd68447fa7d2f23a95eee88b)` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 10:41:33.863679 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85c6214b-d254-4c33-a79b-986f06196d3a None admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 10:41:33.865555 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 175/333] 10.222.0.25 () {56 vars in 1177 bytes} [Sun Aug 7 10:41:33 2022] GET /identity/v3/users/5a4d08b8bd68447fa7d2f23a95eee88b => generated 293 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 10:41:33.872189 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70523a34-6fde-46b0-806b-166eea98d68a None admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 10:41:33.942239 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=100 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 10:41:33.989071 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70523a34-6fde-46b0-806b-166eea98d68a None admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 10:41:34.026228 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-995495a9-6d9a-4963-8e34-7fe342b234d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 10:41:34.027408 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-995495a9-6d9a-4963-8e34-7fe342b234d9 None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:41:34.027694 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-995495a9-6d9a-4963-8e34-7fe342b234d9 None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:41:34.027973 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-995495a9-6d9a-4963-8e34-7fe342b234d9 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:41:34.028705 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-995495a9-6d9a-4963-8e34-7fe342b234d9 None admin] RBAC: Adding query filter params (role.id=b98dfb3b9fee4644af2887a3ac3fd2e6, user.id=5a4d08b8bd68447fa7d2f23a95eee88b, scope.system=all) {{(pid=80143) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 10:41:34.029017 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-995495a9-6d9a-4963-8e34-7fe342b234d9 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 10:41:34.030467 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-995495a9-6d9a-4963-8e34-7fe342b234d9 None admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 10:41:34.043082 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 159/334] 10.222.0.25 () {56 vars in 1308 bytes} [Sun Aug 7 10:41:33 2022] GET /identity/v3/role_assignments?user.id=5a4d08b8bd68447fa7d2f23a95eee88b&scope.system=all&role.id=b98dfb3b9fee4644af2887a3ac3fd2e6 => generated 510 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 10:41:35.437688 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0a31846-6b63-4dd8-bd13-5ac14be80546 None None] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:41:35.438180 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0a31846-6b63-4dd8-bd13-5ac14be80546 None None] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:41:35.438252 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0a31846-6b63-4dd8-bd13-5ac14be80546 None None] PATH_INFO: `/` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:41:35.439866 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 176/335] 10.222.0.25 () {54 vars in 927 bytes} [Sun Aug 7 10:41:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 07 10:41:35.445435 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee900027-d243-42c3-b779-c79633223e58 None None] REQUEST_METHOD: `POST` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:41:35.445940 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee900027-d243-42c3-b779-c79633223e58 None None] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:41:35.446093 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee900027-d243-42c3-b779-c79633223e58 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:41:35.475968 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.auth.core [None req-ee900027-d243-42c3-b779-c79633223e58 None None] MFA Rules not processed for user `bac4e6b76fe64217ac8fa44e71cffe6a`. Rule list: `[]` (Enabled: `True`). {{(pid=80143) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 10:41:35.477178 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-ee900027-d243-42c3-b779-c79633223e58 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 10:41:35.566608 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 160/336] 10.222.0.25 () {58 vars in 1024 bytes} [Sun Aug 7 10:41:35 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 07 10:41:35.608379 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0a31846-6b63-4dd8-bd13-5ac14be80546 None None] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 10:41:35.750480 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0a31846-6b63-4dd8-bd13-5ac14be80546 None None] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 10:41:35.797558 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5aa35600-a0cb-44ef-be52-7def4d58c7dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 10:41:35.798207 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aa35600-a0cb-44ef-be52-7def4d58c7dd None admin] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:41:35.798487 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aa35600-a0cb-44ef-be52-7def4d58c7dd None admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:41:35.798761 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aa35600-a0cb-44ef-be52-7def4d58c7dd None admin] PATH_INFO: `/v3/roles/member` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:41:35.819071 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5aa35600-a0cb-44ef-be52-7def4d58c7dd None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 07 10:41:35.825223 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application [None req-5aa35600-a0cb-44ef-be52-7def4d58c7dd None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 07 10:41:35.825223 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 07 10:41:35.825223 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 07 10:41:35.825223 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 07 10:41:35.825223 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 07 10:41:35.825223 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 07 10:41:35.825223 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 07 10:41:35.825223 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 07 10:41:35.825223 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 07 10:41:35.825223 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 07 10:41:35.825223 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 07 10:41:35.825223 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 07 10:41:35.825223 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 07 10:41:35.825223 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 07 10:41:35.825223 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 07 10:41:35.825223 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 07 10:41:35.826117 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 07 10:41:35.826117 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 07 10:41:35.826117 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 07 10:41:35.826117 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 07 10:41:35.826117 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 07 10:41:35.826117 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 07 10:41:35.826117 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 07 10:41:35.826117 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 07 10:41:35.826117 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 07 10:41:35.826117 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self.get_or_create( Aug 07 10:41:35.826117 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 07 10:41:35.826117 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application with Lock( Aug 07 10:41:35.826117 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 07 10:41:35.826117 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self._enter() Aug 07 10:41:35.826117 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 07 10:41:35.826117 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 07 10:41:35.826900 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 07 10:41:35.826900 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self.creator() Aug 07 10:41:35.826900 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 07 10:41:35.826900 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application created_value = creator( Aug 07 10:41:35.826900 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 07 10:41:35.826900 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 07 10:41:35.826900 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 07 10:41:35.826900 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 07 10:41:35.826900 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 07 10:41:35.826900 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 07 10:41:35.826900 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 07 10:41:35.826900 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application Aug 07 10:41:35.826900 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 177/337] 10.222.0.25 () {56 vars in 1099 bytes} [Sun Aug 7 10:41:35 2022] GET /identity/v3/roles/member => generated 84 bytes in 219 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 07 10:41:35.833151 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee900027-d243-42c3-b779-c79633223e58 None None] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 10:41:35.898144 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=100 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 10:41:35.950793 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44181b49-1946-4454-abcd-e430260ef08b None None] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:41:35.951188 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44181b49-1946-4454-abcd-e430260ef08b None None] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:41:35.951404 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44181b49-1946-4454-abcd-e430260ef08b None None] PATH_INFO: `/` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:41:35.952845 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 178/338] 10.222.0.25 () {54 vars in 927 bytes} [Sun Aug 7 10:41:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 07 10:41:35.960464 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93eb3806-56d5-4734-a599-11cc4b2e050e None None] REQUEST_METHOD: `POST` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:41:35.960824 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93eb3806-56d5-4734-a599-11cc4b2e050e None None] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:41:35.961100 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93eb3806-56d5-4734-a599-11cc4b2e050e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:41:35.988339 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee900027-d243-42c3-b779-c79633223e58 None None] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 10:41:35.990720 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.auth.core [None req-93eb3806-56d5-4734-a599-11cc4b2e050e None None] MFA Rules not processed for user `bac4e6b76fe64217ac8fa44e71cffe6a`. Rule list: `[]` (Enabled: `True`). {{(pid=80144) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 10:41:35.991947 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.fernet_utils [None req-93eb3806-56d5-4734-a599-11cc4b2e050e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80144) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 10:41:36.019705 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=100 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 10:41:36.052085 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c19163c-c48b-4701-8cf3-71aa4d31ba19 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 10:41:36.052702 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c19163c-c48b-4701-8cf3-71aa4d31ba19 None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:41:36.052981 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c19163c-c48b-4701-8cf3-71aa4d31ba19 None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:41:36.053261 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c19163c-c48b-4701-8cf3-71aa4d31ba19 None admin] PATH_INFO: `/v3/roles` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:41:36.053942 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c19163c-c48b-4701-8cf3-71aa4d31ba19 None admin] RBAC: Adding query filter params (name=member) {{(pid=80143) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 10:41:36.054245 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c19163c-c48b-4701-8cf3-71aa4d31ba19 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 10:41:36.055640 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c19163c-c48b-4701-8cf3-71aa4d31ba19 None admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 10:41:36.078656 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 161/339] 10.222.0.25 () {56 vars in 1101 bytes} [Sun Aug 7 10:41:35 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 241 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 10:41:36.081803 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c19163c-c48b-4701-8cf3-71aa4d31ba19 None admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 10:41:36.130798 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 179/340] 10.222.0.25 () {58 vars in 1024 bytes} [Sun Aug 7 10:41:35 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 07 10:41:36.211806 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93eb3806-56d5-4734-a599-11cc4b2e050e None None] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 10:41:36.226970 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c19163c-c48b-4701-8cf3-71aa4d31ba19 None admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 10:41:36.244042 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=100 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 10:41:36.296150 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e356a81f-bc1f-4ef0-b4ad-5a63da32625f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 10:41:36.296701 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e356a81f-bc1f-4ef0-b4ad-5a63da32625f None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:41:36.296983 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e356a81f-bc1f-4ef0-b4ad-5a63da32625f None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:41:36.297251 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e356a81f-bc1f-4ef0-b4ad-5a63da32625f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:41:36.419246 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e356a81f-bc1f-4ef0-b4ad-5a63da32625f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 10:41:36.422345 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e356a81f-bc1f-4ef0-b4ad-5a63da32625f None admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 10:41:36.438579 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93eb3806-56d5-4734-a599-11cc4b2e050e None None] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 10:41:36.508346 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4081dfcb-1797-4f0f-9ec8-c1ae021bede7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 10:41:36.508999 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4081dfcb-1797-4f0f-9ec8-c1ae021bede7 None admin] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:41:36.509418 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4081dfcb-1797-4f0f-9ec8-c1ae021bede7 None admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:41:36.509702 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4081dfcb-1797-4f0f-9ec8-c1ae021bede7 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:41:36.535617 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4081dfcb-1797-4f0f-9ec8-c1ae021bede7 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 07 10:41:36.540727 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application [None req-4081dfcb-1797-4f0f-9ec8-c1ae021bede7 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Aug 07 10:41:36.540727 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 07 10:41:36.540727 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 07 10:41:36.540727 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 07 10:41:36.540727 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 07 10:41:36.540727 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 07 10:41:36.540727 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 07 10:41:36.540727 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 07 10:41:36.540727 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 07 10:41:36.540727 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 07 10:41:36.540727 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 07 10:41:36.540727 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 07 10:41:36.540727 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 07 10:41:36.540727 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 07 10:41:36.540727 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 07 10:41:36.540727 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 07 10:41:36.541597 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 07 10:41:36.541597 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 07 10:41:36.541597 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 07 10:41:36.541597 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 07 10:41:36.541597 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 07 10:41:36.541597 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 07 10:41:36.541597 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 07 10:41:36.541597 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 07 10:41:36.541597 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 07 10:41:36.541597 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self.get_or_create( Aug 07 10:41:36.541597 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 07 10:41:36.541597 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application with Lock( Aug 07 10:41:36.541597 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 07 10:41:36.541597 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self._enter() Aug 07 10:41:36.541597 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 07 10:41:36.541597 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 07 10:41:36.542455 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 07 10:41:36.542455 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self.creator() Aug 07 10:41:36.542455 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 07 10:41:36.542455 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application created_value = creator( Aug 07 10:41:36.542455 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 07 10:41:36.542455 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 07 10:41:36.542455 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 07 10:41:36.542455 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 07 10:41:36.542455 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 07 10:41:36.542455 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 07 10:41:36.542455 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Aug 07 10:41:36.542455 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application Aug 07 10:41:36.547433 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 180/341] 10.222.0.25 () {56 vars in 1099 bytes} [Sun Aug 7 10:41:36 2022] GET /identity/v3/roles/reader => generated 84 bytes in 336 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 07 10:41:36.549435 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2055fdc2-91f6-4916-9a05-182173dec10a None None] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:41:36.549724 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2055fdc2-91f6-4916-9a05-182173dec10a None None] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:41:36.550007 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2055fdc2-91f6-4916-9a05-182173dec10a None None] PATH_INFO: `/` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:41:36.551646 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 181/342] 10.222.0.25 () {54 vars in 927 bytes} [Sun Aug 7 10:41:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 07 10:41:36.560460 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c4e1068-ca29-4618-933f-0f3ad0110153 None None] REQUEST_METHOD: `POST` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:41:36.560789 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c4e1068-ca29-4618-933f-0f3ad0110153 None None] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:41:36.561259 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c4e1068-ca29-4618-933f-0f3ad0110153 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:41:36.595177 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 162/343] 10.222.0.25 () {58 vars in 1282 bytes} [Sun Aug 7 10:41:36 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 510 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 07 10:41:36.596148 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e356a81f-bc1f-4ef0-b4ad-5a63da32625f None admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 10:41:36.606945 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.auth.core [None req-6c4e1068-ca29-4618-933f-0f3ad0110153 None None] MFA Rules not processed for user `bac4e6b76fe64217ac8fa44e71cffe6a`. Rule list: `[]` (Enabled: `True`). {{(pid=80144) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 10:41:36.608175 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.fernet_utils [None req-6c4e1068-ca29-4618-933f-0f3ad0110153 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80144) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 10:41:36.611251 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=100 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 10:41:36.615714 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=100 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 10:41:36.713087 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e356a81f-bc1f-4ef0-b4ad-5a63da32625f None admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 10:41:36.741261 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 182/344] 10.222.0.25 () {58 vars in 1024 bytes} [Sun Aug 7 10:41:36 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 182 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 07 10:41:36.742127 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c4e1068-ca29-4618-933f-0f3ad0110153 None None] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 10:41:36.754572 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-768e0833-788d-4abb-98d2-157026e3b2f6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 10:41:36.755222 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-768e0833-788d-4abb-98d2-157026e3b2f6 None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:41:36.755500 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-768e0833-788d-4abb-98d2-157026e3b2f6 None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:41:36.755767 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-768e0833-788d-4abb-98d2-157026e3b2f6 None admin] PATH_INFO: `/v3/roles` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:41:36.756432 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-768e0833-788d-4abb-98d2-157026e3b2f6 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80143) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 10:41:36.756732 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-768e0833-788d-4abb-98d2-157026e3b2f6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 10:41:36.758086 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-768e0833-788d-4abb-98d2-157026e3b2f6 None admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 10:41:36.767384 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 163/345] 10.222.0.25 () {56 vars in 1101 bytes} [Sun Aug 7 10:41:36 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 10:41:36.768682 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a6694a8-02b6-4923-ac9c-e83ca0dbf6c2 None None] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:41:36.768968 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a6694a8-02b6-4923-ac9c-e83ca0dbf6c2 None None] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:41:36.769242 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a6694a8-02b6-4923-ac9c-e83ca0dbf6c2 None None] PATH_INFO: `/` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:41:36.771486 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 164/346] 10.222.0.25 () {54 vars in 927 bytes} [Sun Aug 7 10:41:36 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 07 10:41:36.774684 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a6694a8-02b6-4923-ac9c-e83ca0dbf6c2 None None] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 10:41:36.865485 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=100 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 10:41:36.872758 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c4e1068-ca29-4618-933f-0f3ad0110153 None None] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 10:41:36.895667 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a6694a8-02b6-4923-ac9c-e83ca0dbf6c2 None None] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 10:41:36.929372 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-715e4b0c-5aae-48c3-b6ac-549ea0f3a9a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 10:41:36.930987 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-715e4b0c-5aae-48c3-b6ac-549ea0f3a9a2 None admin] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:41:36.931482 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-715e4b0c-5aae-48c3-b6ac-549ea0f3a9a2 None admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:41:36.932160 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-715e4b0c-5aae-48c3-b6ac-549ea0f3a9a2 None admin] PATH_INFO: `/v3/projects/90718391b09e493bb296767dc8a3c0ce` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:41:36.933875 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-715e4b0c-5aae-48c3-b6ac-549ea0f3a9a2 None admin] RBAC: Authorizing `identity:get_project(project_id=90718391b09e493bb296767dc8a3c0ce)` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 10:41:36.936259 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-715e4b0c-5aae-48c3-b6ac-549ea0f3a9a2 None admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 10:41:36.942570 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f19840b-70f6-43bc-9f2a-d61169be020a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 10:41:36.942570 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f19840b-70f6-43bc-9f2a-d61169be020a None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:41:36.942570 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f19840b-70f6-43bc-9f2a-d61169be020a None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:41:36.942570 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f19840b-70f6-43bc-9f2a-d61169be020a None admin] PATH_INFO: `/v3/users/affbb5676bbf4e7d901f9afdb707c399` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:41:36.942902 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f19840b-70f6-43bc-9f2a-d61169be020a None admin] RBAC: Authorizing `identity:get_user(user_id=affbb5676bbf4e7d901f9afdb707c399)` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 10:41:36.943125 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 183/347] 10.222.0.25 () {56 vars in 1186 bytes} [Sun Aug 7 10:41:36 2022] GET /identity/v3/projects/90718391b09e493bb296767dc8a3c0ce => generated 298 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 10:41:36.943904 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f19840b-70f6-43bc-9f2a-d61169be020a None admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 10:41:36.944599 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce9688a6-fb9c-4151-8cf2-25cc28a5073d None None] REQUEST_METHOD: `POST` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:41:36.945031 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce9688a6-fb9c-4151-8cf2-25cc28a5073d None None] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:41:36.945440 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce9688a6-fb9c-4151-8cf2-25cc28a5073d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:41:36.945908 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 165/348] 10.222.0.25 () {56 vars in 1177 bytes} [Sun Aug 7 10:41:36 2022] GET /identity/v3/users/affbb5676bbf4e7d901f9afdb707c399 => generated 293 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 07 10:41:36.946800 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f19840b-70f6-43bc-9f2a-d61169be020a None admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 10:41:36.962482 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=100 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 10:41:36.970294 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.auth.core [None req-ce9688a6-fb9c-4151-8cf2-25cc28a5073d None None] MFA Rules not processed for user `bac4e6b76fe64217ac8fa44e71cffe6a`. Rule list: `[]` (Enabled: `True`). {{(pid=80144) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 10:41:36.972572 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.fernet_utils [None req-ce9688a6-fb9c-4151-8cf2-25cc28a5073d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80144) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 10:41:37.066992 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 184/349] 10.222.0.25 () {58 vars in 1024 bytes} [Sun Aug 7 10:41:36 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 07 10:41:37.068217 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce9688a6-fb9c-4151-8cf2-25cc28a5073d None None] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 10:41:37.077648 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f19840b-70f6-43bc-9f2a-d61169be020a None admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 10:41:37.116712 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6caa5197-026d-4219-985f-960d9246abbd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 10:41:37.117286 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6caa5197-026d-4219-985f-960d9246abbd None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:41:37.117569 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6caa5197-026d-4219-985f-960d9246abbd None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:41:37.117845 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6caa5197-026d-4219-985f-960d9246abbd None admin] PATH_INFO: `/v3/roles/member` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:41:37.129853 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6caa5197-026d-4219-985f-960d9246abbd None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 07 10:41:37.137673 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application [None req-6caa5197-026d-4219-985f-960d9246abbd None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 07 10:41:37.137673 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 07 10:41:37.137673 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 07 10:41:37.137673 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 07 10:41:37.137673 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 07 10:41:37.137673 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 07 10:41:37.137673 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 07 10:41:37.137673 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 07 10:41:37.137673 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 07 10:41:37.137673 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 07 10:41:37.137673 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 07 10:41:37.137673 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 07 10:41:37.137673 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 07 10:41:37.137673 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 07 10:41:37.137673 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 07 10:41:37.137673 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 07 10:41:37.139473 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 07 10:41:37.139473 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 07 10:41:37.139473 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 07 10:41:37.139473 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 07 10:41:37.139473 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 07 10:41:37.139473 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 07 10:41:37.139473 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 07 10:41:37.139473 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 07 10:41:37.139473 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 07 10:41:37.139473 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.get_or_create( Aug 07 10:41:37.139473 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 07 10:41:37.139473 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application with Lock( Aug 07 10:41:37.139473 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 07 10:41:37.139473 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self._enter() Aug 07 10:41:37.139473 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 07 10:41:37.139473 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 07 10:41:37.140260 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 07 10:41:37.140260 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.creator() Aug 07 10:41:37.140260 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 07 10:41:37.140260 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application created_value = creator( Aug 07 10:41:37.140260 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 07 10:41:37.140260 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 07 10:41:37.140260 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 07 10:41:37.140260 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 07 10:41:37.140260 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 07 10:41:37.140260 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 07 10:41:37.140260 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 07 10:41:37.140260 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application Aug 07 10:41:37.140260 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 166/350] 10.222.0.25 () {56 vars in 1099 bytes} [Sun Aug 7 10:41:36 2022] GET /identity/v3/roles/member => generated 84 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 07 10:41:37.140260 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6caa5197-026d-4219-985f-960d9246abbd None admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 10:41:37.142601 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=100 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 10:41:37.193392 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce9688a6-fb9c-4151-8cf2-25cc28a5073d None None] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 10:41:37.230810 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bef3d733-11c1-410b-a0de-863623874950 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 10:41:37.231853 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bef3d733-11c1-410b-a0de-863623874950 None admin] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:41:37.232341 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bef3d733-11c1-410b-a0de-863623874950 None admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:41:37.232806 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bef3d733-11c1-410b-a0de-863623874950 None admin] PATH_INFO: `/v3/groups/f26eb533e57a456d8f95265e204d21b5` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:41:37.237222 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=100 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 10:41:37.240078 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bef3d733-11c1-410b-a0de-863623874950 None admin] RBAC: Authorizing `identity:get_group(group_id=f26eb533e57a456d8f95265e204d21b5)` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 10:41:37.241727 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bef3d733-11c1-410b-a0de-863623874950 None admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 10:41:37.248328 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 185/351] 10.222.0.25 () {56 vars in 1180 bytes} [Sun Aug 7 10:41:37 2022] GET /identity/v3/groups/f26eb533e57a456d8f95265e204d21b5 => generated 226 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 10:41:37.250831 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bef3d733-11c1-410b-a0de-863623874950 None admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 10:41:37.260285 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6caa5197-026d-4219-985f-960d9246abbd None admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 10:41:37.299958 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52a4d7de-0ce1-404a-b2ab-4b4fedb19d37 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 10:41:37.300685 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52a4d7de-0ce1-404a-b2ab-4b4fedb19d37 None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:41:37.300974 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52a4d7de-0ce1-404a-b2ab-4b4fedb19d37 None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:41:37.301252 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52a4d7de-0ce1-404a-b2ab-4b4fedb19d37 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:41:37.302080 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52a4d7de-0ce1-404a-b2ab-4b4fedb19d37 None admin] RBAC: Adding query filter params (role.id=567b9b8669874c9986d68642dc9ef845, user.id=affbb5676bbf4e7d901f9afdb707c399, scope.system=all) {{(pid=80143) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 10:41:37.302400 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52a4d7de-0ce1-404a-b2ab-4b4fedb19d37 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 10:41:37.303975 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52a4d7de-0ce1-404a-b2ab-4b4fedb19d37 None admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 10:41:37.318251 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 167/352] 10.222.0.25 () {56 vars in 1308 bytes} [Sun Aug 7 10:41:37 2022] GET /identity/v3/role_assignments?user.id=affbb5676bbf4e7d901f9afdb707c399&scope.system=all&role.id=567b9b8669874c9986d68642dc9ef845 => generated 510 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 10:41:37.319288 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52a4d7de-0ce1-404a-b2ab-4b4fedb19d37 None admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 10:41:37.356759 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bef3d733-11c1-410b-a0de-863623874950 None admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 10:41:37.393340 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=100 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 10:41:37.395376 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a615fd5-1e52-47fa-b3e9-ef428110596c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 10:41:37.396029 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a615fd5-1e52-47fa-b3e9-ef428110596c None admin] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:41:37.396311 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a615fd5-1e52-47fa-b3e9-ef428110596c None admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:41:37.396578 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a615fd5-1e52-47fa-b3e9-ef428110596c None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:41:37.409281 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a615fd5-1e52-47fa-b3e9-ef428110596c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 07 10:41:37.414697 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application [None req-5a615fd5-1e52-47fa-b3e9-ef428110596c None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 07 10:41:37.414697 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 07 10:41:37.414697 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 07 10:41:37.414697 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 07 10:41:37.414697 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 07 10:41:37.414697 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 07 10:41:37.414697 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 07 10:41:37.414697 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 07 10:41:37.414697 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 07 10:41:37.414697 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 07 10:41:37.414697 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 07 10:41:37.414697 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 07 10:41:37.414697 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 07 10:41:37.414697 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 07 10:41:37.414697 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 07 10:41:37.414697 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 07 10:41:37.415669 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 07 10:41:37.415669 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 07 10:41:37.415669 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 07 10:41:37.415669 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 07 10:41:37.415669 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 07 10:41:37.415669 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 07 10:41:37.415669 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 07 10:41:37.415669 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 07 10:41:37.415669 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 07 10:41:37.415669 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self.get_or_create( Aug 07 10:41:37.415669 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 07 10:41:37.415669 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application with Lock( Aug 07 10:41:37.415669 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 07 10:41:37.415669 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self._enter() Aug 07 10:41:37.415669 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 07 10:41:37.415669 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 07 10:41:37.416449 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 07 10:41:37.416449 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self.creator() Aug 07 10:41:37.416449 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 07 10:41:37.416449 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application created_value = creator( Aug 07 10:41:37.416449 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 07 10:41:37.416449 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 07 10:41:37.416449 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 07 10:41:37.416449 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 07 10:41:37.416449 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 07 10:41:37.416449 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 07 10:41:37.416449 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 07 10:41:37.416449 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application Aug 07 10:41:37.423035 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 186/353] 10.222.0.25 () {56 vars in 1114 bytes} [Sun Aug 7 10:41:37 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 07 10:41:37.424017 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a615fd5-1e52-47fa-b3e9-ef428110596c None admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 10:41:37.443624 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52a4d7de-0ce1-404a-b2ab-4b4fedb19d37 None admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 10:41:37.488426 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-895b4d6a-f9fa-433f-8a20-4fc1d60f0945 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 10:41:37.489176 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-895b4d6a-f9fa-433f-8a20-4fc1d60f0945 None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:41:37.489462 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-895b4d6a-f9fa-433f-8a20-4fc1d60f0945 None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:41:37.489914 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=100 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 10:41:37.490728 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-895b4d6a-f9fa-433f-8a20-4fc1d60f0945 None admin] PATH_INFO: `/v3/roles` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:41:37.491692 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-895b4d6a-f9fa-433f-8a20-4fc1d60f0945 None admin] RBAC: Adding query filter params (name=member) {{(pid=80143) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 10:41:37.492174 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-895b4d6a-f9fa-433f-8a20-4fc1d60f0945 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 10:41:37.495554 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-895b4d6a-f9fa-433f-8a20-4fc1d60f0945 None admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 10:41:37.506468 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 168/354] 10.222.0.25 () {56 vars in 1101 bytes} [Sun Aug 7 10:41:37 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 10:41:37.507376 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-895b4d6a-f9fa-433f-8a20-4fc1d60f0945 None admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 10:41:37.561080 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a615fd5-1e52-47fa-b3e9-ef428110596c None admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 10:41:37.624257 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56a4131a-28e9-41c2-af8a-cb158ff8d825 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 10:41:37.624744 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-895b4d6a-f9fa-433f-8a20-4fc1d60f0945 None admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 10:41:37.625277 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56a4131a-28e9-41c2-af8a-cb158ff8d825 None admin] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:41:37.625277 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56a4131a-28e9-41c2-af8a-cb158ff8d825 None admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:41:37.625580 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56a4131a-28e9-41c2-af8a-cb158ff8d825 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:41:37.626119 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56a4131a-28e9-41c2-af8a-cb158ff8d825 None admin] RBAC: Adding query filter params (group.id=f26eb533e57a456d8f95265e204d21b5, role.id=b98dfb3b9fee4644af2887a3ac3fd2e6, scope.project.id=90718391b09e493bb296767dc8a3c0ce) {{(pid=80144) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 10:41:37.626427 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56a4131a-28e9-41c2-af8a-cb158ff8d825 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 10:41:37.627881 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56a4131a-28e9-41c2-af8a-cb158ff8d825 None admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 10:41:37.639072 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 187/355] 10.222.0.25 () {56 vars in 1376 bytes} [Sun Aug 7 10:41:37 2022] GET /identity/v3/role_assignments?group.id=f26eb533e57a456d8f95265e204d21b5&scope.project.id=90718391b09e493bb296767dc8a3c0ce&role.id=b98dfb3b9fee4644af2887a3ac3fd2e6 => generated 260 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 10:41:37.640063 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56a4131a-28e9-41c2-af8a-cb158ff8d825 None admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 10:41:37.659319 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=100 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 10:41:37.664716 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6afa5b8-7c95-4fac-873f-114329c6f7be None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 10:41:37.665454 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6afa5b8-7c95-4fac-873f-114329c6f7be None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:41:37.665749 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6afa5b8-7c95-4fac-873f-114329c6f7be None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:41:37.666031 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6afa5b8-7c95-4fac-873f-114329c6f7be None admin] PATH_INFO: `/v3/roles` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:41:37.667261 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6afa5b8-7c95-4fac-873f-114329c6f7be None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80143) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 10:41:37.667581 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6afa5b8-7c95-4fac-873f-114329c6f7be None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 10:41:37.669091 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6afa5b8-7c95-4fac-873f-114329c6f7be None admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 10:41:37.679743 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 169/356] 10.222.0.25 () {56 vars in 1111 bytes} [Sun Aug 7 10:41:37 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 10:41:37.692256 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6afa5b8-7c95-4fac-873f-114329c6f7be None admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 10:41:37.817812 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6afa5b8-7c95-4fac-873f-114329c6f7be None admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 10:41:37.842836 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56a4131a-28e9-41c2-af8a-cb158ff8d825 None admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 10:41:37.863460 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a14daa2-e636-428a-a5c3-d26497b9960f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 10:41:37.864096 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a14daa2-e636-428a-a5c3-d26497b9960f None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:41:37.864379 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a14daa2-e636-428a-a5c3-d26497b9960f None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:41:37.864659 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a14daa2-e636-428a-a5c3-d26497b9960f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:41:37.884335 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=100 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 10:41:37.905782 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1608127c-096d-45ae-8b66-012bb790b1ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 10:41:37.905782 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1608127c-096d-45ae-8b66-012bb790b1ee None admin] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:41:37.905782 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1608127c-096d-45ae-8b66-012bb790b1ee None admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:41:37.906110 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1608127c-096d-45ae-8b66-012bb790b1ee None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:41:37.939420 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=100 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 10:41:37.959755 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a14daa2-e636-428a-a5c3-d26497b9960f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 10:41:37.961315 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a14daa2-e636-428a-a5c3-d26497b9960f None admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 10:41:37.996963 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1608127c-096d-45ae-8b66-012bb790b1ee None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 10:41:37.998618 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1608127c-096d-45ae-8b66-012bb790b1ee None admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 10:41:38.074197 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 170/357] 10.222.0.25 () {58 vars in 1282 bytes} [Sun Aug 7 10:41:37 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 383 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 07 10:41:38.076316 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bd65a35-d7d0-43ea-8a87-bc47b99c9c0d None None] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:41:38.076607 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bd65a35-d7d0-43ea-8a87-bc47b99c9c0d None None] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:41:38.076888 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bd65a35-d7d0-43ea-8a87-bc47b99c9c0d None None] PATH_INFO: `/` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:41:38.078427 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 171/358] 10.222.0.25 () {54 vars in 927 bytes} [Sun Aug 7 10:41:38 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 07 10:41:38.080311 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd216c8a-7fac-4fbd-a22d-1dd9442b1887 None None] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:41:38.080311 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd216c8a-7fac-4fbd-a22d-1dd9442b1887 None None] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:41:38.080502 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd216c8a-7fac-4fbd-a22d-1dd9442b1887 None None] PATH_INFO: `/` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:41:38.082129 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 172/359] 10.222.0.25 () {54 vars in 927 bytes} [Sun Aug 7 10:41:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 07 10:41:38.082904 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd216c8a-7fac-4fbd-a22d-1dd9442b1887 None None] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 10:41:38.135527 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 188/360] 10.222.0.25 () {58 vars in 1282 bytes} [Sun Aug 7 10:41:37 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 496 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 07 10:41:38.137280 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81586f30-35f1-4203-9e9b-427bc336f453 None None] REQUEST_METHOD: `POST` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:41:38.137781 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81586f30-35f1-4203-9e9b-427bc336f453 None None] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:41:38.138259 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81586f30-35f1-4203-9e9b-427bc336f453 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:41:38.160383 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.auth.core [None req-81586f30-35f1-4203-9e9b-427bc336f453 None None] MFA Rules not processed for user `bac4e6b76fe64217ac8fa44e71cffe6a`. Rule list: `[]` (Enabled: `True`). {{(pid=80144) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 10:41:38.161562 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.fernet_utils [None req-81586f30-35f1-4203-9e9b-427bc336f453 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80144) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 10:41:38.182465 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=100 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 10:41:38.202564 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=100 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 10:41:38.222843 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd216c8a-7fac-4fbd-a22d-1dd9442b1887 None None] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 10:41:38.259734 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 189/361] 10.222.0.25 () {58 vars in 1024 bytes} [Sun Aug 7 10:41:38 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 07 10:41:38.262529 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5024567a-d1cc-4e08-8893-e9b41fc022c3 None None] REQUEST_METHOD: `POST` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:41:38.263192 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5024567a-d1cc-4e08-8893-e9b41fc022c3 None None] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:41:38.263803 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5024567a-d1cc-4e08-8893-e9b41fc022c3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:41:38.267590 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-256107da-6b1d-4da3-99fe-97112ca68e77 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 10:41:38.268288 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-256107da-6b1d-4da3-99fe-97112ca68e77 None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:41:38.268568 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-256107da-6b1d-4da3-99fe-97112ca68e77 None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:41:38.268841 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-256107da-6b1d-4da3-99fe-97112ca68e77 None admin] PATH_INFO: `/v3/projects/90718391b09e493bb296767dc8a3c0ce` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:41:38.270068 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-256107da-6b1d-4da3-99fe-97112ca68e77 None admin] RBAC: Authorizing `identity:get_project(project_id=90718391b09e493bb296767dc8a3c0ce)` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 10:41:38.271667 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-256107da-6b1d-4da3-99fe-97112ca68e77 None admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 10:41:38.273856 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 173/362] 10.222.0.25 () {56 vars in 1186 bytes} [Sun Aug 7 10:41:38 2022] GET /identity/v3/projects/90718391b09e493bb296767dc8a3c0ce => generated 298 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 10:41:38.274622 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-256107da-6b1d-4da3-99fe-97112ca68e77 None admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 10:41:38.294068 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.auth.core [None req-5024567a-d1cc-4e08-8893-e9b41fc022c3 None None] MFA Rules not processed for user `bac4e6b76fe64217ac8fa44e71cffe6a`. Rule list: `[]` (Enabled: `True`). {{(pid=80144) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 10:41:38.295295 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.fernet_utils [None req-5024567a-d1cc-4e08-8893-e9b41fc022c3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80144) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 10:41:38.388454 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 190/363] 10.222.0.25 () {58 vars in 1024 bytes} [Sun Aug 7 10:41:38 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 07 10:41:38.389499 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5024567a-d1cc-4e08-8893-e9b41fc022c3 None None] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 10:41:38.398881 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-256107da-6b1d-4da3-99fe-97112ca68e77 None admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 10:41:38.446580 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c065bda5-3227-4aad-86fa-87ebc6698752 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 10:41:38.447607 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c065bda5-3227-4aad-86fa-87ebc6698752 None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:41:38.448722 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c065bda5-3227-4aad-86fa-87ebc6698752 None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:41:38.449063 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c065bda5-3227-4aad-86fa-87ebc6698752 None admin] PATH_INFO: `/v3/projects/d88586f68b734382abb9836b290aeb54` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:41:38.450512 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c065bda5-3227-4aad-86fa-87ebc6698752 None admin] RBAC: Authorizing `identity:get_project(project_id=d88586f68b734382abb9836b290aeb54)` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 10:41:38.452079 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c065bda5-3227-4aad-86fa-87ebc6698752 None admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 10:41:38.454724 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 174/364] 10.222.0.25 () {56 vars in 1186 bytes} [Sun Aug 7 10:41:38 2022] GET /identity/v3/projects/d88586f68b734382abb9836b290aeb54 => generated 302 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 10:41:38.459532 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c065bda5-3227-4aad-86fa-87ebc6698752 None admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 10:41:38.492137 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=100 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 10:41:38.499642 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=100 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 10:41:38.512013 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5024567a-d1cc-4e08-8893-e9b41fc022c3 None None] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 10:41:38.560972 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ce52711-8c73-4212-a2dd-cf835ac513a4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 10:41:38.561912 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ce52711-8c73-4212-a2dd-cf835ac513a4 None admin] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:41:38.562190 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ce52711-8c73-4212-a2dd-cf835ac513a4 None admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:41:38.562461 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ce52711-8c73-4212-a2dd-cf835ac513a4 None admin] PATH_INFO: `/v3/groups/f26eb533e57a456d8f95265e204d21b5` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:41:38.564225 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ce52711-8c73-4212-a2dd-cf835ac513a4 None admin] RBAC: Authorizing `identity:get_group(group_id=f26eb533e57a456d8f95265e204d21b5)` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 10:41:38.565627 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ce52711-8c73-4212-a2dd-cf835ac513a4 None admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 10:41:38.571038 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 191/365] 10.222.0.25 () {56 vars in 1180 bytes} [Sun Aug 7 10:41:38 2022] GET /identity/v3/groups/f26eb533e57a456d8f95265e204d21b5 => generated 226 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 10:41:38.572665 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ce52711-8c73-4212-a2dd-cf835ac513a4 None admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 10:41:38.601469 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c065bda5-3227-4aad-86fa-87ebc6698752 None admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 10:41:38.644423 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a489b4ef-e222-489b-8d08-9480ff7129a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 10:41:38.645071 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a489b4ef-e222-489b-8d08-9480ff7129a1 None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:41:38.645321 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a489b4ef-e222-489b-8d08-9480ff7129a1 None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:41:38.645586 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a489b4ef-e222-489b-8d08-9480ff7129a1 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:41:38.656890 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a489b4ef-e222-489b-8d08-9480ff7129a1 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 07 10:41:38.662015 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application [None req-a489b4ef-e222-489b-8d08-9480ff7129a1 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 07 10:41:38.662015 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 07 10:41:38.662015 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 07 10:41:38.662015 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 07 10:41:38.662015 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 07 10:41:38.662015 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 07 10:41:38.662015 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 07 10:41:38.662015 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 07 10:41:38.662015 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 07 10:41:38.662015 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 07 10:41:38.662015 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 07 10:41:38.662015 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 07 10:41:38.662015 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 07 10:41:38.662015 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 07 10:41:38.662015 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 07 10:41:38.662015 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 07 10:41:38.662890 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 07 10:41:38.662890 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 07 10:41:38.662890 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 07 10:41:38.662890 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 07 10:41:38.662890 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 07 10:41:38.662890 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 07 10:41:38.662890 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 07 10:41:38.662890 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 07 10:41:38.662890 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 07 10:41:38.662890 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.get_or_create( Aug 07 10:41:38.662890 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 07 10:41:38.662890 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application with Lock( Aug 07 10:41:38.662890 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 07 10:41:38.662890 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self._enter() Aug 07 10:41:38.662890 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 07 10:41:38.662890 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 07 10:41:38.663733 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 07 10:41:38.663733 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.creator() Aug 07 10:41:38.663733 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 07 10:41:38.663733 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application created_value = creator( Aug 07 10:41:38.663733 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 07 10:41:38.663733 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 07 10:41:38.663733 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 07 10:41:38.663733 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 07 10:41:38.663733 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 07 10:41:38.663733 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 07 10:41:38.663733 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 07 10:41:38.663733 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application Aug 07 10:41:38.663733 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 175/366] 10.222.0.25 () {56 vars in 1114 bytes} [Sun Aug 7 10:41:38 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 208 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 07 10:41:38.664392 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a489b4ef-e222-489b-8d08-9480ff7129a1 None admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 10:41:38.703760 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ce52711-8c73-4212-a2dd-cf835ac513a4 None admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 10:41:38.750220 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-484834dd-0dc9-4ff3-8199-5015b1759719 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 10:41:38.750790 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-484834dd-0dc9-4ff3-8199-5015b1759719 None admin] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:41:38.751121 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-484834dd-0dc9-4ff3-8199-5015b1759719 None admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:41:38.751400 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-484834dd-0dc9-4ff3-8199-5015b1759719 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:41:38.767389 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=100 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 10:41:38.772334 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-484834dd-0dc9-4ff3-8199-5015b1759719 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 07 10:41:38.779181 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application [None req-484834dd-0dc9-4ff3-8199-5015b1759719 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 07 10:41:38.779181 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 07 10:41:38.779181 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 07 10:41:38.779181 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 07 10:41:38.779181 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 07 10:41:38.779181 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 07 10:41:38.779181 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 07 10:41:38.779181 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 07 10:41:38.779181 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 07 10:41:38.779181 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 07 10:41:38.779181 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 07 10:41:38.779181 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 07 10:41:38.779181 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 07 10:41:38.779181 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 07 10:41:38.779181 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 07 10:41:38.779181 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 07 10:41:38.780065 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 07 10:41:38.780065 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 07 10:41:38.780065 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 07 10:41:38.780065 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 07 10:41:38.780065 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 07 10:41:38.780065 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 07 10:41:38.780065 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 07 10:41:38.780065 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 07 10:41:38.780065 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 07 10:41:38.780065 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self.get_or_create( Aug 07 10:41:38.780065 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 07 10:41:38.780065 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application with Lock( Aug 07 10:41:38.780065 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 07 10:41:38.780065 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self._enter() Aug 07 10:41:38.780065 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 07 10:41:38.780065 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 07 10:41:38.781237 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 07 10:41:38.781237 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self.creator() Aug 07 10:41:38.781237 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 07 10:41:38.781237 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application created_value = creator( Aug 07 10:41:38.781237 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 07 10:41:38.781237 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 07 10:41:38.781237 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 07 10:41:38.781237 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 07 10:41:38.781237 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 07 10:41:38.781237 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 07 10:41:38.781237 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 07 10:41:38.781237 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application Aug 07 10:41:38.781237 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 192/367] 10.222.0.25 () {56 vars in 1096 bytes} [Sun Aug 7 10:41:38 2022] GET /identity/v3/roles/admin => generated 83 bytes in 209 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 07 10:41:38.781237 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-484834dd-0dc9-4ff3-8199-5015b1759719 None admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 10:41:38.797379 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a489b4ef-e222-489b-8d08-9480ff7129a1 None admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 10:41:38.801377 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=100 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 10:41:38.836262 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f48701ec-eac9-436b-9531-3e78aa54df3e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 10:41:38.836902 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f48701ec-eac9-436b-9531-3e78aa54df3e None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:41:38.837182 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f48701ec-eac9-436b-9531-3e78aa54df3e None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:41:38.837455 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f48701ec-eac9-436b-9531-3e78aa54df3e None admin] PATH_INFO: `/v3/groups/f26eb533e57a456d8f95265e204d21b5` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:41:38.839629 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f48701ec-eac9-436b-9531-3e78aa54df3e None admin] RBAC: Authorizing `identity:get_group(group_id=f26eb533e57a456d8f95265e204d21b5)` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 10:41:38.842430 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f48701ec-eac9-436b-9531-3e78aa54df3e None admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 10:41:38.844454 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 176/368] 10.222.0.25 () {56 vars in 1180 bytes} [Sun Aug 7 10:41:38 2022] GET /identity/v3/groups/f26eb533e57a456d8f95265e204d21b5 => generated 226 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 10:41:38.845516 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f48701ec-eac9-436b-9531-3e78aa54df3e None admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 10:41:38.919971 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-484834dd-0dc9-4ff3-8199-5015b1759719 None admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 10:41:38.967162 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3125c488-c51a-4888-9eab-8fad01ef7258 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 10:41:38.967786 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3125c488-c51a-4888-9eab-8fad01ef7258 None admin] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:41:38.968068 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3125c488-c51a-4888-9eab-8fad01ef7258 None admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:41:38.968338 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3125c488-c51a-4888-9eab-8fad01ef7258 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:41:38.969066 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3125c488-c51a-4888-9eab-8fad01ef7258 None admin] RBAC: Adding query filter params (group.id=f26eb533e57a456d8f95265e204d21b5, role.id=1512d531ebe248ea8e2d7d56d754b230, scope.project.id=90718391b09e493bb296767dc8a3c0ce) {{(pid=80144) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 10:41:38.969372 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3125c488-c51a-4888-9eab-8fad01ef7258 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 10:41:38.970799 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3125c488-c51a-4888-9eab-8fad01ef7258 None admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 10:41:38.977753 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f48701ec-eac9-436b-9531-3e78aa54df3e None admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 10:41:38.977918 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 193/369] 10.222.0.25 () {56 vars in 1376 bytes} [Sun Aug 7 10:41:38 2022] GET /identity/v3/role_assignments?group.id=f26eb533e57a456d8f95265e204d21b5&scope.project.id=90718391b09e493bb296767dc8a3c0ce&role.id=1512d531ebe248ea8e2d7d56d754b230 => generated 260 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 10:41:38.978029 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3125c488-c51a-4888-9eab-8fad01ef7258 None admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 10:41:39.030532 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e458435f-3656-4653-b48b-29e239e4881b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 10:41:39.031238 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e458435f-3656-4653-b48b-29e239e4881b None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:41:39.031618 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e458435f-3656-4653-b48b-29e239e4881b None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:41:39.031794 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e458435f-3656-4653-b48b-29e239e4881b None admin] PATH_INFO: `/v3/roles` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:41:39.032586 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e458435f-3656-4653-b48b-29e239e4881b None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80143) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 10:41:39.033660 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e458435f-3656-4653-b48b-29e239e4881b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 10:41:39.034300 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e458435f-3656-4653-b48b-29e239e4881b None admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 10:41:39.044200 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 177/370] 10.222.0.25 () {56 vars in 1111 bytes} [Sun Aug 7 10:41:38 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 10:41:39.045353 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e458435f-3656-4653-b48b-29e239e4881b None admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 10:41:39.066952 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=100 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 10:41:39.095161 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=100 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 10:41:39.129242 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3125c488-c51a-4888-9eab-8fad01ef7258 None admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 10:41:39.176958 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36e2e856-7db9-466b-93a4-04f56eff8014 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 10:41:39.177166 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e458435f-3656-4653-b48b-29e239e4881b None admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 10:41:39.177915 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36e2e856-7db9-466b-93a4-04f56eff8014 None admin] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:41:39.178516 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36e2e856-7db9-466b-93a4-04f56eff8014 None admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:41:39.178959 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36e2e856-7db9-466b-93a4-04f56eff8014 None admin] PATH_INFO: `/v3/roles` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:41:39.179774 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36e2e856-7db9-466b-93a4-04f56eff8014 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80144) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 10:41:39.181026 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36e2e856-7db9-466b-93a4-04f56eff8014 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 10:41:39.183295 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36e2e856-7db9-466b-93a4-04f56eff8014 None admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 10:41:39.194182 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 194/371] 10.222.0.25 () {56 vars in 1099 bytes} [Sun Aug 7 10:41:38 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 216 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 10:41:39.195398 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36e2e856-7db9-466b-93a4-04f56eff8014 None admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 10:41:39.220650 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afeeb78f-8f28-4d6b-899a-feb86360ef2a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 10:41:39.221409 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afeeb78f-8f28-4d6b-899a-feb86360ef2a None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:41:39.222179 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afeeb78f-8f28-4d6b-899a-feb86360ef2a None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:41:39.222179 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afeeb78f-8f28-4d6b-899a-feb86360ef2a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:41:39.223702 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afeeb78f-8f28-4d6b-899a-feb86360ef2a None admin] RBAC: Adding query filter params (group.id=f26eb533e57a456d8f95265e204d21b5, role.id=b98dfb3b9fee4644af2887a3ac3fd2e6, scope.project.id=d88586f68b734382abb9836b290aeb54) {{(pid=80143) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 10:41:39.224183 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afeeb78f-8f28-4d6b-899a-feb86360ef2a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 10:41:39.226116 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afeeb78f-8f28-4d6b-899a-feb86360ef2a None admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 10:41:39.232659 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 178/372] 10.222.0.25 () {56 vars in 1376 bytes} [Sun Aug 7 10:41:39 2022] GET /identity/v3/role_assignments?group.id=f26eb533e57a456d8f95265e204d21b5&scope.project.id=d88586f68b734382abb9836b290aeb54&role.id=b98dfb3b9fee4644af2887a3ac3fd2e6 => generated 260 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 10:41:39.234589 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afeeb78f-8f28-4d6b-899a-feb86360ef2a None admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 10:41:39.326173 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36e2e856-7db9-466b-93a4-04f56eff8014 None admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 10:41:39.332139 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=100 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 10:41:39.361236 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afeeb78f-8f28-4d6b-899a-feb86360ef2a None admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 10:41:39.378818 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-670a4f6d-b581-4e70-bdc8-6f38987f10fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 10:41:39.379490 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-670a4f6d-b581-4e70-bdc8-6f38987f10fd None admin] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:41:39.379776 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-670a4f6d-b581-4e70-bdc8-6f38987f10fd None admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:41:39.380056 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-670a4f6d-b581-4e70-bdc8-6f38987f10fd None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:41:39.392449 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=100 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 10:41:39.409498 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e09b294-e820-48c9-9c48-55119503f1b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 10:41:39.410525 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e09b294-e820-48c9-9c48-55119503f1b5 None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:41:39.411065 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e09b294-e820-48c9-9c48-55119503f1b5 None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:41:39.411614 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e09b294-e820-48c9-9c48-55119503f1b5 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:41:39.477132 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-670a4f6d-b581-4e70-bdc8-6f38987f10fd None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 10:41:39.478845 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-670a4f6d-b581-4e70-bdc8-6f38987f10fd None admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 10:41:39.502312 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e09b294-e820-48c9-9c48-55119503f1b5 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 10:41:39.504150 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e09b294-e820-48c9-9c48-55119503f1b5 None admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 10:41:39.598288 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 195/373] 10.222.0.25 () {58 vars in 1282 bytes} [Sun Aug 7 10:41:39 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 403 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 07 10:41:39.602477 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=100 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 10:41:39.604902 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-670a4f6d-b581-4e70-bdc8-6f38987f10fd None admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 10:41:39.643772 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 179/374] 10.222.0.25 () {58 vars in 1282 bytes} [Sun Aug 7 10:41:39 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 410 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 07 10:41:39.647590 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e09b294-e820-48c9-9c48-55119503f1b5 None admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 10:41:39.678987 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=100 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 10:41:39.781711 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-670a4f6d-b581-4e70-bdc8-6f38987f10fd None admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 10:41:39.840441 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7e298e7-c7d4-4eca-892a-c32a8f1ec0e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 10:41:39.841361 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7e298e7-c7d4-4eca-892a-c32a8f1ec0e5 None admin] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:41:39.841975 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7e298e7-c7d4-4eca-892a-c32a8f1ec0e5 None admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:41:39.843024 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7e298e7-c7d4-4eca-892a-c32a8f1ec0e5 None admin] PATH_INFO: `/v3/projects/d88586f68b734382abb9836b290aeb54` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:41:39.845145 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7e298e7-c7d4-4eca-892a-c32a8f1ec0e5 None admin] RBAC: Authorizing `identity:get_project(project_id=d88586f68b734382abb9836b290aeb54)` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 10:41:39.846883 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7e298e7-c7d4-4eca-892a-c32a8f1ec0e5 None admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 10:41:39.851626 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 196/375] 10.222.0.25 () {56 vars in 1186 bytes} [Sun Aug 7 10:41:39 2022] GET /identity/v3/projects/d88586f68b734382abb9836b290aeb54 => generated 302 bytes in 248 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 07 10:41:39.855098 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7e298e7-c7d4-4eca-892a-c32a8f1ec0e5 None admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 10:41:39.858450 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e09b294-e820-48c9-9c48-55119503f1b5 None admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 10:41:39.909708 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-424dfc4f-5dfa-48b1-965d-38866b6ca578 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 10:41:39.910597 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-424dfc4f-5dfa-48b1-965d-38866b6ca578 None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:41:39.911075 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-424dfc4f-5dfa-48b1-965d-38866b6ca578 None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:41:39.911509 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-424dfc4f-5dfa-48b1-965d-38866b6ca578 None admin] PATH_INFO: `/v3/projects/d5ac47e3b09f43d19a23a0f457fe9e23` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:41:39.912812 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-424dfc4f-5dfa-48b1-965d-38866b6ca578 None admin] RBAC: Authorizing `identity:get_project(project_id=d5ac47e3b09f43d19a23a0f457fe9e23)` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 10:41:39.914381 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-424dfc4f-5dfa-48b1-965d-38866b6ca578 None admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 10:41:39.917351 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 180/376] 10.222.0.25 () {56 vars in 1186 bytes} [Sun Aug 7 10:41:39 2022] GET /identity/v3/projects/d5ac47e3b09f43d19a23a0f457fe9e23 => generated 344 bytes in 269 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 10:41:39.922680 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-424dfc4f-5dfa-48b1-965d-38866b6ca578 None admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 10:41:39.985830 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=100 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 10:41:40.063738 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-424dfc4f-5dfa-48b1-965d-38866b6ca578 None admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 10:41:40.089577 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7e298e7-c7d4-4eca-892a-c32a8f1ec0e5 None admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 10:41:40.102031 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=100 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 10:41:40.125571 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea03b225-8124-4331-bb37-484702c5fd13 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 10:41:40.126214 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea03b225-8124-4331-bb37-484702c5fd13 None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:41:40.126498 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea03b225-8124-4331-bb37-484702c5fd13 None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:41:40.126775 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea03b225-8124-4331-bb37-484702c5fd13 None admin] PATH_INFO: `/v3/groups/f827aa44240646458903fb806aa670c7` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:41:40.133453 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea03b225-8124-4331-bb37-484702c5fd13 None admin] RBAC: Authorizing `identity:get_group(group_id=f827aa44240646458903fb806aa670c7)` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 10:41:40.135155 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea03b225-8124-4331-bb37-484702c5fd13 None admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 10:41:40.137703 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 181/377] 10.222.0.25 () {56 vars in 1180 bytes} [Sun Aug 7 10:41:39 2022] GET /identity/v3/groups/f827aa44240646458903fb806aa670c7 => generated 229 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 07 10:41:40.142854 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b097bc0c-2420-4c3e-ad5f-192ede086469 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 10:41:40.143222 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea03b225-8124-4331-bb37-484702c5fd13 None admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 10:41:40.143856 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b097bc0c-2420-4c3e-ad5f-192ede086469 None admin] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:41:40.144382 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b097bc0c-2420-4c3e-ad5f-192ede086469 None admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:41:40.144820 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b097bc0c-2420-4c3e-ad5f-192ede086469 None admin] PATH_INFO: `/v3/groups/f26eb533e57a456d8f95265e204d21b5` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:41:40.146118 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b097bc0c-2420-4c3e-ad5f-192ede086469 None admin] RBAC: Authorizing `identity:get_group(group_id=f26eb533e57a456d8f95265e204d21b5)` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 10:41:40.148035 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b097bc0c-2420-4c3e-ad5f-192ede086469 None admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 10:41:40.150052 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 197/378] 10.222.0.25 () {56 vars in 1180 bytes} [Sun Aug 7 10:41:39 2022] GET /identity/v3/groups/f26eb533e57a456d8f95265e204d21b5 => generated 226 bytes in 295 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 07 10:41:40.156307 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b097bc0c-2420-4c3e-ad5f-192ede086469 None admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 10:41:40.257500 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea03b225-8124-4331-bb37-484702c5fd13 None admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 10:41:40.268187 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=100 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 10:41:40.273685 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b097bc0c-2420-4c3e-ad5f-192ede086469 None admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 10:41:40.295494 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5172a650-417b-40c5-b55b-9c26771110f8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 10:41:40.296122 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5172a650-417b-40c5-b55b-9c26771110f8 None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:41:40.296407 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5172a650-417b-40c5-b55b-9c26771110f8 None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:41:40.296682 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5172a650-417b-40c5-b55b-9c26771110f8 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:41:40.297445 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5172a650-417b-40c5-b55b-9c26771110f8 None admin] RBAC: Adding query filter params (group.id=f827aa44240646458903fb806aa670c7, role.id=81b1f1bcee2a473088fbbd327c1c5b23, scope.project.id=d5ac47e3b09f43d19a23a0f457fe9e23) {{(pid=80143) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 10:41:40.297722 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5172a650-417b-40c5-b55b-9c26771110f8 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 10:41:40.299278 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5172a650-417b-40c5-b55b-9c26771110f8 None admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 10:41:40.305649 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 182/379] 10.222.0.25 () {56 vars in 1376 bytes} [Sun Aug 7 10:41:40 2022] GET /identity/v3/role_assignments?group.id=f827aa44240646458903fb806aa670c7&scope.project.id=d5ac47e3b09f43d19a23a0f457fe9e23&role.id=81b1f1bcee2a473088fbbd327c1c5b23 => generated 260 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 10:41:40.312294 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-845ef832-1828-4066-9d5f-280c9bf89049 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 10:41:40.312937 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-845ef832-1828-4066-9d5f-280c9bf89049 None admin] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:41:40.313220 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-845ef832-1828-4066-9d5f-280c9bf89049 None admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:41:40.313884 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-845ef832-1828-4066-9d5f-280c9bf89049 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:41:40.314652 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-845ef832-1828-4066-9d5f-280c9bf89049 None admin] RBAC: Adding query filter params (group.id=f26eb533e57a456d8f95265e204d21b5, role.id=1512d531ebe248ea8e2d7d56d754b230, scope.project.id=d88586f68b734382abb9836b290aeb54) {{(pid=80144) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 10:41:40.315000 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-845ef832-1828-4066-9d5f-280c9bf89049 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 10:41:40.315888 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d46a6bc7-3e9c-49b6-9925-ab474bc79623 None None] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:41:40.316378 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d46a6bc7-3e9c-49b6-9925-ab474bc79623 None None] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:41:40.316491 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-845ef832-1828-4066-9d5f-280c9bf89049 None admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 10:41:40.316970 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d46a6bc7-3e9c-49b6-9925-ab474bc79623 None None] PATH_INFO: `/` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:41:40.319133 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 183/380] 10.222.0.25 () {54 vars in 927 bytes} [Sun Aug 7 10:41:40 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 07 10:41:40.326956 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 198/381] 10.222.0.25 () {56 vars in 1376 bytes} [Sun Aug 7 10:41:40 2022] GET /identity/v3/role_assignments?group.id=f26eb533e57a456d8f95265e204d21b5&scope.project.id=d88586f68b734382abb9836b290aeb54&role.id=1512d531ebe248ea8e2d7d56d754b230 => generated 260 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 10:41:40.329492 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4755a651-eaa7-4585-bf3c-59e2e2ec36d7 None None] REQUEST_METHOD: `POST` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:41:40.330030 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4755a651-eaa7-4585-bf3c-59e2e2ec36d7 None None] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:41:40.330512 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4755a651-eaa7-4585-bf3c-59e2e2ec36d7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:41:40.355988 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.auth.core [None req-4755a651-eaa7-4585-bf3c-59e2e2ec36d7 None None] MFA Rules not processed for user `bac4e6b76fe64217ac8fa44e71cffe6a`. Rule list: `[]` (Enabled: `True`). {{(pid=80143) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 10:41:40.357461 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-4755a651-eaa7-4585-bf3c-59e2e2ec36d7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 10:41:40.460691 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 184/382] 10.222.0.25 () {58 vars in 1024 bytes} [Sun Aug 7 10:41:40 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 07 10:41:40.502466 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-845ef832-1828-4066-9d5f-280c9bf89049 None admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 10:41:40.556935 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=100 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 10:41:40.630303 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-845ef832-1828-4066-9d5f-280c9bf89049 None admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 10:41:40.668755 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7a4c028-422c-4638-94b0-a411297e240d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 10:41:40.669400 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7a4c028-422c-4638-94b0-a411297e240d None admin] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:41:40.669686 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7a4c028-422c-4638-94b0-a411297e240d None admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:41:40.669963 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7a4c028-422c-4638-94b0-a411297e240d None admin] PATH_INFO: `/v3/roles/member` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:41:40.681575 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7a4c028-422c-4638-94b0-a411297e240d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 07 10:41:40.686728 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application [None req-e7a4c028-422c-4638-94b0-a411297e240d None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 07 10:41:40.686728 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 07 10:41:40.686728 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 07 10:41:40.686728 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 07 10:41:40.686728 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 07 10:41:40.686728 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 07 10:41:40.686728 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 07 10:41:40.686728 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 07 10:41:40.686728 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 07 10:41:40.686728 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 07 10:41:40.686728 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 07 10:41:40.686728 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 07 10:41:40.686728 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 07 10:41:40.686728 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 07 10:41:40.686728 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 07 10:41:40.686728 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 07 10:41:40.687686 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 07 10:41:40.687686 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 07 10:41:40.687686 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 07 10:41:40.687686 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 07 10:41:40.687686 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 07 10:41:40.687686 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 07 10:41:40.687686 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 07 10:41:40.687686 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 07 10:41:40.687686 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 07 10:41:40.687686 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self.get_or_create( Aug 07 10:41:40.687686 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 07 10:41:40.687686 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application with Lock( Aug 07 10:41:40.687686 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 07 10:41:40.687686 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self._enter() Aug 07 10:41:40.687686 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 07 10:41:40.687686 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 07 10:41:40.689033 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 07 10:41:40.689033 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self.creator() Aug 07 10:41:40.689033 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 07 10:41:40.689033 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application created_value = creator( Aug 07 10:41:40.689033 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 07 10:41:40.689033 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 07 10:41:40.689033 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 07 10:41:40.689033 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 07 10:41:40.689033 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 07 10:41:40.689033 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 07 10:41:40.689033 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 07 10:41:40.689033 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application Aug 07 10:41:40.689033 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 199/383] 10.222.0.25 () {56 vars in 1099 bytes} [Sun Aug 7 10:41:40 2022] GET /identity/v3/roles/member => generated 84 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 07 10:41:40.693967 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4755a651-eaa7-4585-bf3c-59e2e2ec36d7 None None] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 10:41:40.815549 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4755a651-eaa7-4585-bf3c-59e2e2ec36d7 None None] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 10:41:40.817196 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=100 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 10:41:40.857146 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-556ca157-2e23-416f-939e-770145c6124f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 10:41:40.857773 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-556ca157-2e23-416f-939e-770145c6124f None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:41:40.858066 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-556ca157-2e23-416f-939e-770145c6124f None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:41:40.858344 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-556ca157-2e23-416f-939e-770145c6124f None admin] PATH_INFO: `/v3/roles` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:41:40.859046 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-556ca157-2e23-416f-939e-770145c6124f None admin] RBAC: Adding query filter params (name=member) {{(pid=80143) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 10:41:40.859356 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-556ca157-2e23-416f-939e-770145c6124f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 10:41:40.860822 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-556ca157-2e23-416f-939e-770145c6124f None admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 10:41:40.872640 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 185/384] 10.222.0.25 () {56 vars in 1101 bytes} [Sun Aug 7 10:41:40 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 10:41:40.878555 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7a4c028-422c-4638-94b0-a411297e240d None admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 10:41:40.988583 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=100 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 10:41:40.993111 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7a4c028-422c-4638-94b0-a411297e240d None admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 10:41:41.033642 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ccb70ce-9ff6-4d82-944a-298a28521ade None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 10:41:41.034290 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ccb70ce-9ff6-4d82-944a-298a28521ade None admin] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:41:41.034628 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ccb70ce-9ff6-4d82-944a-298a28521ade None admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:41:41.034915 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ccb70ce-9ff6-4d82-944a-298a28521ade None admin] PATH_INFO: `/v3/groups/f26eb533e57a456d8f95265e204d21b5` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:41:41.036064 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ccb70ce-9ff6-4d82-944a-298a28521ade None admin] RBAC: Authorizing `identity:get_group(group_id=f26eb533e57a456d8f95265e204d21b5)` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 10:41:41.037521 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ccb70ce-9ff6-4d82-944a-298a28521ade None admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 10:41:41.039153 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 200/385] 10.222.0.25 () {56 vars in 1180 bytes} [Sun Aug 7 10:41:40 2022] GET /identity/v3/groups/f26eb533e57a456d8f95265e204d21b5 => generated 226 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 07 10:41:41.044852 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-556ca157-2e23-416f-939e-770145c6124f None admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 10:41:41.155160 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-556ca157-2e23-416f-939e-770145c6124f None admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 10:41:41.193144 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe46e427-503b-4d4a-825b-be5b6c7af7f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 10:41:41.193848 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe46e427-503b-4d4a-825b-be5b6c7af7f4 None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:41:41.194131 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe46e427-503b-4d4a-825b-be5b6c7af7f4 None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:41:41.194455 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe46e427-503b-4d4a-825b-be5b6c7af7f4 None admin] PATH_INFO: `/v3/projects/90718391b09e493bb296767dc8a3c0ce` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:41:41.195624 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe46e427-503b-4d4a-825b-be5b6c7af7f4 None admin] RBAC: Authorizing `identity:get_project(project_id=90718391b09e493bb296767dc8a3c0ce)` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 10:41:41.197068 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe46e427-503b-4d4a-825b-be5b6c7af7f4 None admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 10:41:41.203112 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 186/386] 10.222.0.25 () {56 vars in 1186 bytes} [Sun Aug 7 10:41:41 2022] GET /identity/v3/projects/90718391b09e493bb296767dc8a3c0ce => generated 298 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 10:41:41.204808 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ccb70ce-9ff6-4d82-944a-298a28521ade None admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 10:41:41.315569 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ccb70ce-9ff6-4d82-944a-298a28521ade None admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 10:41:41.351489 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bda17c82-ad27-45fe-bd70-927e463d8432 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 10:41:41.352162 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bda17c82-ad27-45fe-bd70-927e463d8432 None admin] REQUEST_METHOD: `PUT` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:41:41.352448 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bda17c82-ad27-45fe-bd70-927e463d8432 None admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:41:41.352732 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bda17c82-ad27-45fe-bd70-927e463d8432 None admin] PATH_INFO: `/v3/projects/90718391b09e493bb296767dc8a3c0ce/groups/f26eb533e57a456d8f95265e204d21b5/roles/b98dfb3b9fee4644af2887a3ac3fd2e6` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:41:41.355226 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bda17c82-ad27-45fe-bd70-927e463d8432 None admin] RBAC: Authorizing `identity:create_grant(project_id=90718391b09e493bb296767dc8a3c0ce, group_id=f26eb533e57a456d8f95265e204d21b5, role_id=b98dfb3b9fee4644af2887a3ac3fd2e6)` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 10:41:41.356765 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bda17c82-ad27-45fe-bd70-927e463d8432 None admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 10:41:41.372626 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 201/387] 10.222.0.25 () {58 vars in 1442 bytes} [Sun Aug 7 10:41:41 2022] PUT /identity/v3/projects/90718391b09e493bb296767dc8a3c0ce/groups/f26eb533e57a456d8f95265e204d21b5/roles/b98dfb3b9fee4644af2887a3ac3fd2e6 => generated 0 bytes in 168 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 07 10:41:41.628250 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e660ad30-45c2-47db-8d7d-84140e329ec9 None None] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:41:41.628459 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e660ad30-45c2-47db-8d7d-84140e329ec9 None None] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:41:41.628985 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e660ad30-45c2-47db-8d7d-84140e329ec9 None None] PATH_INFO: `/` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:41:41.630752 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 187/388] 10.222.0.25 () {54 vars in 927 bytes} [Sun Aug 7 10:41:41 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 07 10:41:41.639094 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a5ba752-b649-4222-b903-65a5669da1b8 None None] REQUEST_METHOD: `POST` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:41:41.640081 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a5ba752-b649-4222-b903-65a5669da1b8 None None] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:41:41.640562 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a5ba752-b649-4222-b903-65a5669da1b8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:41:41.661633 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.auth.core [None req-1a5ba752-b649-4222-b903-65a5669da1b8 None None] MFA Rules not processed for user `bac4e6b76fe64217ac8fa44e71cffe6a`. Rule list: `[]` (Enabled: `True`). {{(pid=80144) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 10:41:41.663212 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.fernet_utils [None req-1a5ba752-b649-4222-b903-65a5669da1b8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80144) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 10:41:41.684906 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 10:41:41.753808 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 202/389] 10.222.0.25 () {58 vars in 1024 bytes} [Sun Aug 7 10:41:41 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 07 10:41:41.791287 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e660ad30-45c2-47db-8d7d-84140e329ec9 None None] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 10:41:41.842393 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=100 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 10:41:41.964165 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e660ad30-45c2-47db-8d7d-84140e329ec9 None None] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 10:41:42.011225 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8479b5c1-6dc8-4e78-87aa-002fe14e6a27 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 10:41:42.011872 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8479b5c1-6dc8-4e78-87aa-002fe14e6a27 None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:41:42.012153 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8479b5c1-6dc8-4e78-87aa-002fe14e6a27 None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:41:42.012423 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8479b5c1-6dc8-4e78-87aa-002fe14e6a27 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:41:42.024574 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8479b5c1-6dc8-4e78-87aa-002fe14e6a27 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 07 10:41:42.029402 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application [None req-8479b5c1-6dc8-4e78-87aa-002fe14e6a27 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 07 10:41:42.029402 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 07 10:41:42.029402 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 07 10:41:42.029402 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 07 10:41:42.029402 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 07 10:41:42.029402 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 07 10:41:42.029402 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 07 10:41:42.029402 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 07 10:41:42.029402 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 07 10:41:42.029402 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 07 10:41:42.029402 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 07 10:41:42.029402 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 07 10:41:42.029402 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 07 10:41:42.029402 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 07 10:41:42.029402 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 07 10:41:42.029402 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 07 10:41:42.030277 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 07 10:41:42.030277 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 07 10:41:42.030277 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 07 10:41:42.030277 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 07 10:41:42.030277 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 07 10:41:42.030277 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 07 10:41:42.030277 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 07 10:41:42.030277 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 07 10:41:42.030277 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 07 10:41:42.030277 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.get_or_create( Aug 07 10:41:42.030277 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 07 10:41:42.030277 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application with Lock( Aug 07 10:41:42.030277 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 07 10:41:42.030277 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self._enter() Aug 07 10:41:42.030277 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 07 10:41:42.030277 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 07 10:41:42.031108 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 07 10:41:42.031108 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.creator() Aug 07 10:41:42.031108 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 07 10:41:42.031108 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application created_value = creator( Aug 07 10:41:42.031108 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 07 10:41:42.031108 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 07 10:41:42.031108 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 07 10:41:42.031108 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 07 10:41:42.031108 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 07 10:41:42.031108 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 07 10:41:42.031108 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 07 10:41:42.031108 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application Aug 07 10:41:42.033221 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 188/390] 10.222.0.25 () {56 vars in 1114 bytes} [Sun Aug 7 10:41:41 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 240 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 07 10:41:42.040038 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a5ba752-b649-4222-b903-65a5669da1b8 None None] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 10:41:42.171571 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a5ba752-b649-4222-b903-65a5669da1b8 None None] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 10:41:42.218489 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68ffa0d6-2d77-4001-9d66-b51294ef504f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 10:41:42.219048 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68ffa0d6-2d77-4001-9d66-b51294ef504f None admin] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:41:42.219331 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68ffa0d6-2d77-4001-9d66-b51294ef504f None admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:41:42.219603 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68ffa0d6-2d77-4001-9d66-b51294ef504f None admin] PATH_INFO: `/v3/roles` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:41:42.220286 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68ffa0d6-2d77-4001-9d66-b51294ef504f None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80144) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 10:41:42.220593 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68ffa0d6-2d77-4001-9d66-b51294ef504f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 10:41:42.221956 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68ffa0d6-2d77-4001-9d66-b51294ef504f None admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 10:41:42.231522 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 203/391] 10.222.0.25 () {56 vars in 1111 bytes} [Sun Aug 7 10:41:42 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 10:41:42.238333 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8479b5c1-6dc8-4e78-87aa-002fe14e6a27 None admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 10:41:42.321389 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c61499fd-450f-4541-bf85-fd33128dac92 None None] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:41:42.321691 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c61499fd-450f-4541-bf85-fd33128dac92 None None] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:41:42.321952 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c61499fd-450f-4541-bf85-fd33128dac92 None None] PATH_INFO: `/` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:41:42.323863 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 204/392] 10.222.0.25 () {54 vars in 927 bytes} [Sun Aug 7 10:41:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 07 10:41:42.332127 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c06a721a-8cb0-4a0e-a23d-19c8816674ad None None] REQUEST_METHOD: `POST` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:41:42.332411 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c06a721a-8cb0-4a0e-a23d-19c8816674ad None None] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:41:42.332686 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c06a721a-8cb0-4a0e-a23d-19c8816674ad None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:41:42.365914 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=100 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 10:41:42.378099 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.auth.core [None req-c06a721a-8cb0-4a0e-a23d-19c8816674ad None None] MFA Rules not processed for user `bac4e6b76fe64217ac8fa44e71cffe6a`. Rule list: `[]` (Enabled: `True`). {{(pid=80144) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 10:41:42.381017 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8479b5c1-6dc8-4e78-87aa-002fe14e6a27 None admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 10:41:42.382994 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.fernet_utils [None req-c06a721a-8cb0-4a0e-a23d-19c8816674ad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80144) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 10:41:42.409239 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=100 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 10:41:42.444993 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-673d4faf-e43e-492b-baea-3a3d5fc3f868 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 10:41:42.445640 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-673d4faf-e43e-492b-baea-3a3d5fc3f868 None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:41:42.445925 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-673d4faf-e43e-492b-baea-3a3d5fc3f868 None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:41:42.446202 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-673d4faf-e43e-492b-baea-3a3d5fc3f868 None admin] PATH_INFO: `/v3/groups/f26eb533e57a456d8f95265e204d21b5` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:41:42.447776 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-673d4faf-e43e-492b-baea-3a3d5fc3f868 None admin] RBAC: Authorizing `identity:get_group(group_id=f26eb533e57a456d8f95265e204d21b5)` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 10:41:42.449954 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-673d4faf-e43e-492b-baea-3a3d5fc3f868 None admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 10:41:42.451738 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 189/393] 10.222.0.25 () {56 vars in 1180 bytes} [Sun Aug 7 10:41:42 2022] GET /identity/v3/groups/f26eb533e57a456d8f95265e204d21b5 => generated 226 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 10:41:42.461188 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-673d4faf-e43e-492b-baea-3a3d5fc3f868 None admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 10:41:42.520069 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 205/394] 10.222.0.25 () {58 vars in 1024 bytes} [Sun Aug 7 10:41:42 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 189 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 07 10:41:42.553858 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c06a721a-8cb0-4a0e-a23d-19c8816674ad None None] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 10:41:42.583172 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-673d4faf-e43e-492b-baea-3a3d5fc3f868 None admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 10:41:42.618879 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6cbea76-e2bd-4973-b68e-08252e3be7f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 10:41:42.619554 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6cbea76-e2bd-4973-b68e-08252e3be7f5 None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:41:42.619924 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6cbea76-e2bd-4973-b68e-08252e3be7f5 None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:41:42.620375 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6cbea76-e2bd-4973-b68e-08252e3be7f5 None admin] PATH_INFO: `/v3/projects/90718391b09e493bb296767dc8a3c0ce` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:41:42.626947 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6cbea76-e2bd-4973-b68e-08252e3be7f5 None admin] RBAC: Authorizing `identity:get_project(project_id=90718391b09e493bb296767dc8a3c0ce)` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 10:41:42.626947 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6cbea76-e2bd-4973-b68e-08252e3be7f5 None admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 10:41:42.626947 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 190/395] 10.222.0.25 () {56 vars in 1186 bytes} [Sun Aug 7 10:41:42 2022] GET /identity/v3/projects/90718391b09e493bb296767dc8a3c0ce => generated 298 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 10:41:42.630863 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6cbea76-e2bd-4973-b68e-08252e3be7f5 None admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 10:41:42.689289 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=100 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 10:41:42.774072 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6cbea76-e2bd-4973-b68e-08252e3be7f5 None admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 10:41:42.787319 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c06a721a-8cb0-4a0e-a23d-19c8816674ad None None] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 10:41:42.815705 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31614247-5cd6-461d-9f67-3bcc925f3f7d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 10:41:42.816375 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31614247-5cd6-461d-9f67-3bcc925f3f7d None admin] REQUEST_METHOD: `PUT` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:41:42.816662 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31614247-5cd6-461d-9f67-3bcc925f3f7d None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:41:42.816938 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31614247-5cd6-461d-9f67-3bcc925f3f7d None admin] PATH_INFO: `/v3/projects/90718391b09e493bb296767dc8a3c0ce/groups/f26eb533e57a456d8f95265e204d21b5/roles/1512d531ebe248ea8e2d7d56d754b230` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:41:42.819579 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31614247-5cd6-461d-9f67-3bcc925f3f7d None admin] RBAC: Authorizing `identity:create_grant(project_id=90718391b09e493bb296767dc8a3c0ce, group_id=f26eb533e57a456d8f95265e204d21b5, role_id=1512d531ebe248ea8e2d7d56d754b230)` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 10:41:42.821228 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31614247-5cd6-461d-9f67-3bcc925f3f7d None admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 10:41:42.841791 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 191/396] 10.222.0.25 () {58 vars in 1442 bytes} [Sun Aug 7 10:41:42 2022] PUT /identity/v3/projects/90718391b09e493bb296767dc8a3c0ce/groups/f26eb533e57a456d8f95265e204d21b5/roles/1512d531ebe248ea8e2d7d56d754b230 => generated 0 bytes in 211 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 07 10:41:42.854118 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d004d9fe-3263-4fdf-a09d-da43a0c21129 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 10:41:42.855008 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d004d9fe-3263-4fdf-a09d-da43a0c21129 None admin] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:41:42.855299 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d004d9fe-3263-4fdf-a09d-da43a0c21129 None admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:41:42.855630 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d004d9fe-3263-4fdf-a09d-da43a0c21129 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:41:42.868233 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d004d9fe-3263-4fdf-a09d-da43a0c21129 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 07 10:41:42.874301 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application [None req-d004d9fe-3263-4fdf-a09d-da43a0c21129 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 07 10:41:42.874301 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 07 10:41:42.874301 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 07 10:41:42.874301 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 07 10:41:42.874301 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 07 10:41:42.874301 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 07 10:41:42.874301 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 07 10:41:42.874301 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 07 10:41:42.874301 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 07 10:41:42.874301 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 07 10:41:42.874301 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 07 10:41:42.874301 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 07 10:41:42.874301 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 07 10:41:42.874301 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 07 10:41:42.874301 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 07 10:41:42.874301 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 07 10:41:42.875314 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 07 10:41:42.875314 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 07 10:41:42.875314 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 07 10:41:42.875314 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 07 10:41:42.875314 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 07 10:41:42.875314 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 07 10:41:42.875314 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 07 10:41:42.875314 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 07 10:41:42.875314 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 07 10:41:42.875314 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self.get_or_create( Aug 07 10:41:42.875314 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 07 10:41:42.875314 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application with Lock( Aug 07 10:41:42.875314 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 07 10:41:42.875314 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self._enter() Aug 07 10:41:42.875314 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 07 10:41:42.875314 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 07 10:41:42.876133 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 07 10:41:42.876133 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self.creator() Aug 07 10:41:42.876133 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 07 10:41:42.876133 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application created_value = creator( Aug 07 10:41:42.876133 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 07 10:41:42.876133 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 07 10:41:42.876133 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 07 10:41:42.876133 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 07 10:41:42.876133 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 07 10:41:42.876133 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 07 10:41:42.876133 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 07 10:41:42.876133 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application Aug 07 10:41:42.876133 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 206/397] 10.222.0.25 () {56 vars in 1099 bytes} [Sun Aug 7 10:41:42 2022] GET /identity/v3/roles/member => generated 84 bytes in 322 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 07 10:41:42.884151 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31614247-5cd6-461d-9f67-3bcc925f3f7d None admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 10:41:42.993338 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 10:41:43.010014 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31614247-5cd6-461d-9f67-3bcc925f3f7d None admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 10:41:43.052091 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad5f51ad-07be-4b50-b593-509129426584 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 10:41:43.052690 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad5f51ad-07be-4b50-b593-509129426584 None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:41:43.052976 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad5f51ad-07be-4b50-b593-509129426584 None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:41:43.053254 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad5f51ad-07be-4b50-b593-509129426584 None admin] PATH_INFO: `/v3/roles` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:41:43.053987 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad5f51ad-07be-4b50-b593-509129426584 None admin] RBAC: Adding query filter params (name=member) {{(pid=80143) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 10:41:43.054298 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad5f51ad-07be-4b50-b593-509129426584 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 10:41:43.055689 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad5f51ad-07be-4b50-b593-509129426584 None admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 10:41:43.068797 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 192/398] 10.222.0.25 () {56 vars in 1101 bytes} [Sun Aug 7 10:41:42 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 10:41:43.075232 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d004d9fe-3263-4fdf-a09d-da43a0c21129 None admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 10:41:43.087535 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=100 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 10:41:43.095621 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c60684a3-a2b6-4933-ab10-b33faffb22c9 None None] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:41:43.095969 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c60684a3-a2b6-4933-ab10-b33faffb22c9 None None] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:41:43.096254 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c60684a3-a2b6-4933-ab10-b33faffb22c9 None None] PATH_INFO: `/` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:41:43.098512 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 193/399] 10.222.0.25 () {54 vars in 927 bytes} [Sun Aug 7 10:41:43 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 07 10:41:43.111738 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-485f45ef-014f-43fc-97a9-e5ff32d4a992 None None] REQUEST_METHOD: `POST` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:41:43.112396 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-485f45ef-014f-43fc-97a9-e5ff32d4a992 None None] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:41:43.112683 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-485f45ef-014f-43fc-97a9-e5ff32d4a992 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:41:43.141243 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.auth.core [None req-485f45ef-014f-43fc-97a9-e5ff32d4a992 None None] MFA Rules not processed for user `bac4e6b76fe64217ac8fa44e71cffe6a`. Rule list: `[]` (Enabled: `True`). {{(pid=80143) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 10:41:43.142837 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-485f45ef-014f-43fc-97a9-e5ff32d4a992 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 10:41:43.205077 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d004d9fe-3263-4fdf-a09d-da43a0c21129 None admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 10:41:43.220541 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 194/400] 10.222.0.25 () {58 vars in 1024 bytes} [Sun Aug 7 10:41:43 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 07 10:41:43.244154 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e105da51-e283-47fb-b2b1-c2291949ee64 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 10:41:43.244585 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-485f45ef-014f-43fc-97a9-e5ff32d4a992 None None] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 10:41:43.244856 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e105da51-e283-47fb-b2b1-c2291949ee64 None admin] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:41:43.245146 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e105da51-e283-47fb-b2b1-c2291949ee64 None admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:41:43.245478 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e105da51-e283-47fb-b2b1-c2291949ee64 None admin] PATH_INFO: `/v3/groups/f26eb533e57a456d8f95265e204d21b5` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:41:43.246627 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e105da51-e283-47fb-b2b1-c2291949ee64 None admin] RBAC: Authorizing `identity:get_group(group_id=f26eb533e57a456d8f95265e204d21b5)` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 10:41:43.248410 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e105da51-e283-47fb-b2b1-c2291949ee64 None admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 10:41:43.250311 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 207/401] 10.222.0.25 () {56 vars in 1180 bytes} [Sun Aug 7 10:41:43 2022] GET /identity/v3/groups/f26eb533e57a456d8f95265e204d21b5 => generated 226 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 10:41:43.255415 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e105da51-e283-47fb-b2b1-c2291949ee64 None admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 10:41:43.331488 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=100 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 10:41:43.340059 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=100 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 10:41:43.356111 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-485f45ef-014f-43fc-97a9-e5ff32d4a992 None None] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 10:41:43.366824 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e105da51-e283-47fb-b2b1-c2291949ee64 None admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 10:41:43.403592 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c1424f5-a451-4658-85b4-c1e2da252cab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 10:41:43.404398 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c1424f5-a451-4658-85b4-c1e2da252cab None admin] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:41:43.404626 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74ca2bc9-2f25-4918-bdb3-5aae1e96c122 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 10:41:43.404871 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c1424f5-a451-4658-85b4-c1e2da252cab None admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:41:43.405248 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c1424f5-a451-4658-85b4-c1e2da252cab None admin] PATH_INFO: `/v3/projects/d88586f68b734382abb9836b290aeb54` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:41:43.405373 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74ca2bc9-2f25-4918-bdb3-5aae1e96c122 None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:41:43.405547 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74ca2bc9-2f25-4918-bdb3-5aae1e96c122 None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:41:43.405828 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74ca2bc9-2f25-4918-bdb3-5aae1e96c122 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:41:43.406780 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c1424f5-a451-4658-85b4-c1e2da252cab None admin] RBAC: Authorizing `identity:get_project(project_id=d88586f68b734382abb9836b290aeb54)` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 10:41:43.410635 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c1424f5-a451-4658-85b4-c1e2da252cab None admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 10:41:43.418299 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 208/402] 10.222.0.25 () {56 vars in 1186 bytes} [Sun Aug 7 10:41:43 2022] GET /identity/v3/projects/d88586f68b734382abb9836b290aeb54 => generated 302 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 10:41:43.418299 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c98ce5ff-2288-47cb-8acf-e63cab193b20 None None] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:41:43.418299 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c98ce5ff-2288-47cb-8acf-e63cab193b20 None None] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:41:43.418299 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c98ce5ff-2288-47cb-8acf-e63cab193b20 None None] PATH_INFO: `/` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:41:43.418664 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74ca2bc9-2f25-4918-bdb3-5aae1e96c122 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 07 10:41:43.421815 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 209/403] 10.222.0.25 () {54 vars in 927 bytes} [Sun Aug 7 10:41:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 07 10:41:43.421815 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c98ce5ff-2288-47cb-8acf-e63cab193b20 None None] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 10:41:43.422239 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application [None req-74ca2bc9-2f25-4918-bdb3-5aae1e96c122 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 07 10:41:43.422239 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 07 10:41:43.422239 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 07 10:41:43.422239 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 07 10:41:43.422239 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 07 10:41:43.422239 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 07 10:41:43.422239 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 07 10:41:43.422239 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 07 10:41:43.422239 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 07 10:41:43.422239 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 07 10:41:43.422239 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 07 10:41:43.422239 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 07 10:41:43.422239 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 07 10:41:43.422239 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 07 10:41:43.422239 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 07 10:41:43.422239 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 07 10:41:43.423426 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 07 10:41:43.423426 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 07 10:41:43.423426 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 07 10:41:43.423426 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 07 10:41:43.423426 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 07 10:41:43.423426 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 07 10:41:43.423426 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 07 10:41:43.423426 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 07 10:41:43.423426 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 07 10:41:43.423426 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.get_or_create( Aug 07 10:41:43.423426 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 07 10:41:43.423426 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application with Lock( Aug 07 10:41:43.423426 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 07 10:41:43.423426 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self._enter() Aug 07 10:41:43.423426 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 07 10:41:43.423426 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 07 10:41:43.424412 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 07 10:41:43.424412 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.creator() Aug 07 10:41:43.424412 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 07 10:41:43.424412 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application created_value = creator( Aug 07 10:41:43.424412 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 07 10:41:43.424412 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 07 10:41:43.424412 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 07 10:41:43.424412 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 07 10:41:43.424412 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 07 10:41:43.424412 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 07 10:41:43.424412 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 07 10:41:43.424412 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application Aug 07 10:41:43.424412 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 195/404] 10.222.0.25 () {56 vars in 1096 bytes} [Sun Aug 7 10:41:43 2022] GET /identity/v3/roles/admin => generated 83 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 07 10:41:43.430360 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74ca2bc9-2f25-4918-bdb3-5aae1e96c122 None admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 10:41:43.540675 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c98ce5ff-2288-47cb-8acf-e63cab193b20 None None] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 10:41:43.571194 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74ca2bc9-2f25-4918-bdb3-5aae1e96c122 None admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 10:41:43.579363 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-975aeaba-9ef7-42be-b31a-7599997a1f65 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 10:41:43.580328 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-975aeaba-9ef7-42be-b31a-7599997a1f65 None admin] REQUEST_METHOD: `PUT` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:41:43.580939 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-975aeaba-9ef7-42be-b31a-7599997a1f65 None admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:41:43.581431 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-975aeaba-9ef7-42be-b31a-7599997a1f65 None admin] PATH_INFO: `/v3/projects/d88586f68b734382abb9836b290aeb54/groups/f26eb533e57a456d8f95265e204d21b5/roles/b98dfb3b9fee4644af2887a3ac3fd2e6` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:41:43.583758 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-975aeaba-9ef7-42be-b31a-7599997a1f65 None admin] RBAC: Authorizing `identity:create_grant(project_id=d88586f68b734382abb9836b290aeb54, group_id=f26eb533e57a456d8f95265e204d21b5, role_id=b98dfb3b9fee4644af2887a3ac3fd2e6)` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 10:41:43.585487 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-975aeaba-9ef7-42be-b31a-7599997a1f65 None admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 10:41:43.611320 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 210/405] 10.222.0.25 () {58 vars in 1442 bytes} [Sun Aug 7 10:41:43 2022] PUT /identity/v3/projects/d88586f68b734382abb9836b290aeb54/groups/f26eb533e57a456d8f95265e204d21b5/roles/b98dfb3b9fee4644af2887a3ac3fd2e6 => generated 0 bytes in 185 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 07 10:41:43.614491 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b97ca8d2-d976-4827-8918-c784d36d2b25 None None] REQUEST_METHOD: `POST` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:41:43.614778 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b97ca8d2-d976-4827-8918-c784d36d2b25 None None] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:41:43.615086 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b97ca8d2-d976-4827-8918-c784d36d2b25 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:41:43.616821 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94c7ba63-f960-48cd-aac3-b995ad8e7396 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 10:41:43.618958 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94c7ba63-f960-48cd-aac3-b995ad8e7396 None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:41:43.619331 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94c7ba63-f960-48cd-aac3-b995ad8e7396 None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:41:43.619610 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94c7ba63-f960-48cd-aac3-b995ad8e7396 None admin] PATH_INFO: `/v3/roles` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:41:43.622509 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94c7ba63-f960-48cd-aac3-b995ad8e7396 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80143) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 10:41:43.623496 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94c7ba63-f960-48cd-aac3-b995ad8e7396 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 10:41:43.626313 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94c7ba63-f960-48cd-aac3-b995ad8e7396 None admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 10:41:43.632870 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 10:41:43.647074 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 196/406] 10.222.0.25 () {56 vars in 1099 bytes} [Sun Aug 7 10:41:43 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 217 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 10:41:43.655355 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94c7ba63-f960-48cd-aac3-b995ad8e7396 None admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 10:41:43.661934 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=100 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 10:41:43.664577 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.auth.core [None req-b97ca8d2-d976-4827-8918-c784d36d2b25 None None] MFA Rules not processed for user `bac4e6b76fe64217ac8fa44e71cffe6a`. Rule list: `[]` (Enabled: `True`). {{(pid=80144) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 10:41:43.665948 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.fernet_utils [None req-b97ca8d2-d976-4827-8918-c784d36d2b25 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80144) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 10:41:43.813789 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 211/407] 10.222.0.25 () {58 vars in 1024 bytes} [Sun Aug 7 10:41:43 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 202 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 07 10:41:43.844851 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94c7ba63-f960-48cd-aac3-b995ad8e7396 None admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 10:41:43.845336 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b97ca8d2-d976-4827-8918-c784d36d2b25 None None] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 10:41:43.892535 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cab77471-b478-4290-974e-0061cbaa03c4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 10:41:43.893186 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cab77471-b478-4290-974e-0061cbaa03c4 None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:41:43.893470 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cab77471-b478-4290-974e-0061cbaa03c4 None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:41:43.893748 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cab77471-b478-4290-974e-0061cbaa03c4 None admin] PATH_INFO: `/v3/groups/f827aa44240646458903fb806aa670c7` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:41:43.895457 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cab77471-b478-4290-974e-0061cbaa03c4 None admin] RBAC: Authorizing `identity:get_group(group_id=f827aa44240646458903fb806aa670c7)` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 10:41:43.897440 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cab77471-b478-4290-974e-0061cbaa03c4 None admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 10:41:43.900144 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 197/408] 10.222.0.25 () {56 vars in 1180 bytes} [Sun Aug 7 10:41:43 2022] GET /identity/v3/groups/f827aa44240646458903fb806aa670c7 => generated 229 bytes in 245 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 10:41:43.901683 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e55a02d-901a-407c-a6ef-6000140d21df None None] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:41:43.901970 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e55a02d-901a-407c-a6ef-6000140d21df None None] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:41:43.903154 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e55a02d-901a-407c-a6ef-6000140d21df None None] PATH_INFO: `/` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:41:43.904641 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 198/409] 10.222.0.25 () {54 vars in 927 bytes} [Sun Aug 7 10:41:43 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 07 10:41:43.909399 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e55a02d-901a-407c-a6ef-6000140d21df None None] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 10:41:43.965202 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b97ca8d2-d976-4827-8918-c784d36d2b25 None None] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 10:41:43.984348 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=100 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 10:41:44.002351 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f46e9e2-a8cd-460a-ae46-186b5759ef06 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 10:41:44.003053 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f46e9e2-a8cd-460a-ae46-186b5759ef06 None admin] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:41:44.003338 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f46e9e2-a8cd-460a-ae46-186b5759ef06 None admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:41:44.003617 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f46e9e2-a8cd-460a-ae46-186b5759ef06 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:41:44.005476 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=100 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 10:41:44.016889 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f46e9e2-a8cd-460a-ae46-186b5759ef06 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 07 10:41:44.019008 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e55a02d-901a-407c-a6ef-6000140d21df None None] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 10:41:44.022019 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application [None req-1f46e9e2-a8cd-460a-ae46-186b5759ef06 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 07 10:41:44.022019 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 07 10:41:44.022019 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 07 10:41:44.022019 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 07 10:41:44.022019 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 07 10:41:44.022019 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 07 10:41:44.022019 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 07 10:41:44.022019 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 07 10:41:44.022019 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 07 10:41:44.022019 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 07 10:41:44.022019 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 07 10:41:44.022019 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 07 10:41:44.022019 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 07 10:41:44.022019 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 07 10:41:44.022019 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 07 10:41:44.022019 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 07 10:41:44.022950 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 07 10:41:44.022950 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 07 10:41:44.022950 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 07 10:41:44.022950 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 07 10:41:44.022950 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 07 10:41:44.022950 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 07 10:41:44.022950 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 07 10:41:44.022950 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 07 10:41:44.022950 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 07 10:41:44.022950 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self.get_or_create( Aug 07 10:41:44.022950 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 07 10:41:44.022950 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application with Lock( Aug 07 10:41:44.022950 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 07 10:41:44.022950 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self._enter() Aug 07 10:41:44.022950 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 07 10:41:44.022950 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 07 10:41:44.024214 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 07 10:41:44.024214 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self.creator() Aug 07 10:41:44.024214 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 07 10:41:44.024214 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application created_value = creator( Aug 07 10:41:44.024214 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 07 10:41:44.024214 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 07 10:41:44.024214 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 07 10:41:44.024214 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 07 10:41:44.024214 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 07 10:41:44.024214 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 07 10:41:44.024214 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 07 10:41:44.024214 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application Aug 07 10:41:44.028338 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 212/410] 10.222.0.25 () {56 vars in 1114 bytes} [Sun Aug 7 10:41:43 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 07 10:41:44.029561 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92af2d43-3df8-41d3-88b1-f5dd9d44413d None None] REQUEST_METHOD: `POST` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:41:44.029858 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92af2d43-3df8-41d3-88b1-f5dd9d44413d None None] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:41:44.030142 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92af2d43-3df8-41d3-88b1-f5dd9d44413d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:41:44.049070 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.auth.core [None req-92af2d43-3df8-41d3-88b1-f5dd9d44413d None None] MFA Rules not processed for user `bac4e6b76fe64217ac8fa44e71cffe6a`. Rule list: `[]` (Enabled: `True`). {{(pid=80144) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 10:41:44.050268 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.fernet_utils [None req-92af2d43-3df8-41d3-88b1-f5dd9d44413d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80144) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 10:41:44.054102 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b04ff48-bfeb-46ac-bbc8-339ef59d18fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 10:41:44.054750 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b04ff48-bfeb-46ac-bbc8-339ef59d18fb None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:41:44.055099 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b04ff48-bfeb-46ac-bbc8-339ef59d18fb None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:41:44.055374 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b04ff48-bfeb-46ac-bbc8-339ef59d18fb None admin] PATH_INFO: `/v3/projects/d5ac47e3b09f43d19a23a0f457fe9e23` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:41:44.056500 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b04ff48-bfeb-46ac-bbc8-339ef59d18fb None admin] RBAC: Authorizing `identity:get_project(project_id=d5ac47e3b09f43d19a23a0f457fe9e23)` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 10:41:44.058740 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b04ff48-bfeb-46ac-bbc8-339ef59d18fb None admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 10:41:44.060734 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 199/411] 10.222.0.25 () {56 vars in 1186 bytes} [Sun Aug 7 10:41:43 2022] GET /identity/v3/projects/d5ac47e3b09f43d19a23a0f457fe9e23 => generated 344 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 10:41:44.061515 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b04ff48-bfeb-46ac-bbc8-339ef59d18fb None admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 10:41:44.130745 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 213/412] 10.222.0.25 () {58 vars in 1024 bytes} [Sun Aug 7 10:41:44 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 07 10:41:44.131655 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92af2d43-3df8-41d3-88b1-f5dd9d44413d None None] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 10:41:44.200899 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b04ff48-bfeb-46ac-bbc8-339ef59d18fb None admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 10:41:44.241857 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bca92e9e-db21-457f-b446-28cf75f03392 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 10:41:44.242475 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bca92e9e-db21-457f-b446-28cf75f03392 None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:41:44.242757 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bca92e9e-db21-457f-b446-28cf75f03392 None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:41:44.243085 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bca92e9e-db21-457f-b446-28cf75f03392 None admin] PATH_INFO: `/v3/roles` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:41:44.243756 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bca92e9e-db21-457f-b446-28cf75f03392 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80143) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 10:41:44.244059 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bca92e9e-db21-457f-b446-28cf75f03392 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 10:41:44.246300 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bca92e9e-db21-457f-b446-28cf75f03392 None admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 10:41:44.252175 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=100 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 10:41:44.256295 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 200/413] 10.222.0.25 () {56 vars in 1111 bytes} [Sun Aug 7 10:41:44 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 10:41:44.256879 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bca92e9e-db21-457f-b446-28cf75f03392 None admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 10:41:44.268432 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92af2d43-3df8-41d3-88b1-f5dd9d44413d None None] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 10:41:44.303426 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=100 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 10:41:44.319191 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf157c19-8ac3-42e9-93e7-b74d54dbf344 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 10:41:44.320681 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf157c19-8ac3-42e9-93e7-b74d54dbf344 None admin] REQUEST_METHOD: `PUT` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:41:44.321147 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf157c19-8ac3-42e9-93e7-b74d54dbf344 None admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:41:44.321631 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf157c19-8ac3-42e9-93e7-b74d54dbf344 None admin] PATH_INFO: `/v3/projects/d5ac47e3b09f43d19a23a0f457fe9e23/groups/f827aa44240646458903fb806aa670c7/roles/81b1f1bcee2a473088fbbd327c1c5b23` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:41:44.324386 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf157c19-8ac3-42e9-93e7-b74d54dbf344 None admin] RBAC: Authorizing `identity:create_grant(project_id=d5ac47e3b09f43d19a23a0f457fe9e23, group_id=f827aa44240646458903fb806aa670c7, role_id=81b1f1bcee2a473088fbbd327c1c5b23)` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 10:41:44.326251 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf157c19-8ac3-42e9-93e7-b74d54dbf344 None admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 10:41:44.350977 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 214/414] 10.222.0.25 () {58 vars in 1442 bytes} [Sun Aug 7 10:41:44 2022] PUT /identity/v3/projects/d5ac47e3b09f43d19a23a0f457fe9e23/groups/f827aa44240646458903fb806aa670c7/roles/81b1f1bcee2a473088fbbd327c1c5b23 => generated 0 bytes in 212 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 07 10:41:44.352877 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf157c19-8ac3-42e9-93e7-b74d54dbf344 None admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 10:41:44.420246 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bca92e9e-db21-457f-b446-28cf75f03392 None admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 10:41:44.482797 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf157c19-8ac3-42e9-93e7-b74d54dbf344 None admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 10:41:44.484416 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-294aa54e-e0c8-43f0-9713-367e8dd3c3dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 10:41:44.485079 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-294aa54e-e0c8-43f0-9713-367e8dd3c3dd None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:41:44.485363 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-294aa54e-e0c8-43f0-9713-367e8dd3c3dd None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:41:44.485639 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-294aa54e-e0c8-43f0-9713-367e8dd3c3dd None admin] PATH_INFO: `/v3/roles/member` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:41:44.497853 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-294aa54e-e0c8-43f0-9713-367e8dd3c3dd None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 07 10:41:44.502859 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application [None req-294aa54e-e0c8-43f0-9713-367e8dd3c3dd None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 07 10:41:44.502859 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 07 10:41:44.502859 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 07 10:41:44.502859 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 07 10:41:44.502859 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 07 10:41:44.502859 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 07 10:41:44.502859 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 07 10:41:44.502859 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 07 10:41:44.502859 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 07 10:41:44.502859 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 07 10:41:44.502859 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 07 10:41:44.502859 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 07 10:41:44.502859 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 07 10:41:44.502859 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 07 10:41:44.502859 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 07 10:41:44.502859 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 07 10:41:44.503869 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 07 10:41:44.503869 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 07 10:41:44.503869 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 07 10:41:44.503869 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 07 10:41:44.503869 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 07 10:41:44.503869 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 07 10:41:44.503869 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 07 10:41:44.503869 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 07 10:41:44.503869 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 07 10:41:44.503869 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.get_or_create( Aug 07 10:41:44.503869 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 07 10:41:44.503869 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application with Lock( Aug 07 10:41:44.503869 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 07 10:41:44.503869 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self._enter() Aug 07 10:41:44.503869 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 07 10:41:44.503869 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 07 10:41:44.504644 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 07 10:41:44.504644 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.creator() Aug 07 10:41:44.504644 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 07 10:41:44.504644 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application created_value = creator( Aug 07 10:41:44.504644 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 07 10:41:44.504644 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 07 10:41:44.504644 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 07 10:41:44.504644 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 07 10:41:44.504644 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 07 10:41:44.504644 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 07 10:41:44.504644 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 07 10:41:44.504644 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application Aug 07 10:41:44.504644 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 201/415] 10.222.0.25 () {56 vars in 1099 bytes} [Sun Aug 7 10:41:44 2022] GET /identity/v3/roles/member => generated 84 bytes in 247 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 07 10:41:44.512190 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-294aa54e-e0c8-43f0-9713-367e8dd3c3dd None admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 10:41:44.532808 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7416c1b1-2342-4e62-8898-77bcb4016c82 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 10:41:44.536655 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7416c1b1-2342-4e62-8898-77bcb4016c82 None admin] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:41:44.536929 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7416c1b1-2342-4e62-8898-77bcb4016c82 None admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:41:44.537205 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7416c1b1-2342-4e62-8898-77bcb4016c82 None admin] PATH_INFO: `/v3/groups/f26eb533e57a456d8f95265e204d21b5` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:41:44.539578 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7416c1b1-2342-4e62-8898-77bcb4016c82 None admin] RBAC: Authorizing `identity:get_group(group_id=f26eb533e57a456d8f95265e204d21b5)` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 10:41:44.543410 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7416c1b1-2342-4e62-8898-77bcb4016c82 None admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 10:41:44.547100 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 215/416] 10.222.0.25 () {56 vars in 1180 bytes} [Sun Aug 7 10:41:44 2022] GET /identity/v3/groups/f26eb533e57a456d8f95265e204d21b5 => generated 226 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 10:41:44.553156 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7416c1b1-2342-4e62-8898-77bcb4016c82 None admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 10:41:44.616900 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=100 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 10:41:44.642598 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 10:41:44.692030 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-294aa54e-e0c8-43f0-9713-367e8dd3c3dd None admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 10:41:44.712768 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7416c1b1-2342-4e62-8898-77bcb4016c82 None admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 10:41:44.753490 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87de0823-dfd5-43a9-bd5c-a09ab3067df7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 10:41:44.754138 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87de0823-dfd5-43a9-bd5c-a09ab3067df7 None admin] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:41:44.754426 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87de0823-dfd5-43a9-bd5c-a09ab3067df7 None admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:41:44.754712 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87de0823-dfd5-43a9-bd5c-a09ab3067df7 None admin] PATH_INFO: `/v3/projects/d88586f68b734382abb9836b290aeb54` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:41:44.756487 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87de0823-dfd5-43a9-bd5c-a09ab3067df7 None admin] RBAC: Authorizing `identity:get_project(project_id=d88586f68b734382abb9836b290aeb54)` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 10:41:44.757916 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87de0823-dfd5-43a9-bd5c-a09ab3067df7 None admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 10:41:44.761062 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 216/417] 10.222.0.25 () {56 vars in 1186 bytes} [Sun Aug 7 10:41:44 2022] GET /identity/v3/projects/d88586f68b734382abb9836b290aeb54 => generated 302 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 10:41:44.768013 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87de0823-dfd5-43a9-bd5c-a09ab3067df7 None admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 10:41:44.783853 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9872be6-47fd-4312-af18-3e0e211aea79 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 10:41:44.784475 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9872be6-47fd-4312-af18-3e0e211aea79 None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:41:44.784752 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9872be6-47fd-4312-af18-3e0e211aea79 None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:41:44.785026 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9872be6-47fd-4312-af18-3e0e211aea79 None admin] PATH_INFO: `/v3/roles` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:41:44.785701 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9872be6-47fd-4312-af18-3e0e211aea79 None admin] RBAC: Adding query filter params (name=member) {{(pid=80143) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 10:41:44.786003 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9872be6-47fd-4312-af18-3e0e211aea79 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 10:41:44.787479 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9872be6-47fd-4312-af18-3e0e211aea79 None admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 10:41:44.803904 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 202/418] 10.222.0.25 () {56 vars in 1101 bytes} [Sun Aug 7 10:41:44 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 292 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 10:41:44.814948 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9872be6-47fd-4312-af18-3e0e211aea79 None admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 10:41:44.966522 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87de0823-dfd5-43a9-bd5c-a09ab3067df7 None admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 10:41:45.002662 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9872be6-47fd-4312-af18-3e0e211aea79 None admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 10:41:45.010960 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=100 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 10:41:45.029216 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=100 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 10:41:45.041028 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea70b892-bb59-4e62-8d92-e07f3ace452a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 10:41:45.041894 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea70b892-bb59-4e62-8d92-e07f3ace452a None admin] REQUEST_METHOD: `PUT` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:41:45.042311 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea70b892-bb59-4e62-8d92-e07f3ace452a None admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:41:45.042600 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea70b892-bb59-4e62-8d92-e07f3ace452a None admin] PATH_INFO: `/v3/projects/d88586f68b734382abb9836b290aeb54/groups/f26eb533e57a456d8f95265e204d21b5/roles/1512d531ebe248ea8e2d7d56d754b230` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:41:45.045293 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea70b892-bb59-4e62-8d92-e07f3ace452a None admin] RBAC: Authorizing `identity:create_grant(project_id=d88586f68b734382abb9836b290aeb54, group_id=f26eb533e57a456d8f95265e204d21b5, role_id=1512d531ebe248ea8e2d7d56d754b230)` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 10:41:45.046725 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea70b892-bb59-4e62-8d92-e07f3ace452a None admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 10:41:45.048114 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15262296-064e-4802-995b-996604fc119c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 10:41:45.048735 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15262296-064e-4802-995b-996604fc119c None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:41:45.049023 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15262296-064e-4802-995b-996604fc119c None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:41:45.049301 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15262296-064e-4802-995b-996604fc119c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:41:45.075289 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 217/419] 10.222.0.25 () {58 vars in 1442 bytes} [Sun Aug 7 10:41:44 2022] PUT /identity/v3/projects/d88586f68b734382abb9836b290aeb54/groups/f26eb533e57a456d8f95265e204d21b5/roles/1512d531ebe248ea8e2d7d56d754b230 => generated 0 bytes in 307 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 07 10:41:45.173219 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15262296-064e-4802-995b-996604fc119c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 10:41:45.175077 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15262296-064e-4802-995b-996604fc119c None admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 10:41:45.251867 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-596347be-3b15-4a08-9ea7-afae5af57672 None None] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:41:45.252170 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-596347be-3b15-4a08-9ea7-afae5af57672 None None] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:41:45.252357 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-596347be-3b15-4a08-9ea7-afae5af57672 None None] PATH_INFO: `/` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:41:45.255041 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 218/420] 10.222.0.25 () {54 vars in 927 bytes} [Sun Aug 7 10:41:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 07 10:41:45.261210 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f53bc34-66f4-408b-8338-61aa1d878adc None None] REQUEST_METHOD: `POST` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:41:45.261704 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f53bc34-66f4-408b-8338-61aa1d878adc None None] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:41:45.262133 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f53bc34-66f4-408b-8338-61aa1d878adc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:41:45.291329 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.auth.core [None req-6f53bc34-66f4-408b-8338-61aa1d878adc None None] MFA Rules not processed for user `bac4e6b76fe64217ac8fa44e71cffe6a`. Rule list: `[]` (Enabled: `True`). {{(pid=80144) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 10:41:45.293939 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.fernet_utils [None req-6f53bc34-66f4-408b-8338-61aa1d878adc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80144) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 10:41:45.296828 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 203/421] 10.222.0.25 () {58 vars in 1282 bytes} [Sun Aug 7 10:41:44 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 487 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 07 10:41:45.302472 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15262296-064e-4802-995b-996604fc119c None admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 10:41:45.326797 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=100 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 10:41:45.405120 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 219/422] 10.222.0.25 () {58 vars in 1024 bytes} [Sun Aug 7 10:41:45 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 145 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 07 10:41:45.439028 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15262296-064e-4802-995b-996604fc119c None admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 10:41:45.453182 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f53bc34-66f4-408b-8338-61aa1d878adc None None] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 10:41:45.507660 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b37468e-2ceb-487b-81b9-e2832d043339 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 10:41:45.508487 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b37468e-2ceb-487b-81b9-e2832d043339 None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:41:45.508620 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b37468e-2ceb-487b-81b9-e2832d043339 None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:41:45.508841 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b37468e-2ceb-487b-81b9-e2832d043339 None admin] PATH_INFO: `/v3/projects/90718391b09e493bb296767dc8a3c0ce` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:41:45.515259 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b37468e-2ceb-487b-81b9-e2832d043339 None admin] RBAC: Authorizing `identity:get_project(project_id=90718391b09e493bb296767dc8a3c0ce)` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 10:41:45.516612 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b37468e-2ceb-487b-81b9-e2832d043339 None admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 10:41:45.518562 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 204/423] 10.222.0.25 () {56 vars in 1186 bytes} [Sun Aug 7 10:41:45 2022] GET /identity/v3/projects/90718391b09e493bb296767dc8a3c0ce => generated 298 bytes in 216 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 07 10:41:45.523771 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b37468e-2ceb-487b-81b9-e2832d043339 None admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 10:41:45.579201 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f53bc34-66f4-408b-8338-61aa1d878adc None None] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 10:41:45.595079 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 10:41:45.627328 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-982ec017-0e41-4639-9472-4fb183e9a218 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 10:41:45.628082 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-982ec017-0e41-4639-9472-4fb183e9a218 None admin] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:41:45.630945 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-982ec017-0e41-4639-9472-4fb183e9a218 None admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:41:45.630945 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-982ec017-0e41-4639-9472-4fb183e9a218 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:41:45.645614 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-982ec017-0e41-4639-9472-4fb183e9a218 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 07 10:41:45.650452 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application [None req-982ec017-0e41-4639-9472-4fb183e9a218 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 07 10:41:45.650452 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 07 10:41:45.650452 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 07 10:41:45.650452 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 07 10:41:45.650452 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 07 10:41:45.650452 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 07 10:41:45.650452 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 07 10:41:45.650452 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 07 10:41:45.650452 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 07 10:41:45.650452 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 07 10:41:45.650452 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 07 10:41:45.650452 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 07 10:41:45.650452 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 07 10:41:45.650452 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 07 10:41:45.650452 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 07 10:41:45.650452 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 07 10:41:45.652148 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 07 10:41:45.652148 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 07 10:41:45.652148 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 07 10:41:45.652148 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 07 10:41:45.652148 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 07 10:41:45.652148 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 07 10:41:45.652148 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 07 10:41:45.652148 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 07 10:41:45.652148 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 07 10:41:45.652148 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self.get_or_create( Aug 07 10:41:45.652148 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 07 10:41:45.652148 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application with Lock( Aug 07 10:41:45.652148 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 07 10:41:45.652148 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self._enter() Aug 07 10:41:45.652148 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 07 10:41:45.652148 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 07 10:41:45.653314 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 07 10:41:45.653314 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self.creator() Aug 07 10:41:45.653314 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 07 10:41:45.653314 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application created_value = creator( Aug 07 10:41:45.653314 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 07 10:41:45.653314 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 07 10:41:45.653314 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 07 10:41:45.653314 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 07 10:41:45.653314 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 07 10:41:45.653314 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 07 10:41:45.653314 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 07 10:41:45.653314 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application Aug 07 10:41:45.653314 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 220/424] 10.222.0.25 () {56 vars in 1114 bytes} [Sun Aug 7 10:41:45 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 07 10:41:45.660297 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-982ec017-0e41-4639-9472-4fb183e9a218 None admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 10:41:45.747658 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=100 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 10:41:45.764091 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b37468e-2ceb-487b-81b9-e2832d043339 None admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 10:41:45.829337 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-314fb232-9b58-4e36-a414-de8f3cb59b24 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 10:41:45.830148 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-314fb232-9b58-4e36-a414-de8f3cb59b24 None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:41:45.830434 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-314fb232-9b58-4e36-a414-de8f3cb59b24 None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:41:45.830707 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-314fb232-9b58-4e36-a414-de8f3cb59b24 None admin] PATH_INFO: `/v3/groups/f26eb533e57a456d8f95265e204d21b5` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:41:45.835754 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-314fb232-9b58-4e36-a414-de8f3cb59b24 None admin] RBAC: Authorizing `identity:get_group(group_id=f26eb533e57a456d8f95265e204d21b5)` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 10:41:45.835939 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-982ec017-0e41-4639-9472-4fb183e9a218 None admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 10:41:45.837443 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-314fb232-9b58-4e36-a414-de8f3cb59b24 None admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 10:41:45.840037 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 205/425] 10.222.0.25 () {56 vars in 1180 bytes} [Sun Aug 7 10:41:45 2022] GET /identity/v3/groups/f26eb533e57a456d8f95265e204d21b5 => generated 226 bytes in 316 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 07 10:41:45.846482 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-314fb232-9b58-4e36-a414-de8f3cb59b24 None admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 10:41:45.880254 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9305f589-dc8b-481d-bb49-ae254965b017 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 10:41:45.881014 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9305f589-dc8b-481d-bb49-ae254965b017 None admin] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:41:45.881302 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9305f589-dc8b-481d-bb49-ae254965b017 None admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:41:45.882513 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9305f589-dc8b-481d-bb49-ae254965b017 None admin] PATH_INFO: `/v3/roles` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:41:45.884016 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9305f589-dc8b-481d-bb49-ae254965b017 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80144) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 10:41:45.884335 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9305f589-dc8b-481d-bb49-ae254965b017 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 10:41:45.888242 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9305f589-dc8b-481d-bb49-ae254965b017 None admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 10:41:45.904816 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 221/426] 10.222.0.25 () {56 vars in 1111 bytes} [Sun Aug 7 10:41:45 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 245 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 10:41:45.910271 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9305f589-dc8b-481d-bb49-ae254965b017 None admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 10:41:45.943283 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=100 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 10:41:46.008660 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-314fb232-9b58-4e36-a414-de8f3cb59b24 None admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 10:41:46.029865 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9305f589-dc8b-481d-bb49-ae254965b017 None admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 10:41:46.055070 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c51661b3-51f8-4d1c-a9f0-caa218009ea9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 10:41:46.055682 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c51661b3-51f8-4d1c-a9f0-caa218009ea9 None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:41:46.055962 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c51661b3-51f8-4d1c-a9f0-caa218009ea9 None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:41:46.056243 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c51661b3-51f8-4d1c-a9f0-caa218009ea9 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:41:46.056970 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c51661b3-51f8-4d1c-a9f0-caa218009ea9 None admin] RBAC: Adding query filter params (group.id=f26eb533e57a456d8f95265e204d21b5, role.id=b98dfb3b9fee4644af2887a3ac3fd2e6, scope.project.id=90718391b09e493bb296767dc8a3c0ce) {{(pid=80143) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 10:41:46.057453 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c51661b3-51f8-4d1c-a9f0-caa218009ea9 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 10:41:46.059865 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c51661b3-51f8-4d1c-a9f0-caa218009ea9 None admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 10:41:46.066362 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 206/427] 10.222.0.25 () {56 vars in 1376 bytes} [Sun Aug 7 10:41:45 2022] GET /identity/v3/role_assignments?group.id=f26eb533e57a456d8f95265e204d21b5&scope.project.id=90718391b09e493bb296767dc8a3c0ce&role.id=b98dfb3b9fee4644af2887a3ac3fd2e6 => generated 611 bytes in 220 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 10:41:46.067623 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a31e23f1-7bb0-4039-bc11-ab5ef479e3b8 None None] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:41:46.067906 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a31e23f1-7bb0-4039-bc11-ab5ef479e3b8 None None] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:41:46.068177 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a31e23f1-7bb0-4039-bc11-ab5ef479e3b8 None None] PATH_INFO: `/` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:41:46.069400 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 207/428] 10.222.0.25 () {54 vars in 927 bytes} [Sun Aug 7 10:41:46 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 07 10:41:46.070316 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64c191bc-b37f-4884-97f4-31898e27d3e7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 10:41:46.071145 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64c191bc-b37f-4884-97f4-31898e27d3e7 None admin] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:41:46.071596 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64c191bc-b37f-4884-97f4-31898e27d3e7 None admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:41:46.072240 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64c191bc-b37f-4884-97f4-31898e27d3e7 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:41:46.076091 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ace8e4ce-b490-49d6-acf3-5d4854949a6a None None] REQUEST_METHOD: `POST` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:41:46.076375 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ace8e4ce-b490-49d6-acf3-5d4854949a6a None None] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:41:46.076655 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ace8e4ce-b490-49d6-acf3-5d4854949a6a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:41:46.096426 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.auth.core [None req-ace8e4ce-b490-49d6-acf3-5d4854949a6a None None] MFA Rules not processed for user `bac4e6b76fe64217ac8fa44e71cffe6a`. Rule list: `[]` (Enabled: `True`). {{(pid=80143) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 10:41:46.097670 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-ace8e4ce-b490-49d6-acf3-5d4854949a6a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 10:41:46.117230 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=100 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 10:41:46.186716 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64c191bc-b37f-4884-97f4-31898e27d3e7 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 10:41:46.189313 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64c191bc-b37f-4884-97f4-31898e27d3e7 None admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 10:41:46.236043 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 208/429] 10.222.0.25 () {58 vars in 1024 bytes} [Sun Aug 7 10:41:46 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 161 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 07 10:41:46.257635 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=100 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 10:41:46.281689 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ace8e4ce-b490-49d6-acf3-5d4854949a6a None None] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 10:41:46.358048 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 222/430] 10.222.0.25 () {58 vars in 1282 bytes} [Sun Aug 7 10:41:45 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 448 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Aug 07 10:41:46.363649 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64c191bc-b37f-4884-97f4-31898e27d3e7 None admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 10:41:46.411378 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ace8e4ce-b490-49d6-acf3-5d4854949a6a None None] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 10:41:46.453341 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c17cd76a-ab78-45e1-8823-8637de31d3ca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 10:41:46.454123 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c17cd76a-ab78-45e1-8823-8637de31d3ca None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:41:46.454501 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c17cd76a-ab78-45e1-8823-8637de31d3ca None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:41:46.454869 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c17cd76a-ab78-45e1-8823-8637de31d3ca None admin] PATH_INFO: `/v3/roles/member` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:41:46.474129 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c17cd76a-ab78-45e1-8823-8637de31d3ca None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 07 10:41:46.479219 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application [None req-c17cd76a-ab78-45e1-8823-8637de31d3ca None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 07 10:41:46.479219 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 07 10:41:46.479219 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 07 10:41:46.479219 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 07 10:41:46.479219 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 07 10:41:46.479219 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 07 10:41:46.479219 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 07 10:41:46.479219 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 07 10:41:46.479219 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 07 10:41:46.479219 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 07 10:41:46.479219 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 07 10:41:46.479219 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 07 10:41:46.479219 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 07 10:41:46.479219 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 07 10:41:46.479219 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 07 10:41:46.479219 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 07 10:41:46.480205 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 07 10:41:46.480205 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 07 10:41:46.480205 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 07 10:41:46.480205 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 07 10:41:46.480205 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 07 10:41:46.480205 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 07 10:41:46.480205 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 07 10:41:46.480205 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 07 10:41:46.480205 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 07 10:41:46.480205 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.get_or_create( Aug 07 10:41:46.480205 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 07 10:41:46.480205 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application with Lock( Aug 07 10:41:46.480205 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 07 10:41:46.480205 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self._enter() Aug 07 10:41:46.480205 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 07 10:41:46.480205 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 07 10:41:46.481002 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 07 10:41:46.481002 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.creator() Aug 07 10:41:46.481002 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 07 10:41:46.481002 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application created_value = creator( Aug 07 10:41:46.481002 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 07 10:41:46.481002 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 07 10:41:46.481002 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 07 10:41:46.481002 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 07 10:41:46.481002 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 07 10:41:46.481002 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 07 10:41:46.481002 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 07 10:41:46.481002 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application Aug 07 10:41:46.487454 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 209/431] 10.222.0.25 () {56 vars in 1099 bytes} [Sun Aug 7 10:41:46 2022] GET /identity/v3/roles/member => generated 84 bytes in 206 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 07 10:41:46.488507 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c17cd76a-ab78-45e1-8823-8637de31d3ca None admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 10:41:46.492133 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64c191bc-b37f-4884-97f4-31898e27d3e7 None admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 10:41:46.507853 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=100 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 10:41:46.536717 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=100 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 10:41:46.545016 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0894429-a95d-4055-9da9-57077f32652a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 10:41:46.545996 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0894429-a95d-4055-9da9-57077f32652a None admin] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:41:46.546469 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0894429-a95d-4055-9da9-57077f32652a None admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:41:46.547089 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0894429-a95d-4055-9da9-57077f32652a None admin] PATH_INFO: `/v3/projects/90718391b09e493bb296767dc8a3c0ce` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:41:46.548491 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0894429-a95d-4055-9da9-57077f32652a None admin] RBAC: Authorizing `identity:get_project(project_id=90718391b09e493bb296767dc8a3c0ce)` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 10:41:46.550136 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0894429-a95d-4055-9da9-57077f32652a None admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 10:41:46.552789 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 223/432] 10.222.0.25 () {56 vars in 1186 bytes} [Sun Aug 7 10:41:46 2022] GET /identity/v3/projects/90718391b09e493bb296767dc8a3c0ce => generated 298 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 07 10:41:46.559009 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0894429-a95d-4055-9da9-57077f32652a None admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 10:41:46.622269 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c17cd76a-ab78-45e1-8823-8637de31d3ca None admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 10:41:46.659993 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-752997a3-8811-4f80-b1c6-d30d04209419 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 10:41:46.660656 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-752997a3-8811-4f80-b1c6-d30d04209419 None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:41:46.661172 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-752997a3-8811-4f80-b1c6-d30d04209419 None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:41:46.661634 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-752997a3-8811-4f80-b1c6-d30d04209419 None admin] PATH_INFO: `/v3/roles` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:41:46.662797 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-752997a3-8811-4f80-b1c6-d30d04209419 None admin] RBAC: Adding query filter params (name=member) {{(pid=80143) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 10:41:46.663232 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-752997a3-8811-4f80-b1c6-d30d04209419 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 10:41:46.664720 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-752997a3-8811-4f80-b1c6-d30d04209419 None admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 10:41:46.670776 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0894429-a95d-4055-9da9-57077f32652a None admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 10:41:46.674077 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 210/433] 10.222.0.25 () {56 vars in 1101 bytes} [Sun Aug 7 10:41:46 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 10:41:46.684059 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-752997a3-8811-4f80-b1c6-d30d04209419 None admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 10:41:46.707682 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-432018a2-5a57-4f84-bc52-1446d053d946 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 10:41:46.708953 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-432018a2-5a57-4f84-bc52-1446d053d946 None admin] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:41:46.709383 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-432018a2-5a57-4f84-bc52-1446d053d946 None admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:41:46.709793 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-432018a2-5a57-4f84-bc52-1446d053d946 None admin] PATH_INFO: `/v3/groups/f26eb533e57a456d8f95265e204d21b5` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:41:46.711082 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-432018a2-5a57-4f84-bc52-1446d053d946 None admin] RBAC: Authorizing `identity:get_group(group_id=f26eb533e57a456d8f95265e204d21b5)` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 10:41:46.712669 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-432018a2-5a57-4f84-bc52-1446d053d946 None admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 10:41:46.714416 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 224/434] 10.222.0.25 () {56 vars in 1180 bytes} [Sun Aug 7 10:41:46 2022] GET /identity/v3/groups/f26eb533e57a456d8f95265e204d21b5 => generated 226 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 07 10:41:46.720229 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-432018a2-5a57-4f84-bc52-1446d053d946 None admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 10:41:46.775608 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=100 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 10:41:46.800040 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=100 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 10:41:46.810670 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-752997a3-8811-4f80-b1c6-d30d04209419 None admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 10:41:46.843010 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-432018a2-5a57-4f84-bc52-1446d053d946 None admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 10:41:46.849870 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52e9d34a-6c8b-41b0-b29d-3b782f913f46 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 10:41:46.850668 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52e9d34a-6c8b-41b0-b29d-3b782f913f46 None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:41:46.851118 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52e9d34a-6c8b-41b0-b29d-3b782f913f46 None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:41:46.851604 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52e9d34a-6c8b-41b0-b29d-3b782f913f46 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:41:46.888609 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0844dfc9-c689-4d2f-9470-3f4573d7e47d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 10:41:46.889786 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0844dfc9-c689-4d2f-9470-3f4573d7e47d None admin] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:41:46.890176 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0844dfc9-c689-4d2f-9470-3f4573d7e47d None admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:41:46.890842 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0844dfc9-c689-4d2f-9470-3f4573d7e47d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:41:46.891705 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0844dfc9-c689-4d2f-9470-3f4573d7e47d None admin] RBAC: Adding query filter params (group.id=f26eb533e57a456d8f95265e204d21b5, role.id=1512d531ebe248ea8e2d7d56d754b230, scope.project.id=90718391b09e493bb296767dc8a3c0ce) {{(pid=80144) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 10:41:46.892122 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0844dfc9-c689-4d2f-9470-3f4573d7e47d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 10:41:46.893851 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0844dfc9-c689-4d2f-9470-3f4573d7e47d None admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 10:41:46.902227 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 225/435] 10.222.0.25 () {56 vars in 1376 bytes} [Sun Aug 7 10:41:46 2022] GET /identity/v3/role_assignments?group.id=f26eb533e57a456d8f95265e204d21b5&scope.project.id=90718391b09e493bb296767dc8a3c0ce&role.id=1512d531ebe248ea8e2d7d56d754b230 => generated 611 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 07 10:41:46.906944 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e43cce52-7a50-4bfe-ba4c-45ed51c8b16e None None] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:41:46.906944 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e43cce52-7a50-4bfe-ba4c-45ed51c8b16e None None] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:41:46.906944 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e43cce52-7a50-4bfe-ba4c-45ed51c8b16e None None] PATH_INFO: `/` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:41:46.906944 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 226/436] 10.222.0.25 () {54 vars in 927 bytes} [Sun Aug 7 10:41:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 07 10:41:46.911826 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da5546cb-d18a-4ec4-a22e-c95bc526acab None None] REQUEST_METHOD: `POST` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:41:46.912163 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da5546cb-d18a-4ec4-a22e-c95bc526acab None None] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:41:46.912381 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da5546cb-d18a-4ec4-a22e-c95bc526acab None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:41:46.938555 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.auth.core [None req-da5546cb-d18a-4ec4-a22e-c95bc526acab None None] MFA Rules not processed for user `bac4e6b76fe64217ac8fa44e71cffe6a`. Rule list: `[]` (Enabled: `True`). {{(pid=80144) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 10:41:46.940011 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.fernet_utils [None req-da5546cb-d18a-4ec4-a22e-c95bc526acab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80144) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 10:41:46.943507 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52e9d34a-6c8b-41b0-b29d-3b782f913f46 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 10:41:46.944986 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52e9d34a-6c8b-41b0-b29d-3b782f913f46 None admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 10:41:47.042685 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 227/437] 10.222.0.25 () {58 vars in 1024 bytes} [Sun Aug 7 10:41:46 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 07 10:41:47.051379 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=100 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 10:41:47.076133 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da5546cb-d18a-4ec4-a22e-c95bc526acab None None] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 10:41:47.096099 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 211/438] 10.222.0.25 () {58 vars in 1282 bytes} [Sun Aug 7 10:41:46 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 408 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 07 10:41:47.097358 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52e9d34a-6c8b-41b0-b29d-3b782f913f46 None admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 10:41:47.156720 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=100 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 10:41:47.211713 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da5546cb-d18a-4ec4-a22e-c95bc526acab None None] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 10:41:47.232741 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52e9d34a-6c8b-41b0-b29d-3b782f913f46 None admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 10:41:47.265175 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b6c083c-9815-4c8f-a037-beb65b23e4ff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 10:41:47.265933 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b6c083c-9815-4c8f-a037-beb65b23e4ff None admin] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:41:47.266212 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b6c083c-9815-4c8f-a037-beb65b23e4ff None admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:41:47.266487 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b6c083c-9815-4c8f-a037-beb65b23e4ff None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:41:47.280629 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0adca770-2a3e-4240-9c20-b9d1f72b9fa9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 10:41:47.281536 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0adca770-2a3e-4240-9c20-b9d1f72b9fa9 None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:41:47.281823 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0adca770-2a3e-4240-9c20-b9d1f72b9fa9 None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:41:47.282125 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0adca770-2a3e-4240-9c20-b9d1f72b9fa9 None admin] PATH_INFO: `/v3/projects/d88586f68b734382abb9836b290aeb54` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:41:47.284087 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0adca770-2a3e-4240-9c20-b9d1f72b9fa9 None admin] RBAC: Authorizing `identity:get_project(project_id=d88586f68b734382abb9836b290aeb54)` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 10:41:47.285548 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0adca770-2a3e-4240-9c20-b9d1f72b9fa9 None admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 10:41:47.287220 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 212/439] 10.222.0.25 () {56 vars in 1186 bytes} [Sun Aug 7 10:41:47 2022] GET /identity/v3/projects/d88586f68b734382abb9836b290aeb54 => generated 302 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 10:41:47.288837 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b6c083c-9815-4c8f-a037-beb65b23e4ff None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 07 10:41:47.295466 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0adca770-2a3e-4240-9c20-b9d1f72b9fa9 None admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 10:41:47.295609 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application [None req-6b6c083c-9815-4c8f-a037-beb65b23e4ff None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 07 10:41:47.295609 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 07 10:41:47.295609 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 07 10:41:47.295609 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 07 10:41:47.295609 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 07 10:41:47.295609 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 07 10:41:47.295609 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 07 10:41:47.295609 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 07 10:41:47.295609 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 07 10:41:47.295609 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 07 10:41:47.295609 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 07 10:41:47.295609 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 07 10:41:47.295609 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 07 10:41:47.295609 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 07 10:41:47.295609 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 07 10:41:47.295609 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 07 10:41:47.296428 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 07 10:41:47.296428 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 07 10:41:47.296428 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 07 10:41:47.296428 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 07 10:41:47.296428 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 07 10:41:47.296428 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 07 10:41:47.296428 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 07 10:41:47.296428 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 07 10:41:47.296428 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 07 10:41:47.296428 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self.get_or_create( Aug 07 10:41:47.296428 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 07 10:41:47.296428 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application with Lock( Aug 07 10:41:47.296428 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 07 10:41:47.296428 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self._enter() Aug 07 10:41:47.296428 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 07 10:41:47.296428 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 07 10:41:47.297195 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 07 10:41:47.297195 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self.creator() Aug 07 10:41:47.297195 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 07 10:41:47.297195 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application created_value = creator( Aug 07 10:41:47.297195 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 07 10:41:47.297195 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 07 10:41:47.297195 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 07 10:41:47.297195 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 07 10:41:47.297195 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 07 10:41:47.297195 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 07 10:41:47.297195 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 07 10:41:47.297195 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application Aug 07 10:41:47.297195 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 228/440] 10.222.0.25 () {56 vars in 1096 bytes} [Sun Aug 7 10:41:47 2022] GET /identity/v3/roles/admin => generated 83 bytes in 220 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 07 10:41:47.304622 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b6c083c-9815-4c8f-a037-beb65b23e4ff None admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 10:41:47.343250 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=100 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 10:41:47.415650 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0adca770-2a3e-4240-9c20-b9d1f72b9fa9 None admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 10:41:47.436271 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b6c083c-9815-4c8f-a037-beb65b23e4ff None admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 10:41:47.453364 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a61c6c5-abce-4182-a2da-28ff220fad5f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 10:41:47.454104 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a61c6c5-abce-4182-a2da-28ff220fad5f None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:41:47.454454 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a61c6c5-abce-4182-a2da-28ff220fad5f None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:41:47.454740 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a61c6c5-abce-4182-a2da-28ff220fad5f None admin] PATH_INFO: `/v3/groups/f26eb533e57a456d8f95265e204d21b5` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:41:47.455970 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a61c6c5-abce-4182-a2da-28ff220fad5f None admin] RBAC: Authorizing `identity:get_group(group_id=f26eb533e57a456d8f95265e204d21b5)` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 10:41:47.457475 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a61c6c5-abce-4182-a2da-28ff220fad5f None admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 10:41:47.459082 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 213/441] 10.222.0.25 () {56 vars in 1180 bytes} [Sun Aug 7 10:41:47 2022] GET /identity/v3/groups/f26eb533e57a456d8f95265e204d21b5 => generated 226 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 10:41:47.463326 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=100 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 10:41:47.465002 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a61c6c5-abce-4182-a2da-28ff220fad5f None admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 10:41:47.477457 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acf109fe-061c-4467-b876-3ec7b5563b69 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 10:41:47.478088 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acf109fe-061c-4467-b876-3ec7b5563b69 None admin] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:41:47.478369 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acf109fe-061c-4467-b876-3ec7b5563b69 None admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:41:47.478621 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acf109fe-061c-4467-b876-3ec7b5563b69 None admin] PATH_INFO: `/v3/roles` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:41:47.479880 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acf109fe-061c-4467-b876-3ec7b5563b69 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80144) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 10:41:47.480198 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acf109fe-061c-4467-b876-3ec7b5563b69 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 10:41:47.481606 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acf109fe-061c-4467-b876-3ec7b5563b69 None admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 10:41:47.491846 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 229/442] 10.222.0.25 () {56 vars in 1099 bytes} [Sun Aug 7 10:41:47 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 07 10:41:47.497092 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acf109fe-061c-4467-b876-3ec7b5563b69 None admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 10:41:47.568978 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a61c6c5-abce-4182-a2da-28ff220fad5f None admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 10:41:47.580239 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=100 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 10:41:47.599180 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acf109fe-061c-4467-b876-3ec7b5563b69 None admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 10:41:47.605304 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f49a15f0-c9e2-41f3-9042-9231fcd1859f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 10:41:47.605907 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f49a15f0-c9e2-41f3-9042-9231fcd1859f None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:41:47.606183 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f49a15f0-c9e2-41f3-9042-9231fcd1859f None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:41:47.606477 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f49a15f0-c9e2-41f3-9042-9231fcd1859f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:41:47.607221 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f49a15f0-c9e2-41f3-9042-9231fcd1859f None admin] RBAC: Adding query filter params (group.id=f26eb533e57a456d8f95265e204d21b5, role.id=b98dfb3b9fee4644af2887a3ac3fd2e6, scope.project.id=d88586f68b734382abb9836b290aeb54) {{(pid=80143) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 10:41:47.607525 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f49a15f0-c9e2-41f3-9042-9231fcd1859f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 10:41:47.608936 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f49a15f0-c9e2-41f3-9042-9231fcd1859f None admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 10:41:47.614718 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 214/443] 10.222.0.25 () {56 vars in 1376 bytes} [Sun Aug 7 10:41:47 2022] GET /identity/v3/role_assignments?group.id=f26eb533e57a456d8f95265e204d21b5&scope.project.id=d88586f68b734382abb9836b290aeb54&role.id=b98dfb3b9fee4644af2887a3ac3fd2e6 => generated 611 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 07 10:41:47.614718 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d3312cd-fa2e-4bf7-a4bc-3bc972b5e26b None None] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:41:47.614718 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d3312cd-fa2e-4bf7-a4bc-3bc972b5e26b None None] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:41:47.614718 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d3312cd-fa2e-4bf7-a4bc-3bc972b5e26b None None] PATH_INFO: `/` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:41:47.614718 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 215/444] 10.222.0.25 () {54 vars in 927 bytes} [Sun Aug 7 10:41:47 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 07 10:41:47.623548 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-461d1de3-5e6e-4b9b-b0f0-438387da7a40 None None] REQUEST_METHOD: `POST` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:41:47.623843 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-461d1de3-5e6e-4b9b-b0f0-438387da7a40 None None] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:41:47.624103 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-461d1de3-5e6e-4b9b-b0f0-438387da7a40 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:41:47.641406 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e52284d-60f0-489b-ad28-ff59b6dd59c6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 10:41:47.642037 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e52284d-60f0-489b-ad28-ff59b6dd59c6 None admin] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:41:47.642320 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e52284d-60f0-489b-ad28-ff59b6dd59c6 None admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:41:47.642595 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e52284d-60f0-489b-ad28-ff59b6dd59c6 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:41:47.643528 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.auth.core [None req-461d1de3-5e6e-4b9b-b0f0-438387da7a40 None None] MFA Rules not processed for user `bac4e6b76fe64217ac8fa44e71cffe6a`. Rule list: `[]` (Enabled: `True`). {{(pid=80143) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 10:41:47.644739 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-461d1de3-5e6e-4b9b-b0f0-438387da7a40 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 10:41:47.708357 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=100 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 10:41:47.726472 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e52284d-60f0-489b-ad28-ff59b6dd59c6 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 10:41:47.727757 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 216/445] 10.222.0.25 () {58 vars in 1024 bytes} [Sun Aug 7 10:41:47 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 07 10:41:47.735691 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e52284d-60f0-489b-ad28-ff59b6dd59c6 None admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 10:41:47.750351 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-461d1de3-5e6e-4b9b-b0f0-438387da7a40 None None] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 10:41:47.859056 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 230/446] 10.222.0.25 () {58 vars in 1282 bytes} [Sun Aug 7 10:41:47 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 361 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Aug 07 10:41:47.864680 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e52284d-60f0-489b-ad28-ff59b6dd59c6 None admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 10:41:47.875860 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-461d1de3-5e6e-4b9b-b0f0-438387da7a40 None None] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 10:41:47.878192 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=100 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 10:41:47.921738 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-839737f9-e451-4a9a-858f-af14d66c3267 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 10:41:47.922385 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-839737f9-e451-4a9a-858f-af14d66c3267 None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:41:47.922663 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-839737f9-e451-4a9a-858f-af14d66c3267 None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:41:47.923172 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-839737f9-e451-4a9a-858f-af14d66c3267 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:41:47.936665 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-839737f9-e451-4a9a-858f-af14d66c3267 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 07 10:41:47.941499 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application [None req-839737f9-e451-4a9a-858f-af14d66c3267 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 07 10:41:47.941499 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 07 10:41:47.941499 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 07 10:41:47.941499 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 07 10:41:47.941499 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 07 10:41:47.941499 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 07 10:41:47.941499 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 07 10:41:47.941499 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 07 10:41:47.941499 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 07 10:41:47.941499 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 07 10:41:47.941499 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 07 10:41:47.941499 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 07 10:41:47.941499 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 07 10:41:47.941499 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 07 10:41:47.941499 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 07 10:41:47.941499 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 07 10:41:47.942337 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 07 10:41:47.942337 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 07 10:41:47.942337 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 07 10:41:47.942337 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 07 10:41:47.942337 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 07 10:41:47.942337 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 07 10:41:47.942337 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 07 10:41:47.942337 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 07 10:41:47.942337 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 07 10:41:47.942337 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.get_or_create( Aug 07 10:41:47.942337 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 07 10:41:47.942337 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application with Lock( Aug 07 10:41:47.942337 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 07 10:41:47.942337 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self._enter() Aug 07 10:41:47.942337 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 07 10:41:47.942337 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 07 10:41:47.943627 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 07 10:41:47.943627 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.creator() Aug 07 10:41:47.943627 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 07 10:41:47.943627 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application created_value = creator( Aug 07 10:41:47.943627 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 07 10:41:47.943627 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 07 10:41:47.943627 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 07 10:41:47.943627 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 07 10:41:47.943627 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 07 10:41:47.943627 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 07 10:41:47.943627 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 07 10:41:47.943627 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application Aug 07 10:41:47.947049 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 217/447] 10.222.0.25 () {56 vars in 1114 bytes} [Sun Aug 7 10:41:47 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 07 10:41:47.949614 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-839737f9-e451-4a9a-858f-af14d66c3267 None admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 10:41:47.966117 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=100 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 10:41:47.983384 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e52284d-60f0-489b-ad28-ff59b6dd59c6 None admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 10:41:48.017395 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80d0d454-017b-4704-aebe-a27375460b5b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 10:41:48.018013 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80d0d454-017b-4704-aebe-a27375460b5b None admin] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:41:48.018289 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80d0d454-017b-4704-aebe-a27375460b5b None admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:41:48.018558 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80d0d454-017b-4704-aebe-a27375460b5b None admin] PATH_INFO: `/v3/projects/d5ac47e3b09f43d19a23a0f457fe9e23` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:41:48.019697 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80d0d454-017b-4704-aebe-a27375460b5b None admin] RBAC: Authorizing `identity:get_project(project_id=d5ac47e3b09f43d19a23a0f457fe9e23)` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 10:41:48.021109 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80d0d454-017b-4704-aebe-a27375460b5b None admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 10:41:48.022712 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 231/448] 10.222.0.25 () {56 vars in 1186 bytes} [Sun Aug 7 10:41:47 2022] GET /identity/v3/projects/d5ac47e3b09f43d19a23a0f457fe9e23 => generated 344 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 07 10:41:48.028792 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80d0d454-017b-4704-aebe-a27375460b5b None admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 10:41:48.054419 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-839737f9-e451-4a9a-858f-af14d66c3267 None admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 10:41:48.088138 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6aa4346a-a309-4a0b-8dfc-8c49f28a2b43 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 10:41:48.089346 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aa4346a-a309-4a0b-8dfc-8c49f28a2b43 None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:41:48.089632 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aa4346a-a309-4a0b-8dfc-8c49f28a2b43 None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:41:48.089908 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aa4346a-a309-4a0b-8dfc-8c49f28a2b43 None admin] PATH_INFO: `/v3/roles` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:41:48.090574 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6aa4346a-a309-4a0b-8dfc-8c49f28a2b43 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80143) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 10:41:48.090879 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6aa4346a-a309-4a0b-8dfc-8c49f28a2b43 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 10:41:48.092272 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6aa4346a-a309-4a0b-8dfc-8c49f28a2b43 None admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 10:41:48.099963 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 218/449] 10.222.0.25 () {56 vars in 1111 bytes} [Sun Aug 7 10:41:47 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 07 10:41:48.106622 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6aa4346a-a309-4a0b-8dfc-8c49f28a2b43 None admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 10:41:48.129841 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80d0d454-017b-4704-aebe-a27375460b5b None admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 10:41:48.133461 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=100 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 10:41:48.163198 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-270bb558-167e-4d45-9042-1619bff591b9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 10:41:48.163826 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-270bb558-167e-4d45-9042-1619bff591b9 None admin] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:41:48.164110 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-270bb558-167e-4d45-9042-1619bff591b9 None admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:41:48.164385 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-270bb558-167e-4d45-9042-1619bff591b9 None admin] PATH_INFO: `/v3/groups/f827aa44240646458903fb806aa670c7` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:41:48.165496 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-270bb558-167e-4d45-9042-1619bff591b9 None admin] RBAC: Authorizing `identity:get_group(group_id=f827aa44240646458903fb806aa670c7)` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 10:41:48.167947 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-270bb558-167e-4d45-9042-1619bff591b9 None admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 10:41:48.169502 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 232/450] 10.222.0.25 () {56 vars in 1180 bytes} [Sun Aug 7 10:41:48 2022] GET /identity/v3/groups/f827aa44240646458903fb806aa670c7 => generated 229 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 07 10:41:48.175238 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-270bb558-167e-4d45-9042-1619bff591b9 None admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 10:41:48.197036 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=100 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 10:41:48.209719 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6aa4346a-a309-4a0b-8dfc-8c49f28a2b43 None admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 10:41:48.243780 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4565f577-ba9a-4eb3-96e5-2f770e36f629 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 10:41:48.244393 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4565f577-ba9a-4eb3-96e5-2f770e36f629 None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:41:48.244677 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4565f577-ba9a-4eb3-96e5-2f770e36f629 None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:41:48.245235 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4565f577-ba9a-4eb3-96e5-2f770e36f629 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:41:48.274714 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-270bb558-167e-4d45-9042-1619bff591b9 None admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 10:41:48.308793 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc938926-d196-4ed6-b690-1c055b40e19a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 10:41:48.309396 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc938926-d196-4ed6-b690-1c055b40e19a None admin] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:41:48.309674 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc938926-d196-4ed6-b690-1c055b40e19a None admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:41:48.309946 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc938926-d196-4ed6-b690-1c055b40e19a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:41:48.310659 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc938926-d196-4ed6-b690-1c055b40e19a None admin] RBAC: Adding query filter params (group.id=f827aa44240646458903fb806aa670c7, role.id=81b1f1bcee2a473088fbbd327c1c5b23, scope.project.id=d5ac47e3b09f43d19a23a0f457fe9e23) {{(pid=80144) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 10:41:48.310794 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4565f577-ba9a-4eb3-96e5-2f770e36f629 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 10:41:48.310992 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc938926-d196-4ed6-b690-1c055b40e19a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 10:41:48.312284 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4565f577-ba9a-4eb3-96e5-2f770e36f629 None admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 10:41:48.312382 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc938926-d196-4ed6-b690-1c055b40e19a None admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 10:41:48.318021 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 233/451] 10.222.0.25 () {56 vars in 1376 bytes} [Sun Aug 7 10:41:48 2022] GET /identity/v3/role_assignments?group.id=f827aa44240646458903fb806aa670c7&scope.project.id=d5ac47e3b09f43d19a23a0f457fe9e23&role.id=81b1f1bcee2a473088fbbd327c1c5b23 => generated 611 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 07 10:41:48.339047 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=100 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 10:41:48.421397 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 219/452] 10.222.0.25 () {58 vars in 1282 bytes} [Sun Aug 7 10:41:48 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 315 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Aug 07 10:41:48.427144 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc938926-d196-4ed6-b690-1c055b40e19a None admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 10:41:48.515666 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=100 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 10:41:48.532136 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc938926-d196-4ed6-b690-1c055b40e19a None admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 10:41:48.564561 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6fa90ff-f47c-4660-89d8-c1e3dae99237 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 10:41:48.565182 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6fa90ff-f47c-4660-89d8-c1e3dae99237 None admin] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:41:48.565466 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6fa90ff-f47c-4660-89d8-c1e3dae99237 None admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:41:48.565741 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6fa90ff-f47c-4660-89d8-c1e3dae99237 None admin] PATH_INFO: `/v3/projects/d88586f68b734382abb9836b290aeb54` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:41:48.566853 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6fa90ff-f47c-4660-89d8-c1e3dae99237 None admin] RBAC: Authorizing `identity:get_project(project_id=d88586f68b734382abb9836b290aeb54)` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 10:41:48.568306 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6fa90ff-f47c-4660-89d8-c1e3dae99237 None admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 10:41:48.569903 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 234/453] 10.222.0.25 () {56 vars in 1186 bytes} [Sun Aug 7 10:41:48 2022] GET /identity/v3/projects/d88586f68b734382abb9836b290aeb54 => generated 302 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 10:41:48.576061 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4565f577-ba9a-4eb3-96e5-2f770e36f629 None admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 10:41:48.674180 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4565f577-ba9a-4eb3-96e5-2f770e36f629 None admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 10:41:48.699841 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=100 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 10:41:48.708671 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c47f718d-96d6-4899-a6ab-c94c78e6d57c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 10:41:48.709298 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c47f718d-96d6-4899-a6ab-c94c78e6d57c None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:41:48.709582 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c47f718d-96d6-4899-a6ab-c94c78e6d57c None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:41:48.709860 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c47f718d-96d6-4899-a6ab-c94c78e6d57c None admin] PATH_INFO: `/v3/groups/f26eb533e57a456d8f95265e204d21b5` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:41:48.710998 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c47f718d-96d6-4899-a6ab-c94c78e6d57c None admin] RBAC: Authorizing `identity:get_group(group_id=f26eb533e57a456d8f95265e204d21b5)` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 10:41:48.712431 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c47f718d-96d6-4899-a6ab-c94c78e6d57c None admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 10:41:48.714497 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 220/454] 10.222.0.25 () {56 vars in 1180 bytes} [Sun Aug 7 10:41:48 2022] GET /identity/v3/groups/f26eb533e57a456d8f95265e204d21b5 => generated 226 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 10:41:48.720314 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6fa90ff-f47c-4660-89d8-c1e3dae99237 None admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 10:41:48.816719 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6fa90ff-f47c-4660-89d8-c1e3dae99237 None admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 10:41:48.848797 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b17404d7-7009-4ad9-8ae5-33fc52964017 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 10:41:48.849385 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b17404d7-7009-4ad9-8ae5-33fc52964017 None admin] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:41:48.849669 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b17404d7-7009-4ad9-8ae5-33fc52964017 None admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:41:48.849945 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b17404d7-7009-4ad9-8ae5-33fc52964017 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:41:48.850661 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b17404d7-7009-4ad9-8ae5-33fc52964017 None admin] RBAC: Adding query filter params (group.id=f26eb533e57a456d8f95265e204d21b5, role.id=1512d531ebe248ea8e2d7d56d754b230, scope.project.id=d88586f68b734382abb9836b290aeb54) {{(pid=80144) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 10:41:48.851004 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b17404d7-7009-4ad9-8ae5-33fc52964017 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 10:41:48.852414 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b17404d7-7009-4ad9-8ae5-33fc52964017 None admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 10:41:48.858036 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 235/455] 10.222.0.25 () {56 vars in 1376 bytes} [Sun Aug 7 10:41:48 2022] GET /identity/v3/role_assignments?group.id=f26eb533e57a456d8f95265e204d21b5&scope.project.id=d88586f68b734382abb9836b290aeb54&role.id=1512d531ebe248ea8e2d7d56d754b230 => generated 611 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 10:41:51.354123 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eb7c6dc-0c17-4241-a874-c21c5b5b79fe None None] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:41:51.355083 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eb7c6dc-0c17-4241-a874-c21c5b5b79fe None None] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:41:51.355369 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eb7c6dc-0c17-4241-a874-c21c5b5b79fe None None] PATH_INFO: `/` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:41:51.356972 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 221/456] 10.222.0.25 () {54 vars in 927 bytes} [Sun Aug 7 10:41:51 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 07 10:41:51.362771 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57ade1cb-8aad-40f8-8db5-8d5070df63da None None] REQUEST_METHOD: `POST` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:41:51.363209 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57ade1cb-8aad-40f8-8db5-8d5070df63da None None] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:41:51.363384 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57ade1cb-8aad-40f8-8db5-8d5070df63da None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:41:51.386699 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.auth.core [None req-57ade1cb-8aad-40f8-8db5-8d5070df63da None None] MFA Rules not processed for user `bac4e6b76fe64217ac8fa44e71cffe6a`. Rule list: `[]` (Enabled: `True`). {{(pid=80144) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 10:41:51.387932 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.fernet_utils [None req-57ade1cb-8aad-40f8-8db5-8d5070df63da None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80144) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 10:41:51.404707 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=100 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 10:41:51.488441 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 236/457] 10.222.0.25 () {58 vars in 1024 bytes} [Sun Aug 7 10:41:51 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 07 10:41:51.536419 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1eb7c6dc-0c17-4241-a874-c21c5b5b79fe None None] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 10:41:51.656283 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-290fa5cc-6f8c-4b54-bee4-ec99b35921d7 None None] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:41:51.656577 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-290fa5cc-6f8c-4b54-bee4-ec99b35921d7 None None] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:41:51.656793 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-290fa5cc-6f8c-4b54-bee4-ec99b35921d7 None None] PATH_INFO: `/` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:41:51.658479 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 237/458] 10.222.0.25 () {54 vars in 927 bytes} [Sun Aug 7 10:41:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 07 10:41:51.666837 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36984cff-aa3a-4d66-b3e3-6f8cfbf7cd74 None None] REQUEST_METHOD: `POST` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:41:51.667217 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36984cff-aa3a-4d66-b3e3-6f8cfbf7cd74 None None] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:41:51.667600 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36984cff-aa3a-4d66-b3e3-6f8cfbf7cd74 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:41:51.675640 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1eb7c6dc-0c17-4241-a874-c21c5b5b79fe None None] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 10:41:51.698863 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.auth.core [None req-36984cff-aa3a-4d66-b3e3-6f8cfbf7cd74 None None] MFA Rules not processed for user `bac4e6b76fe64217ac8fa44e71cffe6a`. Rule list: `[]` (Enabled: `True`). {{(pid=80144) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 10:41:51.700189 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.fernet_utils [None req-36984cff-aa3a-4d66-b3e3-6f8cfbf7cd74 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80144) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 10:41:51.713998 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e561d67-7613-4329-8c4c-38d47a9d4f72 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 10:41:51.715337 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e561d67-7613-4329-8c4c-38d47a9d4f72 None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:41:51.715796 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e561d67-7613-4329-8c4c-38d47a9d4f72 None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:41:51.716263 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e561d67-7613-4329-8c4c-38d47a9d4f72 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:41:51.727001 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e561d67-7613-4329-8c4c-38d47a9d4f72 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 10:41:51.728643 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e561d67-7613-4329-8c4c-38d47a9d4f72 None admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 10:41:51.745703 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application [None req-4e561d67-7613-4329-8c4c-38d47a9d4f72 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 07 10:41:51.745703 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 07 10:41:51.745703 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 07 10:41:51.745703 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 07 10:41:51.745703 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 07 10:41:51.745703 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 07 10:41:51.745703 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 07 10:41:51.745703 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 07 10:41:51.745703 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 07 10:41:51.745703 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 07 10:41:51.745703 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 07 10:41:51.745703 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 07 10:41:51.745703 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 07 10:41:51.745703 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 07 10:41:51.745703 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 07 10:41:51.745703 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 07 10:41:51.746632 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 07 10:41:51.746632 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 07 10:41:51.746632 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 07 10:41:51.746632 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 07 10:41:51.746632 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 07 10:41:51.746632 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.get_or_create( Aug 07 10:41:51.746632 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 07 10:41:51.746632 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application with Lock( Aug 07 10:41:51.746632 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 07 10:41:51.746632 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self._enter() Aug 07 10:41:51.746632 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 07 10:41:51.746632 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 07 10:41:51.746632 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 07 10:41:51.746632 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.creator() Aug 07 10:41:51.746632 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 07 10:41:51.746632 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application created_value = creator( Aug 07 10:41:51.746632 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 07 10:41:51.747493 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 07 10:41:51.747493 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 07 10:41:51.747493 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application Aug 07 10:41:51.753718 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 222/459] 10.222.0.25 () {56 vars in 1108 bytes} [Sun Aug 7 10:41:51 2022] GET /identity/v3/domains/Default => generated 87 bytes in 213 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 07 10:41:51.755172 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c74fd768-b16d-4839-aa42-0a359a430b71 None None] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:41:51.755601 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c74fd768-b16d-4839-aa42-0a359a430b71 None None] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:41:51.756024 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c74fd768-b16d-4839-aa42-0a359a430b71 None None] PATH_INFO: `/` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:41:51.757440 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 223/460] 10.222.0.25 () {54 vars in 927 bytes} [Sun Aug 7 10:41:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 07 10:41:51.758731 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c74fd768-b16d-4839-aa42-0a359a430b71 None None] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 10:41:51.785787 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 238/461] 10.222.0.25 () {58 vars in 1024 bytes} [Sun Aug 7 10:41:51 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 07 10:41:51.785787 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-102606d5-1cf9-4f14-b7b9-3713f8081a80 None None] REQUEST_METHOD: `POST` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:41:51.785787 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-102606d5-1cf9-4f14-b7b9-3713f8081a80 None None] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:41:51.785787 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-102606d5-1cf9-4f14-b7b9-3713f8081a80 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:41:51.802086 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.auth.core [None req-102606d5-1cf9-4f14-b7b9-3713f8081a80 None None] MFA Rules not processed for user `bac4e6b76fe64217ac8fa44e71cffe6a`. Rule list: `[]` (Enabled: `True`). {{(pid=80144) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 10:41:51.803348 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.fernet_utils [None req-102606d5-1cf9-4f14-b7b9-3713f8081a80 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80144) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 10:41:51.834389 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=100 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 10:41:51.873572 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 239/462] 10.222.0.25 () {58 vars in 1024 bytes} [Sun Aug 7 10:41:51 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 07 10:41:51.874502 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-102606d5-1cf9-4f14-b7b9-3713f8081a80 None None] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 10:41:51.893416 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=100 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 10:41:51.901539 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c74fd768-b16d-4839-aa42-0a359a430b71 None None] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 10:41:51.949357 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7402e70b-cd29-4bec-aa88-6699ff0904f8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 10:41:51.950435 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7402e70b-cd29-4bec-aa88-6699ff0904f8 None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:41:51.951301 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7402e70b-cd29-4bec-aa88-6699ff0904f8 None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:41:51.951715 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7402e70b-cd29-4bec-aa88-6699ff0904f8 None admin] PATH_INFO: `/v3/domains` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:41:51.952503 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7402e70b-cd29-4bec-aa88-6699ff0904f8 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80143) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 10:41:51.952944 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7402e70b-cd29-4bec-aa88-6699ff0904f8 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 10:41:51.955327 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7402e70b-cd29-4bec-aa88-6699ff0904f8 None admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 10:41:51.977096 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 224/463] 10.222.0.25 () {56 vars in 1109 bytes} [Sun Aug 7 10:41:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 10:41:51.977096 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7402e70b-cd29-4bec-aa88-6699ff0904f8 None admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 10:41:52.012444 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-102606d5-1cf9-4f14-b7b9-3713f8081a80 None None] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 10:41:52.055398 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-827d4418-8b72-4785-be18-f44e590bf589 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 10:41:52.056655 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-827d4418-8b72-4785-be18-f44e590bf589 None admin] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:41:52.057495 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-827d4418-8b72-4785-be18-f44e590bf589 None admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:41:52.057912 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-827d4418-8b72-4785-be18-f44e590bf589 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:41:52.071362 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-827d4418-8b72-4785-be18-f44e590bf589 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 10:41:52.074183 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-827d4418-8b72-4785-be18-f44e590bf589 None admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 10:41:52.097265 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application [None req-827d4418-8b72-4785-be18-f44e590bf589 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 07 10:41:52.097265 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 07 10:41:52.097265 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 07 10:41:52.097265 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 07 10:41:52.097265 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 07 10:41:52.097265 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 07 10:41:52.097265 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 07 10:41:52.097265 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 07 10:41:52.097265 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 07 10:41:52.097265 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 07 10:41:52.097265 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 07 10:41:52.097265 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 07 10:41:52.097265 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 07 10:41:52.097265 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 07 10:41:52.097265 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 07 10:41:52.097265 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 07 10:41:52.098398 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 07 10:41:52.098398 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 07 10:41:52.098398 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 07 10:41:52.098398 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 07 10:41:52.098398 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 07 10:41:52.098398 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self.get_or_create( Aug 07 10:41:52.098398 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 07 10:41:52.098398 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application with Lock( Aug 07 10:41:52.098398 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 07 10:41:52.098398 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self._enter() Aug 07 10:41:52.098398 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 07 10:41:52.098398 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 07 10:41:52.098398 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 07 10:41:52.098398 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self.creator() Aug 07 10:41:52.098398 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 07 10:41:52.098398 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application created_value = creator( Aug 07 10:41:52.098398 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 07 10:41:52.099474 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 07 10:41:52.099474 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 07 10:41:52.099474 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application Aug 07 10:41:52.100057 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7402e70b-cd29-4bec-aa88-6699ff0904f8 None admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 10:41:52.106499 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 240/464] 10.222.0.25 () {56 vars in 1108 bytes} [Sun Aug 7 10:41:51 2022] GET /identity/v3/domains/Default => generated 87 bytes in 227 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 07 10:41:52.106499 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-827d4418-8b72-4785-be18-f44e590bf589 None admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 10:41:52.114208 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=100 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 10:41:52.152130 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d4c0a7c-273e-47ca-99d9-ae7ea63ace8f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 10:41:52.153549 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d4c0a7c-273e-47ca-99d9-ae7ea63ace8f None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:41:52.154026 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d4c0a7c-273e-47ca-99d9-ae7ea63ace8f None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:41:52.154729 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d4c0a7c-273e-47ca-99d9-ae7ea63ace8f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:41:52.169423 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d4c0a7c-273e-47ca-99d9-ae7ea63ace8f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 10:41:52.170477 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=100 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 10:41:52.173068 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d4c0a7c-273e-47ca-99d9-ae7ea63ace8f None admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 10:41:52.193162 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application [None req-4d4c0a7c-273e-47ca-99d9-ae7ea63ace8f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 07 10:41:52.193162 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 07 10:41:52.193162 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 07 10:41:52.193162 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 07 10:41:52.193162 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 07 10:41:52.193162 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 07 10:41:52.193162 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 07 10:41:52.193162 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 07 10:41:52.193162 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 07 10:41:52.193162 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 07 10:41:52.193162 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 07 10:41:52.193162 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 07 10:41:52.193162 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 07 10:41:52.193162 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 07 10:41:52.193162 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 07 10:41:52.193162 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 07 10:41:52.194142 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 07 10:41:52.194142 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 07 10:41:52.194142 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 07 10:41:52.194142 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 07 10:41:52.194142 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 07 10:41:52.194142 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.get_or_create( Aug 07 10:41:52.194142 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 07 10:41:52.194142 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application with Lock( Aug 07 10:41:52.194142 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 07 10:41:52.194142 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self._enter() Aug 07 10:41:52.194142 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 07 10:41:52.194142 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 07 10:41:52.194142 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 07 10:41:52.194142 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.creator() Aug 07 10:41:52.194142 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 07 10:41:52.194142 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application created_value = creator( Aug 07 10:41:52.194142 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 07 10:41:52.194990 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 07 10:41:52.194990 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 07 10:41:52.194990 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application Aug 07 10:41:52.196982 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 225/465] 10.222.0.25 () {56 vars in 1108 bytes} [Sun Aug 7 10:41:51 2022] GET /identity/v3/domains/Default => generated 87 bytes in 224 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 07 10:41:52.198543 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d4c0a7c-273e-47ca-99d9-ae7ea63ace8f None admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 10:41:52.226904 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-827d4418-8b72-4785-be18-f44e590bf589 None admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 10:41:52.273329 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13e63c2b-1f1e-4bfe-b9e4-027c2afef752 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 10:41:52.274444 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13e63c2b-1f1e-4bfe-b9e4-027c2afef752 None admin] REQUEST_METHOD: `POST` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:41:52.274879 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13e63c2b-1f1e-4bfe-b9e4-027c2afef752 None admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:41:52.276056 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13e63c2b-1f1e-4bfe-b9e4-027c2afef752 None admin] PATH_INFO: `/v3/users` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:41:52.277193 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13e63c2b-1f1e-4bfe-b9e4-027c2afef752 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 10:41:52.279113 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13e63c2b-1f1e-4bfe-b9e4-027c2afef752 None admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 10:41:52.315565 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 241/466] 10.222.0.25 () {60 vars in 1132 bytes} [Sun Aug 7 10:41:52 2022] POST /identity/v3/users => generated 246 bytes in 208 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 07 10:41:52.315565 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13e63c2b-1f1e-4bfe-b9e4-027c2afef752 None admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 10:41:52.321363 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d4c0a7c-273e-47ca-99d9-ae7ea63ace8f None admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 10:41:52.361945 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f4411ad-4d27-43c9-a992-263cb0f1e8fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 10:41:52.363025 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f4411ad-4d27-43c9-a992-263cb0f1e8fb None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:41:52.363374 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f4411ad-4d27-43c9-a992-263cb0f1e8fb None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:41:52.363708 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f4411ad-4d27-43c9-a992-263cb0f1e8fb None admin] PATH_INFO: `/v3/domains` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:41:52.364535 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f4411ad-4d27-43c9-a992-263cb0f1e8fb None admin] RBAC: Adding query filter params (name=Default) {{(pid=80143) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 10:41:52.364946 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f4411ad-4d27-43c9-a992-263cb0f1e8fb None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 10:41:52.366565 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f4411ad-4d27-43c9-a992-263cb0f1e8fb None admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 10:41:52.381931 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 226/467] 10.222.0.25 () {56 vars in 1109 bytes} [Sun Aug 7 10:41:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 10:41:52.383834 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c1eba6c-2e17-4a3f-8b2d-ddd3f6a51149 None None] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:41:52.384388 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c1eba6c-2e17-4a3f-8b2d-ddd3f6a51149 None None] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:41:52.384667 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c1eba6c-2e17-4a3f-8b2d-ddd3f6a51149 None None] PATH_INFO: `/` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:41:52.386956 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 227/468] 10.222.0.25 () {54 vars in 927 bytes} [Sun Aug 7 10:41:52 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 07 10:41:52.388573 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c1eba6c-2e17-4a3f-8b2d-ddd3f6a51149 None None] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 10:41:52.421915 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=100 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 10:41:52.445856 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13e63c2b-1f1e-4bfe-b9e4-027c2afef752 None admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 10:41:52.462561 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 10:41:52.493602 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b880497d-2bd5-4816-9ec0-ed9764bdb4be None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 10:41:52.494133 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b880497d-2bd5-4816-9ec0-ed9764bdb4be None admin] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:41:52.494419 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b880497d-2bd5-4816-9ec0-ed9764bdb4be None admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:41:52.495258 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b880497d-2bd5-4816-9ec0-ed9764bdb4be None admin] PATH_INFO: `/v3/domains` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:41:52.495936 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b880497d-2bd5-4816-9ec0-ed9764bdb4be None admin] RBAC: Adding query filter params (name=Default) {{(pid=80144) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 10:41:52.496240 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b880497d-2bd5-4816-9ec0-ed9764bdb4be None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 10:41:52.497632 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b880497d-2bd5-4816-9ec0-ed9764bdb4be None admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 10:41:52.512592 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 242/469] 10.222.0.25 () {56 vars in 1109 bytes} [Sun Aug 7 10:41:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 10:41:52.513868 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fd0a917-eb25-45ee-aa3f-dc16c1f253da None None] REQUEST_METHOD: `POST` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:41:52.514158 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fd0a917-eb25-45ee-aa3f-dc16c1f253da None None] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:41:52.514436 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fd0a917-eb25-45ee-aa3f-dc16c1f253da None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:41:52.534492 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.auth.core [None req-2fd0a917-eb25-45ee-aa3f-dc16c1f253da None None] MFA Rules not processed for user `bac4e6b76fe64217ac8fa44e71cffe6a`. Rule list: `[]` (Enabled: `True`). {{(pid=80144) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 10:41:52.536395 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.fernet_utils [None req-2fd0a917-eb25-45ee-aa3f-dc16c1f253da None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80144) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 10:41:52.567969 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c1eba6c-2e17-4a3f-8b2d-ddd3f6a51149 None None] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 10:41:52.608257 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6432befb-7fd3-4d73-a0bb-8a0866a5d8e7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 10:41:52.609743 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6432befb-7fd3-4d73-a0bb-8a0866a5d8e7 None admin] REQUEST_METHOD: `POST` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:41:52.610263 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6432befb-7fd3-4d73-a0bb-8a0866a5d8e7 None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:41:52.610706 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6432befb-7fd3-4d73-a0bb-8a0866a5d8e7 None admin] PATH_INFO: `/v3/users` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:41:52.611781 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6432befb-7fd3-4d73-a0bb-8a0866a5d8e7 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 10:41:52.614336 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6432befb-7fd3-4d73-a0bb-8a0866a5d8e7 None admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 10:41:52.621022 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 243/470] 10.222.0.25 () {58 vars in 1024 bytes} [Sun Aug 7 10:41:52 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 07 10:41:52.621977 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fd0a917-eb25-45ee-aa3f-dc16c1f253da None None] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 10:41:52.643310 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 228/471] 10.222.0.25 () {60 vars in 1132 bytes} [Sun Aug 7 10:41:52 2022] POST /identity/v3/users => generated 248 bytes in 255 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 07 10:41:52.658996 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6432befb-7fd3-4d73-a0bb-8a0866a5d8e7 None admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 10:41:52.759745 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 10:41:52.764717 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fd0a917-eb25-45ee-aa3f-dc16c1f253da None None] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 10:41:52.769516 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=100 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 10:41:52.791904 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6432befb-7fd3-4d73-a0bb-8a0866a5d8e7 None admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 10:41:52.818772 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75a5450d-e78a-4645-b07d-2258abd7a275 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 10:41:52.824231 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75a5450d-e78a-4645-b07d-2258abd7a275 None admin] REQUEST_METHOD: `POST` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:41:52.824864 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75a5450d-e78a-4645-b07d-2258abd7a275 None admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:41:52.825593 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75a5450d-e78a-4645-b07d-2258abd7a275 None admin] PATH_INFO: `/v3/users` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:41:52.826471 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75a5450d-e78a-4645-b07d-2258abd7a275 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 10:41:52.828997 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75a5450d-e78a-4645-b07d-2258abd7a275 None admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 10:41:52.848704 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfdf7279-c763-47a3-9719-1f47352937fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 10:41:52.850413 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfdf7279-c763-47a3-9719-1f47352937fa None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:41:52.852277 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfdf7279-c763-47a3-9719-1f47352937fa None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:41:52.852746 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfdf7279-c763-47a3-9719-1f47352937fa None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:41:52.867502 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfdf7279-c763-47a3-9719-1f47352937fa None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 10:41:52.869276 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfdf7279-c763-47a3-9719-1f47352937fa None admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 10:41:52.876897 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 244/472] 10.222.0.25 () {60 vars in 1132 bytes} [Sun Aug 7 10:41:52 2022] POST /identity/v3/users => generated 248 bytes in 255 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 07 10:41:52.894307 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application [None req-bfdf7279-c763-47a3-9719-1f47352937fa None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 07 10:41:52.894307 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 07 10:41:52.894307 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 07 10:41:52.894307 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 07 10:41:52.894307 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 07 10:41:52.894307 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 07 10:41:52.894307 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 07 10:41:52.894307 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 07 10:41:52.894307 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 07 10:41:52.894307 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 07 10:41:52.894307 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 07 10:41:52.894307 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 07 10:41:52.894307 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 07 10:41:52.894307 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 07 10:41:52.894307 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 07 10:41:52.894307 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 07 10:41:52.895550 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 07 10:41:52.895550 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 07 10:41:52.895550 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 07 10:41:52.895550 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 07 10:41:52.895550 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 07 10:41:52.895550 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.get_or_create( Aug 07 10:41:52.895550 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 07 10:41:52.895550 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application with Lock( Aug 07 10:41:52.895550 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 07 10:41:52.895550 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self._enter() Aug 07 10:41:52.895550 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 07 10:41:52.895550 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 07 10:41:52.895550 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 07 10:41:52.895550 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.creator() Aug 07 10:41:52.895550 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 07 10:41:52.895550 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application created_value = creator( Aug 07 10:41:52.895550 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 07 10:41:52.896375 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 07 10:41:52.896375 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 07 10:41:52.896375 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application Aug 07 10:41:52.898274 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 229/473] 10.222.0.25 () {56 vars in 1108 bytes} [Sun Aug 7 10:41:52 2022] GET /identity/v3/domains/Default => generated 87 bytes in 240 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 07 10:41:52.905238 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75a5450d-e78a-4645-b07d-2258abd7a275 None admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 10:41:53.005835 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5042c190-08f6-4ae9-b933-bd84d43dd0bf None None] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:41:53.006312 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5042c190-08f6-4ae9-b933-bd84d43dd0bf None None] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:41:53.006693 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5042c190-08f6-4ae9-b933-bd84d43dd0bf None None] PATH_INFO: `/` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:41:53.009396 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 230/474] 10.222.0.25 () {54 vars in 927 bytes} [Sun Aug 7 10:41:53 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 07 10:41:53.017683 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e511ae3a-dc41-4fbb-8b4c-294d3e7aaaf6 None None] REQUEST_METHOD: `POST` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:41:53.018044 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e511ae3a-dc41-4fbb-8b4c-294d3e7aaaf6 None None] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:41:53.018419 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e511ae3a-dc41-4fbb-8b4c-294d3e7aaaf6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:41:53.022905 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75a5450d-e78a-4645-b07d-2258abd7a275 None admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 10:41:53.046283 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.auth.core [None req-e511ae3a-dc41-4fbb-8b4c-294d3e7aaaf6 None None] MFA Rules not processed for user `bac4e6b76fe64217ac8fa44e71cffe6a`. Rule list: `[]` (Enabled: `True`). {{(pid=80143) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 10:41:53.048072 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-e511ae3a-dc41-4fbb-8b4c-294d3e7aaaf6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 10:41:53.073979 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90f1ca08-90cd-4142-87fc-aeb7b8826969 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 10:41:53.074598 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90f1ca08-90cd-4142-87fc-aeb7b8826969 None admin] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:41:53.074879 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90f1ca08-90cd-4142-87fc-aeb7b8826969 None admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:41:53.075864 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90f1ca08-90cd-4142-87fc-aeb7b8826969 None admin] PATH_INFO: `/v3/domains` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:41:53.076066 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90f1ca08-90cd-4142-87fc-aeb7b8826969 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80144) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 10:41:53.076372 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90f1ca08-90cd-4142-87fc-aeb7b8826969 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 10:41:53.078808 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90f1ca08-90cd-4142-87fc-aeb7b8826969 None admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 10:41:53.098562 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 245/475] 10.222.0.25 () {56 vars in 1109 bytes} [Sun Aug 7 10:41:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 10:41:53.104894 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90f1ca08-90cd-4142-87fc-aeb7b8826969 None admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 10:41:53.131082 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 10:41:53.152875 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 231/476] 10.222.0.25 () {58 vars in 1024 bytes} [Sun Aug 7 10:41:53 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 137 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 07 10:41:53.178506 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e511ae3a-dc41-4fbb-8b4c-294d3e7aaaf6 None None] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 10:41:53.238291 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90f1ca08-90cd-4142-87fc-aeb7b8826969 None admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 10:41:53.243709 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=100 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 10:41:53.288974 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11e4ca17-ce7b-431f-a5bd-f0da7c5e593b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 10:41:53.289629 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11e4ca17-ce7b-431f-a5bd-f0da7c5e593b None admin] REQUEST_METHOD: `POST` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:41:53.289942 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11e4ca17-ce7b-431f-a5bd-f0da7c5e593b None admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:41:53.290438 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11e4ca17-ce7b-431f-a5bd-f0da7c5e593b None admin] PATH_INFO: `/v3/users` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:41:53.291493 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11e4ca17-ce7b-431f-a5bd-f0da7c5e593b None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 10:41:53.293317 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11e4ca17-ce7b-431f-a5bd-f0da7c5e593b None admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 10:41:53.298533 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e511ae3a-dc41-4fbb-8b4c-294d3e7aaaf6 None None] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 10:41:53.328183 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 246/477] 10.222.0.25 () {60 vars in 1132 bytes} [Sun Aug 7 10:41:53 2022] POST /identity/v3/users => generated 249 bytes in 224 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 07 10:41:53.352843 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8680d40-cffa-4e13-88be-94605033c1d3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 10:41:53.353531 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8680d40-cffa-4e13-88be-94605033c1d3 None admin] REQUEST_METHOD: `POST` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:41:53.353847 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8680d40-cffa-4e13-88be-94605033c1d3 None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:41:53.354203 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8680d40-cffa-4e13-88be-94605033c1d3 None admin] PATH_INFO: `/v3/roles` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:41:53.354888 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8680d40-cffa-4e13-88be-94605033c1d3 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 10:41:53.356383 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8680d40-cffa-4e13-88be-94605033c1d3 None admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 10:41:53.370585 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.sql.core [None req-c8680d40-cffa-4e13-88be-94605033c1d3 None admin] Conflict role: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Aug 07 10:41:53.370585 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: [SQL: INSERT INTO `role` (id, name, description, extra) VALUES (%(id)s, %(name)s, %(description)s, %(extra)s)] Aug 07 10:41:53.370585 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: [parameters: {'id': 'aa418e5bd88f45299e88832c2d5323e2', 'name': 'anotherrole', 'description': None, 'extra': '{}'}] Aug 07 10:41:53.370585 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=80143) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Aug 07 10:41:53.391831 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application [None req-c8680d40-cffa-4e13-88be-94605033c1d3 None admin] Conflict occurred attempting to store role - Duplicate entry found with name anotherrole.: keystone.exception.Conflict: Conflict occurred attempting to store role - Duplicate entry found with name anotherrole. Aug 07 10:41:53.391831 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 07 10:41:53.391831 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Aug 07 10:41:53.391831 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application self.dialect.do_execute( Aug 07 10:41:53.391831 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Aug 07 10:41:53.391831 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Aug 07 10:41:53.391831 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Aug 07 10:41:53.391831 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application result = self._query(query) Aug 07 10:41:53.391831 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Aug 07 10:41:53.391831 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application conn.query(q) Aug 07 10:41:53.391831 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Aug 07 10:41:53.391831 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Aug 07 10:41:53.391831 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Aug 07 10:41:53.391831 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application result.read() Aug 07 10:41:53.391831 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Aug 07 10:41:53.392886 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Aug 07 10:41:53.392886 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Aug 07 10:41:53.392886 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application packet.raise_for_error() Aug 07 10:41:53.392886 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Aug 07 10:41:53.392886 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Aug 07 10:41:53.392886 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Aug 07 10:41:53.392886 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Aug 07 10:41:53.392886 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Aug 07 10:41:53.392886 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application Aug 07 10:41:53.392886 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Aug 07 10:41:53.392886 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application Aug 07 10:41:53.392886 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 07 10:41:53.392886 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Aug 07 10:41:53.392886 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return method(*args, **kwargs) Aug 07 10:41:53.392886 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 33, in create_role Aug 07 10:41:53.392886 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return ref.to_dict() Aug 07 10:41:53.392886 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Aug 07 10:41:53.392886 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application next(self.gen) Aug 07 10:41:53.395413 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Aug 07 10:41:53.395413 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application yield resource Aug 07 10:41:53.395413 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Aug 07 10:41:53.395413 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application next(self.gen) Aug 07 10:41:53.395413 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Aug 07 10:41:53.395413 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application self.session.rollback() Aug 07 10:41:53.395413 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Aug 07 10:41:53.395413 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application self.force_reraise() Aug 07 10:41:53.395413 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Aug 07 10:41:53.395413 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application raise self.value Aug 07 10:41:53.395413 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Aug 07 10:41:53.395413 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Aug 07 10:41:53.395413 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Aug 07 10:41:53.395413 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application session.commit() Aug 07 10:41:53.395413 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Aug 07 10:41:53.395413 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Aug 07 10:41:53.395413 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Aug 07 10:41:53.397457 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application self._prepare_impl() Aug 07 10:41:53.397457 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Aug 07 10:41:53.397457 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application self.session.flush() Aug 07 10:41:53.397457 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Aug 07 10:41:53.397457 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application self._flush(objects) Aug 07 10:41:53.397457 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Aug 07 10:41:53.397457 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Aug 07 10:41:53.397457 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Aug 07 10:41:53.397457 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application compat.raise_( Aug 07 10:41:53.397457 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Aug 07 10:41:53.397457 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application raise exception Aug 07 10:41:53.397457 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Aug 07 10:41:53.397457 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application flush_context.execute() Aug 07 10:41:53.397457 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Aug 07 10:41:53.397457 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application rec.execute(self) Aug 07 10:41:53.397457 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Aug 07 10:41:53.397457 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Aug 07 10:41:53.403672 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Aug 07 10:41:53.403672 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application _emit_insert_statements( Aug 07 10:41:53.403672 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1097, in _emit_insert_statements Aug 07 10:41:53.403672 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application c = connection._execute_20( Aug 07 10:41:53.403672 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1631, in _execute_20 Aug 07 10:41:53.403672 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Aug 07 10:41:53.403672 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 332, in _execute_on_connection Aug 07 10:41:53.403672 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Aug 07 10:41:53.403672 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1498, in _execute_clauseelement Aug 07 10:41:53.403672 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application ret = self._execute_context( Aug 07 10:41:53.403672 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1862, in _execute_context Aug 07 10:41:53.403672 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Aug 07 10:41:53.403672 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2041, in _handle_dbapi_exception Aug 07 10:41:53.403672 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Aug 07 10:41:53.403672 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Aug 07 10:41:53.403672 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application raise exception Aug 07 10:41:53.405628 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Aug 07 10:41:53.405628 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application self.dialect.do_execute( Aug 07 10:41:53.405628 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Aug 07 10:41:53.405628 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Aug 07 10:41:53.405628 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Aug 07 10:41:53.405628 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application result = self._query(query) Aug 07 10:41:53.405628 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Aug 07 10:41:53.405628 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application conn.query(q) Aug 07 10:41:53.405628 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Aug 07 10:41:53.405628 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Aug 07 10:41:53.405628 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Aug 07 10:41:53.405628 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application result.read() Aug 07 10:41:53.405628 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Aug 07 10:41:53.405628 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Aug 07 10:41:53.405628 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Aug 07 10:41:53.405628 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application packet.raise_for_error() Aug 07 10:41:53.407646 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Aug 07 10:41:53.407646 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Aug 07 10:41:53.407646 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Aug 07 10:41:53.407646 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Aug 07 10:41:53.407646 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Aug 07 10:41:53.407646 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application [SQL: INSERT INTO `role` (id, name, description, extra) VALUES (%(id)s, %(name)s, %(description)s, %(extra)s)] Aug 07 10:41:53.407646 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application [parameters: {'id': 'aa418e5bd88f45299e88832c2d5323e2', 'name': 'anotherrole', 'description': None, 'extra': '{}'}] Aug 07 10:41:53.407646 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Aug 07 10:41:53.407646 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application Aug 07 10:41:53.407646 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 07 10:41:53.407646 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application Aug 07 10:41:53.407646 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 07 10:41:53.407646 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 07 10:41:53.407646 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 07 10:41:53.407646 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 07 10:41:53.407646 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 07 10:41:53.409554 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 07 10:41:53.409554 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 07 10:41:53.409554 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 07 10:41:53.409554 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 07 10:41:53.409554 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 07 10:41:53.409554 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 07 10:41:53.409554 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 115, in post Aug 07 10:41:53.409554 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application ref = PROVIDERS.role_api.create_role( Aug 07 10:41:53.409554 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 07 10:41:53.409554 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 07 10:41:53.409554 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1298, in create_role Aug 07 10:41:53.409554 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application ret = self.driver.create_role(role_id, role) Aug 07 10:41:53.409554 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Aug 07 10:41:53.409554 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Aug 07 10:41:53.409554 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store role - Duplicate entry found with name anotherrole. Aug 07 10:41:53.409554 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application Aug 07 10:41:53.416098 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 232/478] 10.222.0.25 () {60 vars in 1131 bytes} [Sun Aug 7 10:41:53 2022] POST /identity/v3/roles => generated 144 bytes in 215 msecs (HTTP/1.1 409) 5 headers in 183 bytes (2 switches on core 0) Aug 07 10:41:53.416237 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11e4ca17-ce7b-431f-a5bd-f0da7c5e593b None admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 10:41:53.514970 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 10:41:53.547124 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11e4ca17-ce7b-431f-a5bd-f0da7c5e593b None admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 10:41:53.584246 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bf8dfd3-335a-456d-a68e-3a0e13c63a8f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 10:41:53.585122 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bf8dfd3-335a-456d-a68e-3a0e13c63a8f None admin] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:41:53.585602 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bf8dfd3-335a-456d-a68e-3a0e13c63a8f None admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:41:53.586055 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bf8dfd3-335a-456d-a68e-3a0e13c63a8f None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:41:53.596982 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bf8dfd3-335a-456d-a68e-3a0e13c63a8f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 07 10:41:53.601843 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application [None req-7bf8dfd3-335a-456d-a68e-3a0e13c63a8f None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 07 10:41:53.601843 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 07 10:41:53.601843 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 07 10:41:53.601843 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 07 10:41:53.601843 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 07 10:41:53.601843 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 07 10:41:53.601843 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 07 10:41:53.601843 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 07 10:41:53.601843 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 07 10:41:53.601843 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 07 10:41:53.601843 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 07 10:41:53.601843 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 07 10:41:53.601843 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 07 10:41:53.601843 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 07 10:41:53.601843 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 07 10:41:53.601843 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 07 10:41:53.602886 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 07 10:41:53.602886 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 07 10:41:53.602886 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 07 10:41:53.602886 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 07 10:41:53.602886 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 07 10:41:53.602886 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 07 10:41:53.602886 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 07 10:41:53.602886 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 07 10:41:53.602886 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 07 10:41:53.602886 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self.get_or_create( Aug 07 10:41:53.602886 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 07 10:41:53.602886 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application with Lock( Aug 07 10:41:53.602886 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 07 10:41:53.602886 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self._enter() Aug 07 10:41:53.602886 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 07 10:41:53.602886 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 07 10:41:53.604455 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 07 10:41:53.604455 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self.creator() Aug 07 10:41:53.604455 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 07 10:41:53.604455 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application created_value = creator( Aug 07 10:41:53.604455 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 07 10:41:53.604455 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 07 10:41:53.604455 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 07 10:41:53.604455 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 07 10:41:53.604455 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 07 10:41:53.604455 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 07 10:41:53.604455 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 07 10:41:53.604455 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application Aug 07 10:41:53.607069 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 247/479] 10.222.0.25 () {56 vars in 1114 bytes} [Sun Aug 7 10:41:53 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 208 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 07 10:41:53.613154 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8680d40-cffa-4e13-88be-94605033c1d3 None admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 10:41:53.731270 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8680d40-cffa-4e13-88be-94605033c1d3 None admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 10:41:53.753234 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 10:41:53.777819 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fec1da12-9635-4fd5-929b-bf836667b222 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 10:41:53.778452 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fec1da12-9635-4fd5-929b-bf836667b222 None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:41:53.778737 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fec1da12-9635-4fd5-929b-bf836667b222 None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:41:53.779045 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fec1da12-9635-4fd5-929b-bf836667b222 None admin] PATH_INFO: `/v3/roles` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:41:53.779727 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fec1da12-9635-4fd5-929b-bf836667b222 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80143) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 10:41:53.780035 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fec1da12-9635-4fd5-929b-bf836667b222 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 10:41:53.781405 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fec1da12-9635-4fd5-929b-bf836667b222 None admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 10:41:53.790451 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 233/480] 10.222.0.25 () {56 vars in 1111 bytes} [Sun Aug 7 10:41:53 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 10:41:54.622450 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ddecf6b-f1fc-4ee2-8c41-f765d1c863da None None] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:41:54.623005 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ddecf6b-f1fc-4ee2-8c41-f765d1c863da None None] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:41:54.623084 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ddecf6b-f1fc-4ee2-8c41-f765d1c863da None None] PATH_INFO: `/` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:41:54.624663 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 248/481] 10.222.0.25 () {54 vars in 927 bytes} [Sun Aug 7 10:41:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 07 10:41:54.629635 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80e1a5f2-64ed-4de5-b75e-0c547a2d6cb2 None None] REQUEST_METHOD: `POST` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:41:54.630025 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80e1a5f2-64ed-4de5-b75e-0c547a2d6cb2 None None] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:41:54.630215 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80e1a5f2-64ed-4de5-b75e-0c547a2d6cb2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:41:54.653985 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.auth.core [None req-80e1a5f2-64ed-4de5-b75e-0c547a2d6cb2 None None] MFA Rules not processed for user `bac4e6b76fe64217ac8fa44e71cffe6a`. Rule list: `[]` (Enabled: `True`). {{(pid=80143) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 10:41:54.655314 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-80e1a5f2-64ed-4de5-b75e-0c547a2d6cb2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 10:41:54.730851 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 234/482] 10.222.0.25 () {58 vars in 1024 bytes} [Sun Aug 7 10:41:54 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 07 10:41:54.763461 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ddecf6b-f1fc-4ee2-8c41-f765d1c863da None None] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 10:41:54.905989 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ddecf6b-f1fc-4ee2-8c41-f765d1c863da None None] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 10:41:54.953359 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e3450f2-da34-4203-a4d0-74c9eb5e1de3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 10:41:54.954134 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e3450f2-da34-4203-a4d0-74c9eb5e1de3 None admin] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:41:54.954427 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e3450f2-da34-4203-a4d0-74c9eb5e1de3 None admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:41:54.954707 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e3450f2-da34-4203-a4d0-74c9eb5e1de3 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:41:54.963262 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=100 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 10:41:54.978028 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e3450f2-da34-4203-a4d0-74c9eb5e1de3 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 07 10:41:54.983634 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application [None req-2e3450f2-da34-4203-a4d0-74c9eb5e1de3 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 07 10:41:54.983634 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 07 10:41:54.983634 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 07 10:41:54.983634 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 07 10:41:54.983634 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 07 10:41:54.983634 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 07 10:41:54.983634 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 07 10:41:54.983634 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 07 10:41:54.983634 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 07 10:41:54.983634 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 07 10:41:54.983634 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 07 10:41:54.983634 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 07 10:41:54.983634 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 07 10:41:54.983634 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 07 10:41:54.983634 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 07 10:41:54.983634 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 07 10:41:54.984503 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 07 10:41:54.984503 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 07 10:41:54.984503 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 07 10:41:54.984503 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 07 10:41:54.984503 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 07 10:41:54.984503 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 07 10:41:54.984503 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 07 10:41:54.984503 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 07 10:41:54.984503 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 07 10:41:54.984503 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self.get_or_create( Aug 07 10:41:54.984503 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 07 10:41:54.984503 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application with Lock( Aug 07 10:41:54.984503 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 07 10:41:54.984503 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self._enter() Aug 07 10:41:54.984503 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 07 10:41:54.984503 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 07 10:41:54.985281 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 07 10:41:54.985281 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self.creator() Aug 07 10:41:54.985281 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 07 10:41:54.985281 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application created_value = creator( Aug 07 10:41:54.985281 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 07 10:41:54.985281 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 07 10:41:54.985281 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 07 10:41:54.985281 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 07 10:41:54.985281 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 07 10:41:54.985281 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 07 10:41:54.985281 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 07 10:41:54.985281 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application Aug 07 10:41:54.985281 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 249/483] 10.222.0.25 () {56 vars in 1102 bytes} [Sun Aug 7 10:41:54 2022] GET /identity/v3/roles/service => generated 85 bytes in 222 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 07 10:41:54.991007 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80e1a5f2-64ed-4de5-b75e-0c547a2d6cb2 None None] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 10:41:55.099848 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80e1a5f2-64ed-4de5-b75e-0c547a2d6cb2 None None] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 10:41:55.127332 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=100 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 10:41:55.144742 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cf3bdf1-6dee-431e-be5b-1cac1c591fb9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 10:41:55.145366 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cf3bdf1-6dee-431e-be5b-1cac1c591fb9 None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:41:55.145647 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cf3bdf1-6dee-431e-be5b-1cac1c591fb9 None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:41:55.145923 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cf3bdf1-6dee-431e-be5b-1cac1c591fb9 None admin] PATH_INFO: `/v3/roles` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:41:55.146598 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cf3bdf1-6dee-431e-be5b-1cac1c591fb9 None admin] RBAC: Adding query filter params (name=service) {{(pid=80143) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 10:41:55.146902 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cf3bdf1-6dee-431e-be5b-1cac1c591fb9 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 10:41:55.148306 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cf3bdf1-6dee-431e-be5b-1cac1c591fb9 None admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 10:41:55.157259 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 235/484] 10.222.0.25 () {56 vars in 1103 bytes} [Sun Aug 7 10:41:54 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 10:41:55.162576 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e3450f2-da34-4203-a4d0-74c9eb5e1de3 None admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 10:41:55.302603 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e3450f2-da34-4203-a4d0-74c9eb5e1de3 None admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 10:41:55.345731 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91ea397d-3015-4673-9405-43e219def241 None None] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:41:55.346039 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91ea397d-3015-4673-9405-43e219def241 None None] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:41:55.346627 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91ea397d-3015-4673-9405-43e219def241 None None] PATH_INFO: `/` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:41:55.349685 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5654c04d-8382-4ca7-b263-0459cfd8c552 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 10:41:55.349685 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5654c04d-8382-4ca7-b263-0459cfd8c552 None admin] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:41:55.349873 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 236/485] 10.222.0.25 () {54 vars in 927 bytes} [Sun Aug 7 10:41:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 07 10:41:55.349945 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5654c04d-8382-4ca7-b263-0459cfd8c552 None admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:41:55.349945 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5654c04d-8382-4ca7-b263-0459cfd8c552 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:41:55.352752 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a791da4-25b9-43f0-96bb-572f657ffd85 None None] REQUEST_METHOD: `POST` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:41:55.353057 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a791da4-25b9-43f0-96bb-572f657ffd85 None None] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:41:55.353310 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a791da4-25b9-43f0-96bb-572f657ffd85 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:41:55.370371 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5654c04d-8382-4ca7-b263-0459cfd8c552 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 10:41:55.372046 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.auth.core [None req-6a791da4-25b9-43f0-96bb-572f657ffd85 None None] MFA Rules not processed for user `bac4e6b76fe64217ac8fa44e71cffe6a`. Rule list: `[]` (Enabled: `True`). {{(pid=80143) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 10:41:55.372555 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5654c04d-8382-4ca7-b263-0459cfd8c552 None admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 10:41:55.374940 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-6a791da4-25b9-43f0-96bb-572f657ffd85 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 10:41:55.390805 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application [None req-5654c04d-8382-4ca7-b263-0459cfd8c552 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 07 10:41:55.390805 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 07 10:41:55.390805 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 07 10:41:55.390805 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 07 10:41:55.390805 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 07 10:41:55.390805 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 07 10:41:55.390805 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 07 10:41:55.390805 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 07 10:41:55.390805 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 07 10:41:55.390805 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 07 10:41:55.390805 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 07 10:41:55.390805 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 07 10:41:55.390805 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 07 10:41:55.390805 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 07 10:41:55.390805 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 07 10:41:55.390805 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 07 10:41:55.391757 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 07 10:41:55.391757 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 07 10:41:55.391757 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 07 10:41:55.391757 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 07 10:41:55.391757 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 07 10:41:55.391757 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self.get_or_create( Aug 07 10:41:55.391757 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 07 10:41:55.391757 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application with Lock( Aug 07 10:41:55.391757 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 07 10:41:55.391757 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self._enter() Aug 07 10:41:55.391757 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 07 10:41:55.391757 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 07 10:41:55.391757 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 07 10:41:55.391757 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self.creator() Aug 07 10:41:55.391757 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 07 10:41:55.391757 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application created_value = creator( Aug 07 10:41:55.391757 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 07 10:41:55.392583 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 07 10:41:55.392583 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 07 10:41:55.392583 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application Aug 07 10:41:55.394160 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 250/486] 10.222.0.25 () {56 vars in 1108 bytes} [Sun Aug 7 10:41:55 2022] GET /identity/v3/domains/Default => generated 87 bytes in 232 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 07 10:41:55.399847 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5654c04d-8382-4ca7-b263-0459cfd8c552 None admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 10:41:55.477382 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=100 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 10:41:55.515025 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 237/487] 10.222.0.25 () {58 vars in 1024 bytes} [Sun Aug 7 10:41:55 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 157 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 07 10:41:55.533478 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a791da4-25b9-43f0-96bb-572f657ffd85 None None] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 10:41:55.569476 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5654c04d-8382-4ca7-b263-0459cfd8c552 None admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 10:41:55.607833 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65ecd12a-7993-424b-88c0-a481753703a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 10:41:55.608774 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65ecd12a-7993-424b-88c0-a481753703a0 None admin] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:41:55.609249 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65ecd12a-7993-424b-88c0-a481753703a0 None admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:41:55.609709 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65ecd12a-7993-424b-88c0-a481753703a0 None admin] PATH_INFO: `/v3/domains` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:41:55.610562 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65ecd12a-7993-424b-88c0-a481753703a0 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80144) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 10:41:55.611130 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65ecd12a-7993-424b-88c0-a481753703a0 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 10:41:55.613088 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65ecd12a-7993-424b-88c0-a481753703a0 None admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 10:41:55.628206 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 251/488] 10.222.0.25 () {56 vars in 1109 bytes} [Sun Aug 7 10:41:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 229 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 10:41:55.630156 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-833ab5b7-853e-4634-b43f-a592fffa616b None None] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:41:55.630710 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-833ab5b7-853e-4634-b43f-a592fffa616b None None] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:41:55.631109 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-833ab5b7-853e-4634-b43f-a592fffa616b None None] PATH_INFO: `/` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:41:55.632501 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 252/489] 10.222.0.25 () {54 vars in 927 bytes} [Sun Aug 7 10:41:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 07 10:41:55.637932 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-833ab5b7-853e-4634-b43f-a592fffa616b None None] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 10:41:55.652504 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a791da4-25b9-43f0-96bb-572f657ffd85 None None] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 10:41:55.655470 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=100 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 10:41:55.695888 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5464cf20-1e1f-4096-b9ce-2c558c247775 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 10:41:55.697203 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5464cf20-1e1f-4096-b9ce-2c558c247775 None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:41:55.698580 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5464cf20-1e1f-4096-b9ce-2c558c247775 None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:41:55.699028 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5464cf20-1e1f-4096-b9ce-2c558c247775 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:41:55.712595 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5464cf20-1e1f-4096-b9ce-2c558c247775 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 07 10:41:55.720700 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application [None req-5464cf20-1e1f-4096-b9ce-2c558c247775 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 07 10:41:55.720700 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 07 10:41:55.720700 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 07 10:41:55.720700 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 07 10:41:55.720700 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 07 10:41:55.720700 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 07 10:41:55.720700 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 07 10:41:55.720700 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 07 10:41:55.720700 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 07 10:41:55.720700 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 07 10:41:55.720700 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 07 10:41:55.720700 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 07 10:41:55.720700 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 07 10:41:55.720700 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 07 10:41:55.720700 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 07 10:41:55.720700 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 07 10:41:55.721762 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 07 10:41:55.721762 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 07 10:41:55.721762 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 07 10:41:55.721762 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 07 10:41:55.721762 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 07 10:41:55.721762 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 07 10:41:55.721762 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 07 10:41:55.721762 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 07 10:41:55.721762 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 07 10:41:55.721762 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.get_or_create( Aug 07 10:41:55.721762 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 07 10:41:55.721762 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application with Lock( Aug 07 10:41:55.721762 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 07 10:41:55.721762 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self._enter() Aug 07 10:41:55.721762 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 07 10:41:55.721762 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 07 10:41:55.722546 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 07 10:41:55.722546 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.creator() Aug 07 10:41:55.722546 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 07 10:41:55.722546 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application created_value = creator( Aug 07 10:41:55.722546 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 07 10:41:55.722546 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 07 10:41:55.722546 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 07 10:41:55.722546 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 07 10:41:55.722546 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 07 10:41:55.722546 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 07 10:41:55.722546 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 07 10:41:55.722546 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application Aug 07 10:41:55.724423 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 238/490] 10.222.0.25 () {56 vars in 1102 bytes} [Sun Aug 7 10:41:55 2022] GET /identity/v3/roles/service => generated 85 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 07 10:41:55.727115 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-517a8d55-ae9e-4a22-8303-2a0336dabc5b None None] REQUEST_METHOD: `POST` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:41:55.727115 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-517a8d55-ae9e-4a22-8303-2a0336dabc5b None None] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:41:55.727115 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-517a8d55-ae9e-4a22-8303-2a0336dabc5b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:41:55.751896 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.auth.core [None req-517a8d55-ae9e-4a22-8303-2a0336dabc5b None None] MFA Rules not processed for user `bac4e6b76fe64217ac8fa44e71cffe6a`. Rule list: `[]` (Enabled: `True`). {{(pid=80143) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 10:41:55.754043 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-517a8d55-ae9e-4a22-8303-2a0336dabc5b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 10:41:55.768609 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-833ab5b7-853e-4634-b43f-a592fffa616b None None] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 10:41:55.786697 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=100 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 10:41:55.816339 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dba0606-d5e9-4c24-9487-8e07709016b9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 10:41:55.817204 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dba0606-d5e9-4c24-9487-8e07709016b9 None admin] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:41:55.817666 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dba0606-d5e9-4c24-9487-8e07709016b9 None admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:41:55.818109 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dba0606-d5e9-4c24-9487-8e07709016b9 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:41:55.826848 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dba0606-d5e9-4c24-9487-8e07709016b9 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 10:41:55.828634 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dba0606-d5e9-4c24-9487-8e07709016b9 None admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 10:41:55.842574 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application [None req-1dba0606-d5e9-4c24-9487-8e07709016b9 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Aug 07 10:41:55.842574 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 07 10:41:55.842574 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 07 10:41:55.842574 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 07 10:41:55.842574 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 07 10:41:55.842574 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 07 10:41:55.842574 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 07 10:41:55.842574 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 07 10:41:55.842574 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 07 10:41:55.842574 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 07 10:41:55.842574 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 07 10:41:55.842574 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 07 10:41:55.842574 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 07 10:41:55.842574 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 07 10:41:55.842574 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 07 10:41:55.842574 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 07 10:41:55.843701 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 07 10:41:55.843701 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 07 10:41:55.843701 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 07 10:41:55.843701 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 07 10:41:55.843701 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 07 10:41:55.843701 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 07 10:41:55.843701 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 07 10:41:55.843701 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 07 10:41:55.843701 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 07 10:41:55.843701 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self.get_or_create( Aug 07 10:41:55.843701 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 07 10:41:55.843701 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application with Lock( Aug 07 10:41:55.843701 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 07 10:41:55.843701 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self._enter() Aug 07 10:41:55.843701 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 07 10:41:55.843701 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 07 10:41:55.843701 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 07 10:41:55.843701 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self.creator() Aug 07 10:41:55.844822 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 07 10:41:55.844822 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application created_value = creator( Aug 07 10:41:55.844822 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 07 10:41:55.844822 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 07 10:41:55.844822 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 07 10:41:55.844822 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 07 10:41:55.844822 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 07 10:41:55.844822 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 07 10:41:55.844822 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Aug 07 10:41:55.844822 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application Aug 07 10:41:55.846139 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 253/491] 10.222.0.25 () {56 vars in 1093 bytes} [Sun Aug 7 10:41:55 2022] GET /identity/v3/users/nova => generated 82 bytes in 213 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 07 10:41:55.847115 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dba0606-d5e9-4c24-9487-8e07709016b9 None admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 10:41:55.847627 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 239/492] 10.222.0.25 () {58 vars in 1024 bytes} [Sun Aug 7 10:41:55 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 07 10:41:55.854660 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-517a8d55-ae9e-4a22-8303-2a0336dabc5b None None] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 10:41:55.976102 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dba0606-d5e9-4c24-9487-8e07709016b9 None admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 10:41:55.985817 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=100 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 10:41:55.993873 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-517a8d55-ae9e-4a22-8303-2a0336dabc5b None None] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 10:41:56.020719 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8730522a-05d5-4a84-886d-91a29d625a99 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 10:41:56.021337 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8730522a-05d5-4a84-886d-91a29d625a99 None admin] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:41:56.021616 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8730522a-05d5-4a84-886d-91a29d625a99 None admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:41:56.021980 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8730522a-05d5-4a84-886d-91a29d625a99 None admin] PATH_INFO: `/v3/roles` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:41:56.022541 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8730522a-05d5-4a84-886d-91a29d625a99 None admin] RBAC: Adding query filter params (name=service) {{(pid=80144) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 10:41:56.022848 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8730522a-05d5-4a84-886d-91a29d625a99 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 10:41:56.024251 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8730522a-05d5-4a84-886d-91a29d625a99 None admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 10:41:56.029345 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85624ebf-a3cd-4269-a08b-a18200b50acb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 10:41:56.029958 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85624ebf-a3cd-4269-a08b-a18200b50acb None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:41:56.030245 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85624ebf-a3cd-4269-a08b-a18200b50acb None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:41:56.030521 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85624ebf-a3cd-4269-a08b-a18200b50acb None admin] PATH_INFO: `/v3/users` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:41:56.031283 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85624ebf-a3cd-4269-a08b-a18200b50acb None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80143) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 10:41:56.031590 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85624ebf-a3cd-4269-a08b-a18200b50acb None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 10:41:56.037295 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 254/493] 10.222.0.25 () {56 vars in 1103 bytes} [Sun Aug 7 10:41:55 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 10:41:56.037425 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85624ebf-a3cd-4269-a08b-a18200b50acb None admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 10:41:56.037502 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8730522a-05d5-4a84-886d-91a29d625a99 None admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 10:41:56.069588 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=100 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 10:41:56.070579 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 240/494] 10.222.0.25 () {56 vars in 1133 bytes} [Sun Aug 7 10:41:55 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 219 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 10:41:56.071963 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9c07c05-bb04-4e90-92d6-b06d7eb2c18c None None] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:41:56.072245 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9c07c05-bb04-4e90-92d6-b06d7eb2c18c None None] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:41:56.073474 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9c07c05-bb04-4e90-92d6-b06d7eb2c18c None None] PATH_INFO: `/` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:41:56.079594 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 241/495] 10.222.0.25 () {54 vars in 927 bytes} [Sun Aug 7 10:41:56 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 07 10:41:56.080298 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9c07c05-bb04-4e90-92d6-b06d7eb2c18c None None] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 10:41:56.149927 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8730522a-05d5-4a84-886d-91a29d625a99 None admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 10:41:56.184632 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3def0383-afa6-4f05-ba41-8f201f0fc103 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 10:41:56.185270 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3def0383-afa6-4f05-ba41-8f201f0fc103 None admin] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:41:56.185552 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3def0383-afa6-4f05-ba41-8f201f0fc103 None admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:41:56.185822 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3def0383-afa6-4f05-ba41-8f201f0fc103 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:41:56.189127 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9c07c05-bb04-4e90-92d6-b06d7eb2c18c None None] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 10:41:56.195770 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3def0383-afa6-4f05-ba41-8f201f0fc103 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 07 10:41:56.201113 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application [None req-3def0383-afa6-4f05-ba41-8f201f0fc103 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 07 10:41:56.201113 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 07 10:41:56.201113 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 07 10:41:56.201113 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 07 10:41:56.201113 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 07 10:41:56.201113 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 07 10:41:56.201113 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 07 10:41:56.201113 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 07 10:41:56.201113 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 07 10:41:56.201113 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 07 10:41:56.201113 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 07 10:41:56.201113 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 07 10:41:56.201113 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 07 10:41:56.201113 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 07 10:41:56.201113 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 07 10:41:56.201113 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 07 10:41:56.202067 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 07 10:41:56.202067 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 07 10:41:56.202067 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 07 10:41:56.202067 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 07 10:41:56.202067 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 07 10:41:56.202067 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 07 10:41:56.202067 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 07 10:41:56.202067 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 07 10:41:56.202067 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 07 10:41:56.202067 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self.get_or_create( Aug 07 10:41:56.202067 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 07 10:41:56.202067 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application with Lock( Aug 07 10:41:56.202067 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 07 10:41:56.202067 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self._enter() Aug 07 10:41:56.202067 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 07 10:41:56.202067 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 07 10:41:56.202834 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 07 10:41:56.202834 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self.creator() Aug 07 10:41:56.202834 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 07 10:41:56.202834 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application created_value = creator( Aug 07 10:41:56.202834 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 07 10:41:56.202834 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 07 10:41:56.202834 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 07 10:41:56.202834 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 07 10:41:56.202834 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 07 10:41:56.202834 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 07 10:41:56.202834 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 07 10:41:56.202834 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application Aug 07 10:41:56.202834 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 255/496] 10.222.0.25 () {56 vars in 1102 bytes} [Sun Aug 7 10:41:56 2022] GET /identity/v3/roles/service => generated 85 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 07 10:41:56.203564 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3def0383-afa6-4f05-ba41-8f201f0fc103 None admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 10:41:56.234195 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-236a1e0f-51ae-457d-b2d9-85031c0865bd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 10:41:56.235258 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-236a1e0f-51ae-457d-b2d9-85031c0865bd None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:41:56.235763 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-236a1e0f-51ae-457d-b2d9-85031c0865bd None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:41:56.236222 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-236a1e0f-51ae-457d-b2d9-85031c0865bd None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:41:56.247133 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-236a1e0f-51ae-457d-b2d9-85031c0865bd None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 10:41:56.248883 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-236a1e0f-51ae-457d-b2d9-85031c0865bd None admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 10:41:56.264678 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application [None req-236a1e0f-51ae-457d-b2d9-85031c0865bd None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 07 10:41:56.264678 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 07 10:41:56.264678 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 07 10:41:56.264678 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 07 10:41:56.264678 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 07 10:41:56.264678 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 07 10:41:56.264678 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 07 10:41:56.264678 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 07 10:41:56.264678 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 07 10:41:56.264678 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 07 10:41:56.264678 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 07 10:41:56.264678 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 07 10:41:56.264678 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 07 10:41:56.264678 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 07 10:41:56.264678 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 07 10:41:56.264678 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 07 10:41:56.265760 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 07 10:41:56.265760 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 07 10:41:56.265760 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 07 10:41:56.265760 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 07 10:41:56.265760 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 07 10:41:56.265760 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.get_or_create( Aug 07 10:41:56.265760 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 07 10:41:56.265760 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application with Lock( Aug 07 10:41:56.265760 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 07 10:41:56.265760 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self._enter() Aug 07 10:41:56.265760 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 07 10:41:56.265760 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 07 10:41:56.265760 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 07 10:41:56.265760 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.creator() Aug 07 10:41:56.265760 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 07 10:41:56.265760 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application created_value = creator( Aug 07 10:41:56.265760 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 07 10:41:56.266763 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 07 10:41:56.266763 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 07 10:41:56.266763 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application Aug 07 10:41:56.268448 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 242/497] 10.222.0.25 () {56 vars in 1108 bytes} [Sun Aug 7 10:41:56 2022] GET /identity/v3/domains/Default => generated 87 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 07 10:41:56.269815 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74eaa91f-359d-4348-97a0-2d494b75bfae None None] REQUEST_METHOD: `POST` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:41:56.270095 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74eaa91f-359d-4348-97a0-2d494b75bfae None None] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:41:56.270363 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74eaa91f-359d-4348-97a0-2d494b75bfae None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:41:56.278118 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=100 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 10:41:56.290406 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.auth.core [None req-74eaa91f-359d-4348-97a0-2d494b75bfae None None] MFA Rules not processed for user `bac4e6b76fe64217ac8fa44e71cffe6a`. Rule list: `[]` (Enabled: `True`). {{(pid=80143) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 10:41:56.292186 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-74eaa91f-359d-4348-97a0-2d494b75bfae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 10:41:56.306590 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=100 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 10:41:56.316368 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3def0383-afa6-4f05-ba41-8f201f0fc103 None admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 10:41:56.350697 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba815f02-e3fe-43e5-8cc5-449a1e99f231 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 10:41:56.351361 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba815f02-e3fe-43e5-8cc5-449a1e99f231 None admin] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:41:56.351648 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba815f02-e3fe-43e5-8cc5-449a1e99f231 None admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:41:56.351935 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba815f02-e3fe-43e5-8cc5-449a1e99f231 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:41:56.383288 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 243/498] 10.222.0.25 () {58 vars in 1024 bytes} [Sun Aug 7 10:41:56 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 07 10:41:56.384463 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74eaa91f-359d-4348-97a0-2d494b75bfae None None] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 10:41:56.431730 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba815f02-e3fe-43e5-8cc5-449a1e99f231 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 10:41:56.433306 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba815f02-e3fe-43e5-8cc5-449a1e99f231 None admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 10:41:56.493704 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74eaa91f-359d-4348-97a0-2d494b75bfae None None] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 10:41:56.529674 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=100 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 10:41:56.537823 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15fa249a-e58a-4d70-8315-6265a8895be4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 10:41:56.538437 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15fa249a-e58a-4d70-8315-6265a8895be4 None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:41:56.538714 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15fa249a-e58a-4d70-8315-6265a8895be4 None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:41:56.539008 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15fa249a-e58a-4d70-8315-6265a8895be4 None admin] PATH_INFO: `/v3/roles` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:41:56.539674 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15fa249a-e58a-4d70-8315-6265a8895be4 None admin] RBAC: Adding query filter params (name=service) {{(pid=80143) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 10:41:56.539971 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15fa249a-e58a-4d70-8315-6265a8895be4 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 10:41:56.545866 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 256/499] 10.222.0.25 () {58 vars in 1282 bytes} [Sun Aug 7 10:41:56 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 338 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 07 10:41:56.546020 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15fa249a-e58a-4d70-8315-6265a8895be4 None admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 10:41:56.546096 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba815f02-e3fe-43e5-8cc5-449a1e99f231 None admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 10:41:56.546186 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=100 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 10:41:56.551393 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 244/500] 10.222.0.25 () {56 vars in 1103 bytes} [Sun Aug 7 10:41:56 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 10:41:56.555885 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15fa249a-e58a-4d70-8315-6265a8895be4 None admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 10:41:56.651433 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba815f02-e3fe-43e5-8cc5-449a1e99f231 None admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 10:41:56.662168 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15fa249a-e58a-4d70-8315-6265a8895be4 None admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 10:41:56.686354 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eccc18c3-3026-4ff2-84e1-71a32c8577ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 10:41:56.686985 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eccc18c3-3026-4ff2-84e1-71a32c8577ec None admin] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:41:56.687269 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eccc18c3-3026-4ff2-84e1-71a32c8577ec None admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:41:56.687546 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eccc18c3-3026-4ff2-84e1-71a32c8577ec None admin] PATH_INFO: `/v3/domains` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:41:56.688187 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eccc18c3-3026-4ff2-84e1-71a32c8577ec None admin] RBAC: Adding query filter params (name=Default) {{(pid=80144) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 10:41:56.688492 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eccc18c3-3026-4ff2-84e1-71a32c8577ec None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 10:41:56.689887 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eccc18c3-3026-4ff2-84e1-71a32c8577ec None admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 10:41:56.697985 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-838cb8d8-11a8-40c2-8daf-c689d0d4783d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 10:41:56.698638 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-838cb8d8-11a8-40c2-8daf-c689d0d4783d None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:41:56.698945 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-838cb8d8-11a8-40c2-8daf-c689d0d4783d None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:41:56.699224 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-838cb8d8-11a8-40c2-8daf-c689d0d4783d None admin] PATH_INFO: `/v3/roles/service` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:41:56.700945 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 257/501] 10.222.0.25 () {56 vars in 1109 bytes} [Sun Aug 7 10:41:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 10:41:56.701730 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eccc18c3-3026-4ff2-84e1-71a32c8577ec None admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 10:41:56.714297 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-838cb8d8-11a8-40c2-8daf-c689d0d4783d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 07 10:41:56.719549 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application [None req-838cb8d8-11a8-40c2-8daf-c689d0d4783d None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 07 10:41:56.719549 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 07 10:41:56.719549 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 07 10:41:56.719549 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 07 10:41:56.719549 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 07 10:41:56.719549 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 07 10:41:56.719549 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 07 10:41:56.719549 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 07 10:41:56.719549 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 07 10:41:56.719549 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 07 10:41:56.719549 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 07 10:41:56.719549 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 07 10:41:56.719549 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 07 10:41:56.719549 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 07 10:41:56.719549 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 07 10:41:56.719549 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 07 10:41:56.720617 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 07 10:41:56.720617 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 07 10:41:56.720617 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 07 10:41:56.720617 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 07 10:41:56.720617 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 07 10:41:56.720617 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 07 10:41:56.720617 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 07 10:41:56.720617 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 07 10:41:56.720617 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 07 10:41:56.720617 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.get_or_create( Aug 07 10:41:56.720617 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 07 10:41:56.720617 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application with Lock( Aug 07 10:41:56.720617 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 07 10:41:56.720617 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self._enter() Aug 07 10:41:56.720617 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 07 10:41:56.720617 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 07 10:41:56.721595 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 07 10:41:56.721595 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.creator() Aug 07 10:41:56.721595 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 07 10:41:56.721595 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application created_value = creator( Aug 07 10:41:56.721595 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 07 10:41:56.721595 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 07 10:41:56.721595 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 07 10:41:56.721595 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 07 10:41:56.721595 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 07 10:41:56.721595 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 07 10:41:56.721595 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 07 10:41:56.721595 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application Aug 07 10:41:56.721595 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 245/502] 10.222.0.25 () {56 vars in 1102 bytes} [Sun Aug 7 10:41:56 2022] GET /identity/v3/roles/service => generated 85 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 07 10:41:56.721595 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-838cb8d8-11a8-40c2-8daf-c689d0d4783d None admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 10:41:56.755400 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=100 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 10:41:56.785655 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=100 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 10:41:56.806700 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eccc18c3-3026-4ff2-84e1-71a32c8577ec None admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 10:41:56.828419 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-838cb8d8-11a8-40c2-8daf-c689d0d4783d None admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 10:41:56.839483 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a46052c-341b-4f55-8805-23af815f9830 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 10:41:56.840117 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a46052c-341b-4f55-8805-23af815f9830 None admin] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:41:56.840402 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a46052c-341b-4f55-8805-23af815f9830 None admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:41:56.840676 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a46052c-341b-4f55-8805-23af815f9830 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:41:56.851510 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a46052c-341b-4f55-8805-23af815f9830 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 10:41:56.852952 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a46052c-341b-4f55-8805-23af815f9830 None admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 10:41:56.864055 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ab42c1e-61ec-421a-8cc9-9fd14b8b94eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 10:41:56.864685 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ab42c1e-61ec-421a-8cc9-9fd14b8b94eb None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:41:56.864963 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ab42c1e-61ec-421a-8cc9-9fd14b8b94eb None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:41:56.865237 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ab42c1e-61ec-421a-8cc9-9fd14b8b94eb None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:41:56.867906 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application [None req-2a46052c-341b-4f55-8805-23af815f9830 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 07 10:41:56.867906 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 07 10:41:56.867906 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 07 10:41:56.867906 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 07 10:41:56.867906 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 07 10:41:56.867906 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 07 10:41:56.867906 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 07 10:41:56.867906 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 07 10:41:56.867906 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 07 10:41:56.867906 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 07 10:41:56.867906 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 07 10:41:56.867906 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 07 10:41:56.867906 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 07 10:41:56.867906 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 07 10:41:56.867906 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 07 10:41:56.867906 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 07 10:41:56.868797 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 07 10:41:56.868797 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 07 10:41:56.868797 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 07 10:41:56.868797 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 07 10:41:56.868797 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 07 10:41:56.868797 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self.get_or_create( Aug 07 10:41:56.868797 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 07 10:41:56.868797 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application with Lock( Aug 07 10:41:56.868797 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 07 10:41:56.868797 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self._enter() Aug 07 10:41:56.868797 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 07 10:41:56.868797 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 07 10:41:56.868797 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 07 10:41:56.868797 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self.creator() Aug 07 10:41:56.868797 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 07 10:41:56.868797 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application created_value = creator( Aug 07 10:41:56.868797 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 07 10:41:56.869710 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 07 10:41:56.869710 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 07 10:41:56.869710 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application Aug 07 10:41:56.869710 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 258/503] 10.222.0.25 () {56 vars in 1108 bytes} [Sun Aug 7 10:41:56 2022] GET /identity/v3/domains/Default => generated 87 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 07 10:41:56.870339 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57d210de-da8e-4ee7-a558-3677734626bd None None] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:41:56.870623 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57d210de-da8e-4ee7-a558-3677734626bd None None] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:41:56.870892 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57d210de-da8e-4ee7-a558-3677734626bd None None] PATH_INFO: `/` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:41:56.872124 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 259/504] 10.222.0.25 () {54 vars in 927 bytes} [Sun Aug 7 10:41:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 07 10:41:56.872811 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57d210de-da8e-4ee7-a558-3677734626bd None None] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 10:41:56.877068 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ab42c1e-61ec-421a-8cc9-9fd14b8b94eb None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 10:41:56.878619 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ab42c1e-61ec-421a-8cc9-9fd14b8b94eb None admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 10:41:56.893175 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application [None req-3ab42c1e-61ec-421a-8cc9-9fd14b8b94eb None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 07 10:41:56.893175 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 07 10:41:56.893175 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 07 10:41:56.893175 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 07 10:41:56.893175 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 07 10:41:56.893175 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 07 10:41:56.893175 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 07 10:41:56.893175 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 07 10:41:56.893175 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 07 10:41:56.893175 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 07 10:41:56.893175 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 07 10:41:56.893175 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 07 10:41:56.893175 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 07 10:41:56.893175 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 07 10:41:56.893175 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 07 10:41:56.893175 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 07 10:41:56.894336 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 07 10:41:56.894336 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 07 10:41:56.894336 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 07 10:41:56.894336 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 07 10:41:56.894336 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 07 10:41:56.894336 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.get_or_create( Aug 07 10:41:56.894336 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 07 10:41:56.894336 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application with Lock( Aug 07 10:41:56.894336 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 07 10:41:56.894336 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self._enter() Aug 07 10:41:56.894336 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 07 10:41:56.894336 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 07 10:41:56.894336 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 07 10:41:56.894336 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.creator() Aug 07 10:41:56.894336 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 07 10:41:56.894336 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application created_value = creator( Aug 07 10:41:56.894336 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 07 10:41:56.895389 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 07 10:41:56.895389 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 07 10:41:56.895389 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application Aug 07 10:41:56.895389 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 246/505] 10.222.0.25 () {56 vars in 1108 bytes} [Sun Aug 7 10:41:56 2022] GET /identity/v3/domains/Default => generated 87 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 07 10:41:56.895389 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ab42c1e-61ec-421a-8cc9-9fd14b8b94eb None admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 10:41:56.978653 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57d210de-da8e-4ee7-a558-3677734626bd None None] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 10:41:56.981195 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=100 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 10:41:56.995215 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ab42c1e-61ec-421a-8cc9-9fd14b8b94eb None admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 10:41:57.004085 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=100 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 10:41:57.015741 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bd2d051-18f0-40e2-aa05-3bcbf05322e3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 10:41:57.016421 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bd2d051-18f0-40e2-aa05-3bcbf05322e3 None admin] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:41:57.016706 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bd2d051-18f0-40e2-aa05-3bcbf05322e3 None admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:41:57.016981 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bd2d051-18f0-40e2-aa05-3bcbf05322e3 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:41:57.023921 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bd2d051-18f0-40e2-aa05-3bcbf05322e3 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 10:41:57.025339 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bd2d051-18f0-40e2-aa05-3bcbf05322e3 None admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 10:41:57.030037 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86572a74-4cba-464b-9e73-e2471d65be4a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 10:41:57.030673 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86572a74-4cba-464b-9e73-e2471d65be4a None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:41:57.030982 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86572a74-4cba-464b-9e73-e2471d65be4a None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:41:57.031268 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86572a74-4cba-464b-9e73-e2471d65be4a None admin] PATH_INFO: `/v3/roles` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:41:57.031934 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86572a74-4cba-464b-9e73-e2471d65be4a None admin] RBAC: Adding query filter params (name=service) {{(pid=80143) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 10:41:57.032236 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86572a74-4cba-464b-9e73-e2471d65be4a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 10:41:57.033594 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86572a74-4cba-464b-9e73-e2471d65be4a None admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 10:41:57.036893 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application [None req-9bd2d051-18f0-40e2-aa05-3bcbf05322e3 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Aug 07 10:41:57.036893 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 07 10:41:57.036893 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 07 10:41:57.036893 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 07 10:41:57.036893 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 07 10:41:57.036893 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 07 10:41:57.036893 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 07 10:41:57.036893 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 07 10:41:57.036893 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 07 10:41:57.036893 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 07 10:41:57.036893 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 07 10:41:57.036893 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 07 10:41:57.036893 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 07 10:41:57.036893 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 07 10:41:57.036893 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 07 10:41:57.036893 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 07 10:41:57.037792 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 07 10:41:57.037792 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 07 10:41:57.037792 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 07 10:41:57.037792 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 07 10:41:57.037792 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 07 10:41:57.037792 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 07 10:41:57.037792 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 07 10:41:57.037792 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 07 10:41:57.037792 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 07 10:41:57.037792 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self.get_or_create( Aug 07 10:41:57.037792 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 07 10:41:57.037792 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application with Lock( Aug 07 10:41:57.037792 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 07 10:41:57.037792 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self._enter() Aug 07 10:41:57.037792 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 07 10:41:57.037792 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 07 10:41:57.037792 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 07 10:41:57.037792 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self.creator() Aug 07 10:41:57.038696 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 07 10:41:57.038696 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application created_value = creator( Aug 07 10:41:57.038696 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 07 10:41:57.038696 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 07 10:41:57.038696 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 07 10:41:57.038696 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 07 10:41:57.038696 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 07 10:41:57.038696 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 07 10:41:57.038696 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Aug 07 10:41:57.038696 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application Aug 07 10:41:57.038696 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 260/506] 10.222.0.25 () {56 vars in 1099 bytes} [Sun Aug 7 10:41:56 2022] GET /identity/v3/users/cinder => generated 84 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 07 10:41:57.038696 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bd2d051-18f0-40e2-aa05-3bcbf05322e3 None admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 10:41:57.043226 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 247/507] 10.222.0.25 () {56 vars in 1103 bytes} [Sun Aug 7 10:41:56 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 10:41:57.044602 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26a6de48-419f-4d34-9dfa-c922c74f14d0 None None] REQUEST_METHOD: `POST` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:41:57.044963 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26a6de48-419f-4d34-9dfa-c922c74f14d0 None None] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:41:57.045240 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26a6de48-419f-4d34-9dfa-c922c74f14d0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:41:57.062184 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.auth.core [None req-26a6de48-419f-4d34-9dfa-c922c74f14d0 None None] MFA Rules not processed for user `bac4e6b76fe64217ac8fa44e71cffe6a`. Rule list: `[]` (Enabled: `True`). {{(pid=80143) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 10:41:57.063394 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-26a6de48-419f-4d34-9dfa-c922c74f14d0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 10:41:57.128986 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 248/508] 10.222.0.25 () {58 vars in 1024 bytes} [Sun Aug 7 10:41:57 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 07 10:41:57.130745 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26a6de48-419f-4d34-9dfa-c922c74f14d0 None None] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 10:41:57.148307 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bd2d051-18f0-40e2-aa05-3bcbf05322e3 None admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 10:41:57.181932 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28596963-d4d2-4f42-a932-55e523568406 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 10:41:57.182540 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28596963-d4d2-4f42-a932-55e523568406 None admin] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:41:57.182829 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28596963-d4d2-4f42-a932-55e523568406 None admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:41:57.183133 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28596963-d4d2-4f42-a932-55e523568406 None admin] PATH_INFO: `/v3/domains` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:41:57.183776 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28596963-d4d2-4f42-a932-55e523568406 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80144) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 10:41:57.184083 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28596963-d4d2-4f42-a932-55e523568406 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 10:41:57.185801 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28596963-d4d2-4f42-a932-55e523568406 None admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 10:41:57.198060 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 261/509] 10.222.0.25 () {56 vars in 1109 bytes} [Sun Aug 7 10:41:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 10:41:57.198833 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28596963-d4d2-4f42-a932-55e523568406 None admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 10:41:57.225994 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=100 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 10:41:57.229790 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=100 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 10:41:57.232813 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26a6de48-419f-4d34-9dfa-c922c74f14d0 None None] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 10:41:57.267355 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2127adc-b3a8-4640-a480-27818b708879 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 10:41:57.267960 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2127adc-b3a8-4640-a480-27818b708879 None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:41:57.268237 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2127adc-b3a8-4640-a480-27818b708879 None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:41:57.268506 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2127adc-b3a8-4640-a480-27818b708879 None admin] PATH_INFO: `/v3/domains` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:41:57.269143 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2127adc-b3a8-4640-a480-27818b708879 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80143) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 10:41:57.269439 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2127adc-b3a8-4640-a480-27818b708879 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 10:41:57.270790 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2127adc-b3a8-4640-a480-27818b708879 None admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 10:41:57.281892 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 249/510] 10.222.0.25 () {56 vars in 1109 bytes} [Sun Aug 7 10:41:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 10:41:57.284191 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2127adc-b3a8-4640-a480-27818b708879 None admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 10:41:57.301696 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28596963-d4d2-4f42-a932-55e523568406 None admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 10:41:57.335293 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ef161b6-aefa-4018-95c3-a6277c35cdbb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 10:41:57.335896 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ef161b6-aefa-4018-95c3-a6277c35cdbb None admin] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:41:57.336177 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ef161b6-aefa-4018-95c3-a6277c35cdbb None admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:41:57.336450 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ef161b6-aefa-4018-95c3-a6277c35cdbb None admin] PATH_INFO: `/v3/users` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:41:57.337174 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ef161b6-aefa-4018-95c3-a6277c35cdbb None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=80144) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 10:41:57.337481 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ef161b6-aefa-4018-95c3-a6277c35cdbb None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 10:41:57.338912 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ef161b6-aefa-4018-95c3-a6277c35cdbb None admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 10:41:57.376125 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 262/511] 10.222.0.25 () {56 vars in 1137 bytes} [Sun Aug 7 10:41:57 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 10:41:57.377055 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ef161b6-aefa-4018-95c3-a6277c35cdbb None admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 10:41:57.396890 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2127adc-b3a8-4640-a480-27818b708879 None admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 10:41:57.432679 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4946f32-7111-4c80-8eca-69b00e20e566 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 10:41:57.433678 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4946f32-7111-4c80-8eca-69b00e20e566 None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:41:57.434072 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4946f32-7111-4c80-8eca-69b00e20e566 None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:41:57.434514 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4946f32-7111-4c80-8eca-69b00e20e566 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:41:57.445082 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4946f32-7111-4c80-8eca-69b00e20e566 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 10:41:57.446547 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4946f32-7111-4c80-8eca-69b00e20e566 None admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 10:41:57.460510 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application [None req-a4946f32-7111-4c80-8eca-69b00e20e566 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 07 10:41:57.460510 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 07 10:41:57.460510 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 07 10:41:57.460510 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 07 10:41:57.460510 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 07 10:41:57.460510 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 07 10:41:57.460510 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 07 10:41:57.460510 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 07 10:41:57.460510 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 07 10:41:57.460510 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 07 10:41:57.460510 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 07 10:41:57.460510 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 07 10:41:57.460510 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 07 10:41:57.460510 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 07 10:41:57.460510 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 07 10:41:57.460510 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 07 10:41:57.461535 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 07 10:41:57.461535 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 07 10:41:57.461535 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 07 10:41:57.461535 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 07 10:41:57.461535 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 07 10:41:57.461535 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.get_or_create( Aug 07 10:41:57.461535 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 07 10:41:57.461535 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application with Lock( Aug 07 10:41:57.461535 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 07 10:41:57.461535 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self._enter() Aug 07 10:41:57.461535 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 07 10:41:57.461535 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 07 10:41:57.461535 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 07 10:41:57.461535 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.creator() Aug 07 10:41:57.461535 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 07 10:41:57.461535 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application created_value = creator( Aug 07 10:41:57.461535 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 07 10:41:57.462391 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 07 10:41:57.462391 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 07 10:41:57.462391 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application Aug 07 10:41:57.462391 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 250/512] 10.222.0.25 () {56 vars in 1108 bytes} [Sun Aug 7 10:41:57 2022] GET /identity/v3/domains/Default => generated 87 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 07 10:41:57.462801 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4946f32-7111-4c80-8eca-69b00e20e566 None admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 10:41:57.477266 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=100 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 10:41:57.477507 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=100 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 10:41:57.498269 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ef161b6-aefa-4018-95c3-a6277c35cdbb None admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 10:41:57.531254 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fe7fe82-470d-4e39-b0f7-3784851ae469 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 10:41:57.531881 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fe7fe82-470d-4e39-b0f7-3784851ae469 None admin] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:41:57.532161 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fe7fe82-470d-4e39-b0f7-3784851ae469 None admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:41:57.532435 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fe7fe82-470d-4e39-b0f7-3784851ae469 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:41:57.543233 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fe7fe82-470d-4e39-b0f7-3784851ae469 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 10:41:57.544679 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fe7fe82-470d-4e39-b0f7-3784851ae469 None admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 10:41:57.558601 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application [None req-7fe7fe82-470d-4e39-b0f7-3784851ae469 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 07 10:41:57.558601 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 07 10:41:57.558601 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 07 10:41:57.558601 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 07 10:41:57.558601 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 07 10:41:57.558601 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 07 10:41:57.558601 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 07 10:41:57.558601 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 07 10:41:57.558601 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 07 10:41:57.558601 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 07 10:41:57.558601 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 07 10:41:57.558601 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 07 10:41:57.558601 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 07 10:41:57.558601 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 07 10:41:57.558601 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 07 10:41:57.558601 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 07 10:41:57.559588 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 07 10:41:57.559588 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 07 10:41:57.559588 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 07 10:41:57.559588 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 07 10:41:57.559588 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 07 10:41:57.559588 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self.get_or_create( Aug 07 10:41:57.559588 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 07 10:41:57.559588 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application with Lock( Aug 07 10:41:57.559588 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 07 10:41:57.559588 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self._enter() Aug 07 10:41:57.559588 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 07 10:41:57.559588 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 07 10:41:57.559588 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 07 10:41:57.559588 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self.creator() Aug 07 10:41:57.559588 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 07 10:41:57.559588 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application created_value = creator( Aug 07 10:41:57.559588 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 07 10:41:57.560425 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 07 10:41:57.560425 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 07 10:41:57.560425 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application Aug 07 10:41:57.560425 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 263/513] 10.222.0.25 () {56 vars in 1108 bytes} [Sun Aug 7 10:41:57 2022] GET /identity/v3/domains/Default => generated 87 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 07 10:41:57.560425 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fe7fe82-470d-4e39-b0f7-3784851ae469 None admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 10:41:57.568395 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4946f32-7111-4c80-8eca-69b00e20e566 None admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 10:41:57.601785 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01466ea5-f16b-4549-bbe6-460b273cdb3c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 10:41:57.602413 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01466ea5-f16b-4549-bbe6-460b273cdb3c None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:41:57.602697 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01466ea5-f16b-4549-bbe6-460b273cdb3c None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:41:57.603008 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01466ea5-f16b-4549-bbe6-460b273cdb3c None admin] PATH_INFO: `/v3/projects/service` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:41:57.610135 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01466ea5-f16b-4549-bbe6-460b273cdb3c None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 10:41:57.611640 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01466ea5-f16b-4549-bbe6-460b273cdb3c None admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 10:41:57.623908 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application [None req-01466ea5-f16b-4549-bbe6-460b273cdb3c None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 07 10:41:57.623908 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 07 10:41:57.623908 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 07 10:41:57.623908 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 07 10:41:57.623908 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 07 10:41:57.623908 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 07 10:41:57.623908 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 07 10:41:57.623908 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 07 10:41:57.623908 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 07 10:41:57.623908 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 07 10:41:57.623908 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 07 10:41:57.623908 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 07 10:41:57.623908 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 07 10:41:57.623908 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 07 10:41:57.623908 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 07 10:41:57.624758 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 07 10:41:57.624758 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 07 10:41:57.624758 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 07 10:41:57.624758 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 07 10:41:57.624758 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 07 10:41:57.624758 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 07 10:41:57.624758 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.get_or_create( Aug 07 10:41:57.624758 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 07 10:41:57.624758 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application with Lock( Aug 07 10:41:57.624758 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 07 10:41:57.624758 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self._enter() Aug 07 10:41:57.624758 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 07 10:41:57.624758 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 07 10:41:57.624758 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 07 10:41:57.624758 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.creator() Aug 07 10:41:57.624758 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 07 10:41:57.624758 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application created_value = creator( Aug 07 10:41:57.625622 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 07 10:41:57.625622 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 07 10:41:57.625622 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 07 10:41:57.625622 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 07 10:41:57.625622 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 07 10:41:57.625622 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 07 10:41:57.625622 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 07 10:41:57.625622 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application Aug 07 10:41:57.627060 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 251/514] 10.222.0.25 () {56 vars in 1111 bytes} [Sun Aug 7 10:41:57 2022] GET /identity/v3/projects/service => generated 88 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 07 10:41:57.628242 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01466ea5-f16b-4549-bbe6-460b273cdb3c None admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 10:41:57.657093 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fe7fe82-470d-4e39-b0f7-3784851ae469 None admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 10:41:57.686192 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=100 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 10:41:57.691632 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7f6e7e0-3301-485d-ab4f-36bb5216e66e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 10:41:57.692255 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7f6e7e0-3301-485d-ab4f-36bb5216e66e None admin] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:41:57.692532 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7f6e7e0-3301-485d-ab4f-36bb5216e66e None admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:41:57.692802 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7f6e7e0-3301-485d-ab4f-36bb5216e66e None admin] PATH_INFO: `/v3/users/glance` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:41:57.699349 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=100 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 10:41:57.700196 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7f6e7e0-3301-485d-ab4f-36bb5216e66e None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 10:41:57.701645 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7f6e7e0-3301-485d-ab4f-36bb5216e66e None admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 10:41:57.713751 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application [None req-c7f6e7e0-3301-485d-ab4f-36bb5216e66e None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Aug 07 10:41:57.713751 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 07 10:41:57.713751 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 07 10:41:57.713751 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 07 10:41:57.713751 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 07 10:41:57.713751 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 07 10:41:57.713751 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 07 10:41:57.713751 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 07 10:41:57.713751 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 07 10:41:57.713751 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 07 10:41:57.713751 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 07 10:41:57.713751 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 07 10:41:57.713751 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 07 10:41:57.713751 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 07 10:41:57.713751 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 07 10:41:57.713751 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 07 10:41:57.714826 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 07 10:41:57.714826 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 07 10:41:57.714826 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 07 10:41:57.714826 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 07 10:41:57.714826 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 07 10:41:57.714826 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 07 10:41:57.714826 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 07 10:41:57.714826 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 07 10:41:57.714826 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 07 10:41:57.714826 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self.get_or_create( Aug 07 10:41:57.714826 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 07 10:41:57.714826 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application with Lock( Aug 07 10:41:57.714826 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 07 10:41:57.714826 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self._enter() Aug 07 10:41:57.714826 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 07 10:41:57.714826 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 07 10:41:57.714826 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 07 10:41:57.714826 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self.creator() Aug 07 10:41:57.715764 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 07 10:41:57.715764 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application created_value = creator( Aug 07 10:41:57.715764 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 07 10:41:57.715764 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 07 10:41:57.715764 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 07 10:41:57.715764 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 07 10:41:57.715764 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 07 10:41:57.715764 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 07 10:41:57.715764 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Aug 07 10:41:57.715764 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application Aug 07 10:41:57.715764 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 264/515] 10.222.0.25 () {56 vars in 1099 bytes} [Sun Aug 7 10:41:57 2022] GET /identity/v3/users/glance => generated 84 bytes in 155 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 07 10:41:57.716327 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7f6e7e0-3301-485d-ab4f-36bb5216e66e None admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 10:41:57.737791 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01466ea5-f16b-4549-bbe6-460b273cdb3c None admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 10:41:57.772505 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c732b3d-9abc-4ec2-a1d7-2ba7619ab979 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 10:41:57.773105 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c732b3d-9abc-4ec2-a1d7-2ba7619ab979 None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:41:57.773388 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c732b3d-9abc-4ec2-a1d7-2ba7619ab979 None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:41:57.773664 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c732b3d-9abc-4ec2-a1d7-2ba7619ab979 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:41:57.814356 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7f6e7e0-3301-485d-ab4f-36bb5216e66e None admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 10:41:57.842079 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c732b3d-9abc-4ec2-a1d7-2ba7619ab979 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 10:41:57.843678 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c732b3d-9abc-4ec2-a1d7-2ba7619ab979 None admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 10:41:57.847017 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae63d952-7483-4dda-86fb-1e821447d21a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 10:41:57.847612 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae63d952-7483-4dda-86fb-1e821447d21a None admin] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:41:57.847887 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae63d952-7483-4dda-86fb-1e821447d21a None admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:41:57.848156 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae63d952-7483-4dda-86fb-1e821447d21a None admin] PATH_INFO: `/v3/domains` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:41:57.848787 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae63d952-7483-4dda-86fb-1e821447d21a None admin] RBAC: Adding query filter params (name=Default) {{(pid=80144) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 10:41:57.849089 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae63d952-7483-4dda-86fb-1e821447d21a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 10:41:57.850418 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae63d952-7483-4dda-86fb-1e821447d21a None admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 10:41:57.861691 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 265/516] 10.222.0.25 () {56 vars in 1109 bytes} [Sun Aug 7 10:41:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 10:41:57.862732 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae63d952-7483-4dda-86fb-1e821447d21a None admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 10:41:57.914432 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=100 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 10:41:57.915301 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=100 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 10:41:57.943912 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 252/517] 10.222.0.25 () {58 vars in 1282 bytes} [Sun Aug 7 10:41:57 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 316 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 07 10:41:57.944841 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c732b3d-9abc-4ec2-a1d7-2ba7619ab979 None admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 10:41:57.962435 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae63d952-7483-4dda-86fb-1e821447d21a None admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 10:41:57.995518 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a11e4600-6265-477e-820a-0f57863a1a4c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 10:41:57.996205 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a11e4600-6265-477e-820a-0f57863a1a4c None admin] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:41:57.996557 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a11e4600-6265-477e-820a-0f57863a1a4c None admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:41:57.996901 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a11e4600-6265-477e-820a-0f57863a1a4c None admin] PATH_INFO: `/v3/domains` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:41:57.997608 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a11e4600-6265-477e-820a-0f57863a1a4c None admin] RBAC: Adding query filter params (name=Default) {{(pid=80144) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 10:41:57.998001 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a11e4600-6265-477e-820a-0f57863a1a4c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 10:41:57.999451 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a11e4600-6265-477e-820a-0f57863a1a4c None admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 10:41:58.011353 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 266/518] 10.222.0.25 () {56 vars in 1109 bytes} [Sun Aug 7 10:41:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 10:41:58.012217 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a11e4600-6265-477e-820a-0f57863a1a4c None admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 10:41:58.044488 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c732b3d-9abc-4ec2-a1d7-2ba7619ab979 None admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 10:41:58.079179 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0132414d-4e6e-4a63-9703-b918cb2bdd75 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 10:41:58.079873 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0132414d-4e6e-4a63-9703-b918cb2bdd75 None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:41:58.080154 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0132414d-4e6e-4a63-9703-b918cb2bdd75 None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:41:58.080427 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0132414d-4e6e-4a63-9703-b918cb2bdd75 None admin] PATH_INFO: `/v3/projects` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:41:58.081117 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0132414d-4e6e-4a63-9703-b918cb2bdd75 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80143) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 10:41:58.081422 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0132414d-4e6e-4a63-9703-b918cb2bdd75 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 10:41:58.082863 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0132414d-4e6e-4a63-9703-b918cb2bdd75 None admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 10:41:58.100633 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 253/519] 10.222.0.25 () {56 vars in 1148 bytes} [Sun Aug 7 10:41:57 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 10:41:58.101438 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0132414d-4e6e-4a63-9703-b918cb2bdd75 None admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 10:41:58.110761 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a11e4600-6265-477e-820a-0f57863a1a4c None admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 10:41:58.124187 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=100 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 10:41:58.131995 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=100 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 10:41:58.146338 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb0c8e6d-efce-43ff-9be7-875bff3df939 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 10:41:58.146958 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb0c8e6d-efce-43ff-9be7-875bff3df939 None admin] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:41:58.147243 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb0c8e6d-efce-43ff-9be7-875bff3df939 None admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:41:58.147510 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb0c8e6d-efce-43ff-9be7-875bff3df939 None admin] PATH_INFO: `/v3/users` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:41:58.148229 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb0c8e6d-efce-43ff-9be7-875bff3df939 None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=80144) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 10:41:58.148529 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb0c8e6d-efce-43ff-9be7-875bff3df939 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 10:41:58.149932 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb0c8e6d-efce-43ff-9be7-875bff3df939 None admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 10:41:58.166999 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 267/520] 10.222.0.25 () {56 vars in 1137 bytes} [Sun Aug 7 10:41:58 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 10:41:58.167642 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb0c8e6d-efce-43ff-9be7-875bff3df939 None admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 10:41:58.204231 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0132414d-4e6e-4a63-9703-b918cb2bdd75 None admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 10:41:58.237721 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d5894be-b26b-44cf-a7c3-9989b0c20724 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 10:41:58.238353 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d5894be-b26b-44cf-a7c3-9989b0c20724 None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:41:58.238635 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d5894be-b26b-44cf-a7c3-9989b0c20724 None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:41:58.238911 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d5894be-b26b-44cf-a7c3-9989b0c20724 None admin] PATH_INFO: `/v3/users/neutron` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:41:58.245853 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d5894be-b26b-44cf-a7c3-9989b0c20724 None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 10:41:58.247335 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d5894be-b26b-44cf-a7c3-9989b0c20724 None admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 10:41:58.259854 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application [None req-6d5894be-b26b-44cf-a7c3-9989b0c20724 None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Aug 07 10:41:58.259854 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 07 10:41:58.259854 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 07 10:41:58.259854 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 07 10:41:58.259854 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 07 10:41:58.259854 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 07 10:41:58.259854 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 07 10:41:58.259854 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 07 10:41:58.259854 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 07 10:41:58.259854 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 07 10:41:58.259854 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 07 10:41:58.259854 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 07 10:41:58.259854 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 07 10:41:58.259854 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 07 10:41:58.259854 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 07 10:41:58.259854 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 07 10:41:58.261233 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 07 10:41:58.261233 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 07 10:41:58.261233 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 07 10:41:58.261233 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 07 10:41:58.261233 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 07 10:41:58.261233 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 07 10:41:58.261233 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 07 10:41:58.261233 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 07 10:41:58.261233 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 07 10:41:58.261233 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.get_or_create( Aug 07 10:41:58.261233 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 07 10:41:58.261233 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application with Lock( Aug 07 10:41:58.261233 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 07 10:41:58.261233 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self._enter() Aug 07 10:41:58.261233 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 07 10:41:58.261233 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 07 10:41:58.261233 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 07 10:41:58.261233 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.creator() Aug 07 10:41:58.262099 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 07 10:41:58.262099 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application created_value = creator( Aug 07 10:41:58.262099 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 07 10:41:58.262099 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 07 10:41:58.262099 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 07 10:41:58.262099 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 07 10:41:58.262099 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 07 10:41:58.262099 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 07 10:41:58.262099 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Aug 07 10:41:58.262099 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application Aug 07 10:41:58.262099 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 254/521] 10.222.0.25 () {56 vars in 1102 bytes} [Sun Aug 7 10:41:58 2022] GET /identity/v3/users/neutron => generated 85 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 07 10:41:58.262648 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d5894be-b26b-44cf-a7c3-9989b0c20724 None admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 10:41:58.269817 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb0c8e6d-efce-43ff-9be7-875bff3df939 None admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 10:41:58.302972 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc16fede-6868-430f-a251-a114ce34111f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 10:41:58.303529 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc16fede-6868-430f-a251-a114ce34111f None admin] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:41:58.303812 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc16fede-6868-430f-a251-a114ce34111f None admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:41:58.304094 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc16fede-6868-430f-a251-a114ce34111f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:41:58.315188 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc16fede-6868-430f-a251-a114ce34111f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 10:41:58.316643 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc16fede-6868-430f-a251-a114ce34111f None admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 10:41:58.330415 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application [None req-dc16fede-6868-430f-a251-a114ce34111f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 07 10:41:58.330415 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 07 10:41:58.330415 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 07 10:41:58.330415 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 07 10:41:58.330415 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 07 10:41:58.330415 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 07 10:41:58.330415 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 07 10:41:58.330415 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 07 10:41:58.330415 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 07 10:41:58.330415 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 07 10:41:58.330415 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 07 10:41:58.330415 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 07 10:41:58.330415 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 07 10:41:58.330415 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 07 10:41:58.330415 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 07 10:41:58.330415 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 07 10:41:58.331394 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 07 10:41:58.331394 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 07 10:41:58.331394 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 07 10:41:58.331394 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 07 10:41:58.331394 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 07 10:41:58.331394 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self.get_or_create( Aug 07 10:41:58.331394 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 07 10:41:58.331394 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application with Lock( Aug 07 10:41:58.331394 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 07 10:41:58.331394 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self._enter() Aug 07 10:41:58.331394 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 07 10:41:58.331394 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 07 10:41:58.331394 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 07 10:41:58.331394 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self.creator() Aug 07 10:41:58.331394 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 07 10:41:58.331394 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application created_value = creator( Aug 07 10:41:58.331394 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 07 10:41:58.332239 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 07 10:41:58.332239 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 07 10:41:58.332239 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application Aug 07 10:41:58.332239 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 268/522] 10.222.0.25 () {56 vars in 1108 bytes} [Sun Aug 7 10:41:58 2022] GET /identity/v3/domains/Default => generated 87 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 07 10:41:58.332463 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc16fede-6868-430f-a251-a114ce34111f None admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 10:41:58.354647 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=100 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 10:41:58.357024 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=100 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 10:41:58.364422 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d5894be-b26b-44cf-a7c3-9989b0c20724 None admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 10:41:58.397407 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6d4601b-58ef-4b74-8d94-3a8594015a99 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 10:41:58.398011 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6d4601b-58ef-4b74-8d94-3a8594015a99 None admin] REQUEST_METHOD: `POST` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:41:58.398290 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6d4601b-58ef-4b74-8d94-3a8594015a99 None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:41:58.398563 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6d4601b-58ef-4b74-8d94-3a8594015a99 None admin] PATH_INFO: `/v3/users` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:41:58.399235 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6d4601b-58ef-4b74-8d94-3a8594015a99 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 10:41:58.400675 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6d4601b-58ef-4b74-8d94-3a8594015a99 None admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 10:41:58.424433 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 255/523] 10.222.0.25 () {60 vars in 1132 bytes} [Sun Aug 7 10:41:58 2022] POST /identity/v3/users => generated 247 bytes in 162 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 07 10:41:58.425271 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6d4601b-58ef-4b74-8d94-3a8594015a99 None admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 10:41:58.446693 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc16fede-6868-430f-a251-a114ce34111f None admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 10:41:58.493593 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1309246e-5674-42e0-93b1-b19436e9870e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 10:41:58.494375 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1309246e-5674-42e0-93b1-b19436e9870e None admin] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:41:58.494775 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1309246e-5674-42e0-93b1-b19436e9870e None admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:41:58.495208 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1309246e-5674-42e0-93b1-b19436e9870e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:41:58.496109 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1309246e-5674-42e0-93b1-b19436e9870e None admin] RBAC: Adding query filter params (role.id=fbf375e9760340c9aa9529333eeb25f9, scope.project.id=6de1f3cac02c4bcdb05dd2337176033d, user.id=31d16d1b96104b4eb690398432b9829e) {{(pid=80144) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 10:41:58.496433 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1309246e-5674-42e0-93b1-b19436e9870e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 10:41:58.497912 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1309246e-5674-42e0-93b1-b19436e9870e None admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 10:41:58.504195 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 269/524] 10.222.0.25 () {56 vars in 1374 bytes} [Sun Aug 7 10:41:58 2022] GET /identity/v3/role_assignments?user.id=31d16d1b96104b4eb690398432b9829e&scope.project.id=6de1f3cac02c4bcdb05dd2337176033d&role.id=fbf375e9760340c9aa9529333eeb25f9 => generated 259 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 10:41:58.505184 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1309246e-5674-42e0-93b1-b19436e9870e None admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 10:41:58.563914 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6d4601b-58ef-4b74-8d94-3a8594015a99 None admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 10:41:58.613487 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50446e09-81d9-4e6f-8ebc-2a73944786c6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 10:41:58.614124 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50446e09-81d9-4e6f-8ebc-2a73944786c6 None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:41:58.614409 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50446e09-81d9-4e6f-8ebc-2a73944786c6 None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:41:58.614678 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50446e09-81d9-4e6f-8ebc-2a73944786c6 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:41:58.622815 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=100 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 10:41:58.638626 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 10:41:58.648925 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1309246e-5674-42e0-93b1-b19436e9870e None admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 10:41:58.691169 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b76e0f50-9d48-468d-9c4b-5708e7be9942 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 10:41:58.691169 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b76e0f50-9d48-468d-9c4b-5708e7be9942 None admin] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:41:58.691837 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b76e0f50-9d48-468d-9c4b-5708e7be9942 None admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:41:58.691837 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b76e0f50-9d48-468d-9c4b-5708e7be9942 None admin] PATH_INFO: `/v3/users` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:41:58.692955 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b76e0f50-9d48-468d-9c4b-5708e7be9942 None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=80144) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 10:41:58.693368 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b76e0f50-9d48-468d-9c4b-5708e7be9942 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 10:41:58.694679 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b76e0f50-9d48-468d-9c4b-5708e7be9942 None admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 10:41:58.709049 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50446e09-81d9-4e6f-8ebc-2a73944786c6 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 10:41:58.712853 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50446e09-81d9-4e6f-8ebc-2a73944786c6 None admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 10:41:58.720027 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 270/525] 10.222.0.25 () {56 vars in 1139 bytes} [Sun Aug 7 10:41:58 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 10:41:58.721280 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b76e0f50-9d48-468d-9c4b-5708e7be9942 None admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 10:41:58.829301 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 256/526] 10.222.0.25 () {58 vars in 1282 bytes} [Sun Aug 7 10:41:58 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 404 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 07 10:41:58.830217 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50446e09-81d9-4e6f-8ebc-2a73944786c6 None admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 10:41:58.831043 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b76e0f50-9d48-468d-9c4b-5708e7be9942 None admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 10:41:58.867585 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f31f5f02-b8f3-4519-8286-047a76c83294 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 10:41:58.868632 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f31f5f02-b8f3-4519-8286-047a76c83294 None admin] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:41:58.869176 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f31f5f02-b8f3-4519-8286-047a76c83294 None admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:41:58.869712 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f31f5f02-b8f3-4519-8286-047a76c83294 None admin] PATH_INFO: `/v3/domains` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:41:58.870635 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f31f5f02-b8f3-4519-8286-047a76c83294 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80144) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 10:41:58.871160 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f31f5f02-b8f3-4519-8286-047a76c83294 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 10:41:58.872873 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f31f5f02-b8f3-4519-8286-047a76c83294 None admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 10:41:58.885961 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 271/527] 10.222.0.25 () {56 vars in 1109 bytes} [Sun Aug 7 10:41:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 10:41:58.887138 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f31f5f02-b8f3-4519-8286-047a76c83294 None admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 10:41:58.889978 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=100 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 10:41:58.891864 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=100 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 10:41:58.956965 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50446e09-81d9-4e6f-8ebc-2a73944786c6 None admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 10:41:58.994083 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f31f5f02-b8f3-4519-8286-047a76c83294 None admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 10:41:59.005356 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d05a9da-ba4b-4502-b0dd-b2207098db57 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 10:41:59.006383 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d05a9da-ba4b-4502-b0dd-b2207098db57 None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:41:59.006673 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d05a9da-ba4b-4502-b0dd-b2207098db57 None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:41:59.006960 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d05a9da-ba4b-4502-b0dd-b2207098db57 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:41:59.035477 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a686239-5063-40d7-a5df-dfc34f5c1883 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 10:41:59.036577 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a686239-5063-40d7-a5df-dfc34f5c1883 None admin] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:41:59.037345 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a686239-5063-40d7-a5df-dfc34f5c1883 None admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:41:59.038058 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a686239-5063-40d7-a5df-dfc34f5c1883 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:41:59.052592 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a686239-5063-40d7-a5df-dfc34f5c1883 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 10:41:59.054777 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a686239-5063-40d7-a5df-dfc34f5c1883 None admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 10:41:59.070953 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application [None req-7a686239-5063-40d7-a5df-dfc34f5c1883 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 07 10:41:59.070953 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 07 10:41:59.070953 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 07 10:41:59.070953 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 07 10:41:59.070953 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 07 10:41:59.070953 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 07 10:41:59.070953 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 07 10:41:59.070953 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 07 10:41:59.070953 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 07 10:41:59.070953 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 07 10:41:59.070953 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 07 10:41:59.070953 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 07 10:41:59.070953 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 07 10:41:59.070953 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 07 10:41:59.070953 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 07 10:41:59.070953 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 07 10:41:59.071895 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 07 10:41:59.071895 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 07 10:41:59.071895 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 07 10:41:59.071895 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 07 10:41:59.071895 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 07 10:41:59.071895 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self.get_or_create( Aug 07 10:41:59.071895 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 07 10:41:59.071895 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application with Lock( Aug 07 10:41:59.071895 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 07 10:41:59.071895 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self._enter() Aug 07 10:41:59.071895 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 07 10:41:59.071895 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 07 10:41:59.071895 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 07 10:41:59.071895 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self.creator() Aug 07 10:41:59.071895 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 07 10:41:59.071895 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application created_value = creator( Aug 07 10:41:59.071895 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 07 10:41:59.073131 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 07 10:41:59.073131 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 07 10:41:59.073131 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application Aug 07 10:41:59.075426 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 272/528] 10.222.0.25 () {56 vars in 1108 bytes} [Sun Aug 7 10:41:58 2022] GET /identity/v3/domains/Default => generated 87 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 07 10:41:59.076453 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a686239-5063-40d7-a5df-dfc34f5c1883 None admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 10:41:59.092706 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d05a9da-ba4b-4502-b0dd-b2207098db57 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 10:41:59.095256 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d05a9da-ba4b-4502-b0dd-b2207098db57 None admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 10:41:59.147801 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=100 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 10:41:59.158779 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=100 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 10:41:59.194747 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a686239-5063-40d7-a5df-dfc34f5c1883 None admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 10:41:59.226487 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 257/529] 10.222.0.25 () {58 vars in 1282 bytes} [Sun Aug 7 10:41:58 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 396 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 07 10:41:59.227911 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d05a9da-ba4b-4502-b0dd-b2207098db57 None admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 10:41:59.230035 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb4517f7-510c-4d0f-93d1-b8eac28229e1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 10:41:59.230766 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb4517f7-510c-4d0f-93d1-b8eac28229e1 None admin] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:41:59.231175 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb4517f7-510c-4d0f-93d1-b8eac28229e1 None admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:41:59.231585 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb4517f7-510c-4d0f-93d1-b8eac28229e1 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:41:59.239022 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb4517f7-510c-4d0f-93d1-b8eac28229e1 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 10:41:59.241019 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb4517f7-510c-4d0f-93d1-b8eac28229e1 None admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 10:41:59.254410 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application [None req-eb4517f7-510c-4d0f-93d1-b8eac28229e1 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 07 10:41:59.254410 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 07 10:41:59.254410 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 07 10:41:59.254410 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 07 10:41:59.254410 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 07 10:41:59.254410 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 07 10:41:59.254410 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 07 10:41:59.254410 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 07 10:41:59.254410 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 07 10:41:59.254410 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 07 10:41:59.254410 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 07 10:41:59.254410 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 07 10:41:59.254410 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 07 10:41:59.254410 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 07 10:41:59.254410 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 07 10:41:59.255347 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 07 10:41:59.255347 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 07 10:41:59.255347 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 07 10:41:59.255347 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 07 10:41:59.255347 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 07 10:41:59.255347 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 07 10:41:59.255347 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self.get_or_create( Aug 07 10:41:59.255347 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 07 10:41:59.255347 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application with Lock( Aug 07 10:41:59.255347 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 07 10:41:59.255347 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self._enter() Aug 07 10:41:59.255347 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 07 10:41:59.255347 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 07 10:41:59.255347 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 07 10:41:59.255347 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self.creator() Aug 07 10:41:59.255347 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 07 10:41:59.255347 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application created_value = creator( Aug 07 10:41:59.256189 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 07 10:41:59.256189 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 07 10:41:59.256189 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 07 10:41:59.256189 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 07 10:41:59.256189 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 07 10:41:59.256189 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 07 10:41:59.256189 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 07 10:41:59.256189 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application Aug 07 10:41:59.256189 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 273/530] 10.222.0.25 () {56 vars in 1111 bytes} [Sun Aug 7 10:41:59 2022] GET /identity/v3/projects/service => generated 88 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 07 10:41:59.256657 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb4517f7-510c-4d0f-93d1-b8eac28229e1 None admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 10:41:59.348122 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d05a9da-ba4b-4502-b0dd-b2207098db57 None admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 10:41:59.359513 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb4517f7-510c-4d0f-93d1-b8eac28229e1 None admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 10:41:59.377043 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=100 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 10:41:59.393869 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a040e774-2df0-4619-b233-4c7a4a98011d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 10:41:59.394201 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54ff4989-aa13-4913-80b9-e337cd0ad5fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 10:41:59.394830 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54ff4989-aa13-4913-80b9-e337cd0ad5fb None admin] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:41:59.395135 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54ff4989-aa13-4913-80b9-e337cd0ad5fb None admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:41:59.395452 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54ff4989-aa13-4913-80b9-e337cd0ad5fb None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:41:59.395554 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a040e774-2df0-4619-b233-4c7a4a98011d None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:41:59.396174 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a040e774-2df0-4619-b233-4c7a4a98011d None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:41:59.396639 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a040e774-2df0-4619-b233-4c7a4a98011d None admin] PATH_INFO: `/v3/domains` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:41:59.397570 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a040e774-2df0-4619-b233-4c7a4a98011d None admin] RBAC: Adding query filter params (name=Default) {{(pid=80143) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 10:41:59.398170 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a040e774-2df0-4619-b233-4c7a4a98011d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 10:41:59.399765 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a040e774-2df0-4619-b233-4c7a4a98011d None admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 10:41:59.406541 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54ff4989-aa13-4913-80b9-e337cd0ad5fb None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 10:41:59.408009 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54ff4989-aa13-4913-80b9-e337cd0ad5fb None admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 10:41:59.417990 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 258/531] 10.222.0.25 () {56 vars in 1109 bytes} [Sun Aug 7 10:41:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 10:41:59.419212 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a040e774-2df0-4619-b233-4c7a4a98011d None admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 10:41:59.423712 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application [None req-54ff4989-aa13-4913-80b9-e337cd0ad5fb None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 07 10:41:59.423712 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 07 10:41:59.423712 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 07 10:41:59.423712 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 07 10:41:59.423712 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 07 10:41:59.423712 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 07 10:41:59.423712 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 07 10:41:59.423712 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 07 10:41:59.423712 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 07 10:41:59.423712 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 07 10:41:59.423712 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 07 10:41:59.423712 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 07 10:41:59.423712 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 07 10:41:59.423712 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 07 10:41:59.423712 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 07 10:41:59.423712 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 07 10:41:59.424570 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 07 10:41:59.424570 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 07 10:41:59.424570 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 07 10:41:59.424570 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 07 10:41:59.424570 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 07 10:41:59.424570 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self.get_or_create( Aug 07 10:41:59.424570 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 07 10:41:59.424570 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application with Lock( Aug 07 10:41:59.424570 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 07 10:41:59.424570 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self._enter() Aug 07 10:41:59.424570 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 07 10:41:59.424570 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 07 10:41:59.424570 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 07 10:41:59.424570 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self.creator() Aug 07 10:41:59.424570 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 07 10:41:59.424570 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application created_value = creator( Aug 07 10:41:59.424570 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 07 10:41:59.425397 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 07 10:41:59.425397 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 07 10:41:59.425397 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application Aug 07 10:41:59.425397 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 274/532] 10.222.0.25 () {56 vars in 1108 bytes} [Sun Aug 7 10:41:59 2022] GET /identity/v3/domains/Default => generated 87 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 07 10:41:59.425397 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54ff4989-aa13-4913-80b9-e337cd0ad5fb None admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 10:41:59.436199 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=100 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 10:41:59.537500 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54ff4989-aa13-4913-80b9-e337cd0ad5fb None admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 10:41:59.538327 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a040e774-2df0-4619-b233-4c7a4a98011d None admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 10:41:59.572191 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbdb7ff0-479d-4f16-a9c6-268a97c0716f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 10:41:59.573627 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbdb7ff0-479d-4f16-a9c6-268a97c0716f None admin] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:41:59.574003 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbdb7ff0-479d-4f16-a9c6-268a97c0716f None admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:41:59.574387 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbdb7ff0-479d-4f16-a9c6-268a97c0716f None admin] PATH_INFO: `/v3/projects/service` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:41:59.578131 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed702a2c-a680-4cab-a247-20ef9467d8e7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 10:41:59.578914 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed702a2c-a680-4cab-a247-20ef9467d8e7 None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:41:59.579374 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed702a2c-a680-4cab-a247-20ef9467d8e7 None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:41:59.579791 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed702a2c-a680-4cab-a247-20ef9467d8e7 None admin] PATH_INFO: `/v3/projects` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:41:59.580622 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed702a2c-a680-4cab-a247-20ef9467d8e7 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80143) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 10:41:59.581143 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed702a2c-a680-4cab-a247-20ef9467d8e7 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 10:41:59.581907 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbdb7ff0-479d-4f16-a9c6-268a97c0716f None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 10:41:59.582911 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed702a2c-a680-4cab-a247-20ef9467d8e7 None admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 10:41:59.583716 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbdb7ff0-479d-4f16-a9c6-268a97c0716f None admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 10:41:59.595445 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application [None req-bbdb7ff0-479d-4f16-a9c6-268a97c0716f None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 07 10:41:59.595445 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 07 10:41:59.595445 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 07 10:41:59.595445 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 07 10:41:59.595445 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 07 10:41:59.595445 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 07 10:41:59.595445 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 07 10:41:59.595445 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 07 10:41:59.595445 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 07 10:41:59.595445 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 07 10:41:59.595445 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 07 10:41:59.595445 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 07 10:41:59.595445 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 07 10:41:59.595445 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 07 10:41:59.595445 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 07 10:41:59.596502 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 07 10:41:59.596502 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 07 10:41:59.596502 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 07 10:41:59.596502 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 07 10:41:59.596502 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 07 10:41:59.596502 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 07 10:41:59.596502 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self.get_or_create( Aug 07 10:41:59.596502 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 07 10:41:59.596502 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application with Lock( Aug 07 10:41:59.596502 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 07 10:41:59.596502 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self._enter() Aug 07 10:41:59.596502 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 07 10:41:59.596502 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 07 10:41:59.596502 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 07 10:41:59.596502 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self.creator() Aug 07 10:41:59.596502 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 07 10:41:59.596502 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application created_value = creator( Aug 07 10:41:59.597364 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 07 10:41:59.597364 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 07 10:41:59.597364 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 07 10:41:59.597364 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 07 10:41:59.597364 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 07 10:41:59.597364 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 07 10:41:59.597364 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 07 10:41:59.597364 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application Aug 07 10:41:59.599332 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 259/533] 10.222.0.25 () {56 vars in 1148 bytes} [Sun Aug 7 10:41:59 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 10:41:59.600435 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed702a2c-a680-4cab-a247-20ef9467d8e7 None admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 10:41:59.602311 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 275/534] 10.222.0.25 () {56 vars in 1111 bytes} [Sun Aug 7 10:41:59 2022] GET /identity/v3/projects/service => generated 88 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 07 10:41:59.603301 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbdb7ff0-479d-4f16-a9c6-268a97c0716f None admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 10:41:59.632905 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=100 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 10:41:59.681125 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=100 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 10:41:59.712089 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbdb7ff0-479d-4f16-a9c6-268a97c0716f None admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 10:41:59.720855 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed702a2c-a680-4cab-a247-20ef9467d8e7 None admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 10:41:59.747305 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65fcfa8e-1f58-43f2-b885-d560c4b61428 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 10:41:59.748031 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65fcfa8e-1f58-43f2-b885-d560c4b61428 None admin] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:41:59.748378 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65fcfa8e-1f58-43f2-b885-d560c4b61428 None admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:41:59.748717 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65fcfa8e-1f58-43f2-b885-d560c4b61428 None admin] PATH_INFO: `/v3/projects` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:41:59.749553 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65fcfa8e-1f58-43f2-b885-d560c4b61428 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80144) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 10:41:59.749827 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65fcfa8e-1f58-43f2-b885-d560c4b61428 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 10:41:59.751348 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65fcfa8e-1f58-43f2-b885-d560c4b61428 None admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 10:41:59.761589 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d2c210f-bd07-4fd4-b5ba-095e3e684164 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 10:41:59.762516 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d2c210f-bd07-4fd4-b5ba-095e3e684164 None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:41:59.763034 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d2c210f-bd07-4fd4-b5ba-095e3e684164 None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:41:59.763525 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d2c210f-bd07-4fd4-b5ba-095e3e684164 None admin] PATH_INFO: `/v3/domains` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:41:59.764533 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d2c210f-bd07-4fd4-b5ba-095e3e684164 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80143) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 10:41:59.765035 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d2c210f-bd07-4fd4-b5ba-095e3e684164 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 10:41:59.766679 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d2c210f-bd07-4fd4-b5ba-095e3e684164 None admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 10:41:59.774620 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 276/535] 10.222.0.25 () {56 vars in 1148 bytes} [Sun Aug 7 10:41:59 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 10:41:59.775612 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65fcfa8e-1f58-43f2-b885-d560c4b61428 None admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 10:41:59.784375 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 260/536] 10.222.0.25 () {56 vars in 1109 bytes} [Sun Aug 7 10:41:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 10:41:59.785469 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d2c210f-bd07-4fd4-b5ba-095e3e684164 None admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 10:41:59.864025 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=100 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 10:41:59.877498 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65fcfa8e-1f58-43f2-b885-d560c4b61428 None admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 10:41:59.902094 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d2c210f-bd07-4fd4-b5ba-095e3e684164 None admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 10:41:59.914337 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40f04725-66bd-4af1-a53c-1b3e27b7da02 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 10:41:59.914958 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40f04725-66bd-4af1-a53c-1b3e27b7da02 None admin] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:41:59.915300 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40f04725-66bd-4af1-a53c-1b3e27b7da02 None admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:41:59.915572 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40f04725-66bd-4af1-a53c-1b3e27b7da02 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:41:59.916390 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40f04725-66bd-4af1-a53c-1b3e27b7da02 None admin] RBAC: Adding query filter params (role.id=fbf375e9760340c9aa9529333eeb25f9, scope.project.id=6de1f3cac02c4bcdb05dd2337176033d, user.id=88dbd6dfd1164994a3e16da553561910) {{(pid=80144) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 10:41:59.916698 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40f04725-66bd-4af1-a53c-1b3e27b7da02 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 10:41:59.918175 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40f04725-66bd-4af1-a53c-1b3e27b7da02 None admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 10:41:59.923641 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 277/537] 10.222.0.25 () {56 vars in 1374 bytes} [Sun Aug 7 10:41:59 2022] GET /identity/v3/role_assignments?user.id=88dbd6dfd1164994a3e16da553561910&scope.project.id=6de1f3cac02c4bcdb05dd2337176033d&role.id=fbf375e9760340c9aa9529333eeb25f9 => generated 259 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 10:41:59.924410 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40f04725-66bd-4af1-a53c-1b3e27b7da02 None admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 10:41:59.936011 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=100 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 10:41:59.945551 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fe1511f-c50a-455f-9829-d004e634685b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 10:41:59.947888 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fe1511f-c50a-455f-9829-d004e634685b None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:41:59.948395 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fe1511f-c50a-455f-9829-d004e634685b None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:41:59.948946 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fe1511f-c50a-455f-9829-d004e634685b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:41:59.949919 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fe1511f-c50a-455f-9829-d004e634685b None admin] RBAC: Adding query filter params (role.id=fbf375e9760340c9aa9529333eeb25f9, scope.project.id=6de1f3cac02c4bcdb05dd2337176033d, user.id=a11186d9d0d6488b953ede2f1628148d) {{(pid=80143) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 10:41:59.950444 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fe1511f-c50a-455f-9829-d004e634685b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 10:41:59.952259 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fe1511f-c50a-455f-9829-d004e634685b None admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 10:41:59.958938 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 261/538] 10.222.0.25 () {56 vars in 1374 bytes} [Sun Aug 7 10:41:59 2022] GET /identity/v3/role_assignments?user.id=a11186d9d0d6488b953ede2f1628148d&scope.project.id=6de1f3cac02c4bcdb05dd2337176033d&role.id=fbf375e9760340c9aa9529333eeb25f9 => generated 259 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 10:42:00.089720 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40f04725-66bd-4af1-a53c-1b3e27b7da02 None admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 10:42:00.127651 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc6ef8f8-40f5-4068-ac75-c525741cbc59 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 10:42:00.128209 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc6ef8f8-40f5-4068-ac75-c525741cbc59 None admin] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:42:00.128485 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc6ef8f8-40f5-4068-ac75-c525741cbc59 None admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:42:00.128756 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc6ef8f8-40f5-4068-ac75-c525741cbc59 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:42:00.136749 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc6ef8f8-40f5-4068-ac75-c525741cbc59 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 10:42:00.138212 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc6ef8f8-40f5-4068-ac75-c525741cbc59 None admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 10:42:00.150619 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application [None req-bc6ef8f8-40f5-4068-ac75-c525741cbc59 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 07 10:42:00.150619 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 07 10:42:00.150619 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 07 10:42:00.150619 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 07 10:42:00.150619 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 07 10:42:00.150619 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 07 10:42:00.150619 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 07 10:42:00.150619 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 07 10:42:00.150619 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 07 10:42:00.150619 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 07 10:42:00.150619 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 07 10:42:00.150619 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 07 10:42:00.150619 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 07 10:42:00.150619 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 07 10:42:00.150619 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 07 10:42:00.151768 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 07 10:42:00.151768 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 07 10:42:00.151768 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 07 10:42:00.151768 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 07 10:42:00.151768 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 07 10:42:00.151768 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 07 10:42:00.151768 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self.get_or_create( Aug 07 10:42:00.151768 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 07 10:42:00.151768 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application with Lock( Aug 07 10:42:00.151768 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 07 10:42:00.151768 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self._enter() Aug 07 10:42:00.151768 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 07 10:42:00.151768 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 07 10:42:00.151768 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 07 10:42:00.151768 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self.creator() Aug 07 10:42:00.151768 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 07 10:42:00.151768 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application created_value = creator( Aug 07 10:42:00.152611 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 07 10:42:00.152611 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 07 10:42:00.152611 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 07 10:42:00.152611 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 07 10:42:00.152611 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 07 10:42:00.152611 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 07 10:42:00.152611 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 07 10:42:00.152611 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application Aug 07 10:42:00.152611 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 278/539] 10.222.0.25 () {56 vars in 1111 bytes} [Sun Aug 7 10:41:59 2022] GET /identity/v3/projects/service => generated 88 bytes in 228 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 07 10:42:00.158967 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fe1511f-c50a-455f-9829-d004e634685b None admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 10:42:00.283214 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fe1511f-c50a-455f-9829-d004e634685b None admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 10:42:00.319575 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-254d1224-09d2-4fa9-8c40-722bbb7b19ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 10:42:00.320525 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-254d1224-09d2-4fa9-8c40-722bbb7b19ac None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:42:00.320974 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-254d1224-09d2-4fa9-8c40-722bbb7b19ac None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:42:00.321390 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-254d1224-09d2-4fa9-8c40-722bbb7b19ac None admin] PATH_INFO: `/v3/projects` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:42:00.322234 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-254d1224-09d2-4fa9-8c40-722bbb7b19ac None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80143) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 10:42:00.323234 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-254d1224-09d2-4fa9-8c40-722bbb7b19ac None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 10:42:00.324819 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-254d1224-09d2-4fa9-8c40-722bbb7b19ac None admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 10:42:00.337591 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 262/540] 10.222.0.25 () {56 vars in 1148 bytes} [Sun Aug 7 10:42:00 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 10:42:00.343797 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc6ef8f8-40f5-4068-ac75-c525741cbc59 None admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 10:42:00.368793 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=100 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 10:42:00.502835 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc6ef8f8-40f5-4068-ac75-c525741cbc59 None admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 10:42:00.561580 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da324991-5ea7-4983-bf6b-7b8e82964d50 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 10:42:00.562197 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da324991-5ea7-4983-bf6b-7b8e82964d50 None admin] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:42:00.562480 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da324991-5ea7-4983-bf6b-7b8e82964d50 None admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:42:00.562756 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da324991-5ea7-4983-bf6b-7b8e82964d50 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:42:00.563517 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da324991-5ea7-4983-bf6b-7b8e82964d50 None admin] RBAC: Adding query filter params (role.id=fbf375e9760340c9aa9529333eeb25f9, scope.project.id=6de1f3cac02c4bcdb05dd2337176033d, user.id=be1283c9bf47424c9aa3ad27cab03027) {{(pid=80144) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 10:42:00.563830 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da324991-5ea7-4983-bf6b-7b8e82964d50 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 10:42:00.565249 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da324991-5ea7-4983-bf6b-7b8e82964d50 None admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 10:42:00.575525 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 279/541] 10.222.0.25 () {56 vars in 1374 bytes} [Sun Aug 7 10:42:00 2022] GET /identity/v3/role_assignments?user.id=be1283c9bf47424c9aa3ad27cab03027&scope.project.id=6de1f3cac02c4bcdb05dd2337176033d&role.id=fbf375e9760340c9aa9529333eeb25f9 => generated 259 bytes in 232 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 10:42:00.652934 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc7e75b3-e603-43c2-b221-518aa296dad3 None None] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:42:00.653269 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc7e75b3-e603-43c2-b221-518aa296dad3 None None] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:42:00.653480 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc7e75b3-e603-43c2-b221-518aa296dad3 None None] PATH_INFO: `/` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:42:00.655228 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 263/542] 10.222.0.25 () {54 vars in 927 bytes} [Sun Aug 7 10:42:00 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 07 10:42:00.661184 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98895846-2828-485b-b156-b8f12980335c None None] REQUEST_METHOD: `POST` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:42:00.661478 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98895846-2828-485b-b156-b8f12980335c None None] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:42:00.661763 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98895846-2828-485b-b156-b8f12980335c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:42:00.688747 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.auth.core [None req-98895846-2828-485b-b156-b8f12980335c None None] MFA Rules not processed for user `bac4e6b76fe64217ac8fa44e71cffe6a`. Rule list: `[]` (Enabled: `True`). {{(pid=80144) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 10:42:00.689898 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.fernet_utils [None req-98895846-2828-485b-b156-b8f12980335c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80144) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 10:42:00.740534 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45e568c0-8d74-48a0-bead-a541bb431beb None None] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:42:00.741055 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45e568c0-8d74-48a0-bead-a541bb431beb None None] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:42:00.741055 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45e568c0-8d74-48a0-bead-a541bb431beb None None] PATH_INFO: `/` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:42:00.742675 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 264/543] 10.222.0.25 () {54 vars in 927 bytes} [Sun Aug 7 10:42:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 07 10:42:00.748785 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd178b23-8c11-4ef2-aa36-fc246a9b4944 None None] REQUEST_METHOD: `POST` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:42:00.749086 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd178b23-8c11-4ef2-aa36-fc246a9b4944 None None] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:42:00.749367 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd178b23-8c11-4ef2-aa36-fc246a9b4944 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:42:00.780271 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.auth.core [None req-bd178b23-8c11-4ef2-aa36-fc246a9b4944 None None] MFA Rules not processed for user `bac4e6b76fe64217ac8fa44e71cffe6a`. Rule list: `[]` (Enabled: `True`). {{(pid=80143) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 10:42:00.784309 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-bd178b23-8c11-4ef2-aa36-fc246a9b4944 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 10:42:00.805027 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 280/544] 10.222.0.25 () {58 vars in 1024 bytes} [Sun Aug 7 10:42:00 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 145 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 07 10:42:00.856948 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98895846-2828-485b-b156-b8f12980335c None None] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 10:42:00.865825 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=100 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 10:42:00.878978 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=100 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 10:42:00.943681 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 265/545] 10.222.0.25 () {58 vars in 1024 bytes} [Sun Aug 7 10:42:00 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 196 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 07 10:42:00.967549 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98895846-2828-485b-b156-b8f12980335c None None] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 10:42:00.984032 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd178b23-8c11-4ef2-aa36-fc246a9b4944 None None] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 10:42:01.007824 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca276b8a-3598-4d7b-a9e5-a09839b0f4ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 10:42:01.009813 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca276b8a-3598-4d7b-a9e5-a09839b0f4ec None admin] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:42:01.011000 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca276b8a-3598-4d7b-a9e5-a09839b0f4ec None admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:42:01.011296 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca276b8a-3598-4d7b-a9e5-a09839b0f4ec None admin] PATH_INFO: `/v3/roles/service` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:42:01.033986 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca276b8a-3598-4d7b-a9e5-a09839b0f4ec None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 07 10:42:01.038838 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application [None req-ca276b8a-3598-4d7b-a9e5-a09839b0f4ec None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 07 10:42:01.038838 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 07 10:42:01.038838 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 07 10:42:01.038838 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 07 10:42:01.038838 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 07 10:42:01.038838 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 07 10:42:01.038838 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 07 10:42:01.038838 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 07 10:42:01.038838 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 07 10:42:01.038838 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 07 10:42:01.038838 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 07 10:42:01.038838 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 07 10:42:01.038838 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 07 10:42:01.038838 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 07 10:42:01.038838 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 07 10:42:01.038838 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 07 10:42:01.039709 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 07 10:42:01.039709 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 07 10:42:01.039709 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 07 10:42:01.039709 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 07 10:42:01.039709 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 07 10:42:01.039709 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 07 10:42:01.039709 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 07 10:42:01.039709 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 07 10:42:01.039709 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 07 10:42:01.039709 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self.get_or_create( Aug 07 10:42:01.039709 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 07 10:42:01.039709 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application with Lock( Aug 07 10:42:01.039709 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 07 10:42:01.039709 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self._enter() Aug 07 10:42:01.039709 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 07 10:42:01.039709 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 07 10:42:01.041190 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 07 10:42:01.041190 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self.creator() Aug 07 10:42:01.041190 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 07 10:42:01.041190 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application created_value = creator( Aug 07 10:42:01.041190 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 07 10:42:01.041190 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 07 10:42:01.041190 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 07 10:42:01.041190 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 07 10:42:01.041190 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 07 10:42:01.041190 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 07 10:42:01.041190 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 07 10:42:01.041190 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application Aug 07 10:42:01.041190 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 281/546] 10.222.0.25 () {56 vars in 1102 bytes} [Sun Aug 7 10:42:00 2022] GET /identity/v3/roles/service => generated 85 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 07 10:42:01.045745 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca276b8a-3598-4d7b-a9e5-a09839b0f4ec None admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 10:42:01.117629 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=100 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 10:42:01.172314 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca276b8a-3598-4d7b-a9e5-a09839b0f4ec None admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 10:42:01.217402 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd178b23-8c11-4ef2-aa36-fc246a9b4944 None None] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 10:42:01.236068 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d0de83d-bed4-40ba-a1fe-e0d610bd94c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 10:42:01.236762 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d0de83d-bed4-40ba-a1fe-e0d610bd94c3 None admin] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:42:01.237044 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d0de83d-bed4-40ba-a1fe-e0d610bd94c3 None admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:42:01.237572 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d0de83d-bed4-40ba-a1fe-e0d610bd94c3 None admin] PATH_INFO: `/v3/roles` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:42:01.238241 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d0de83d-bed4-40ba-a1fe-e0d610bd94c3 None admin] RBAC: Adding query filter params (name=service) {{(pid=80144) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 10:42:01.238545 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d0de83d-bed4-40ba-a1fe-e0d610bd94c3 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 10:42:01.239939 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d0de83d-bed4-40ba-a1fe-e0d610bd94c3 None admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 10:42:01.258094 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 282/547] 10.222.0.25 () {56 vars in 1103 bytes} [Sun Aug 7 10:42:01 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 07 10:42:01.264903 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d0de83d-bed4-40ba-a1fe-e0d610bd94c3 None admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 10:42:01.287482 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ac3fa66-3687-4b8c-bdb6-e2a547347198 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 10:42:01.288258 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ac3fa66-3687-4b8c-bdb6-e2a547347198 None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:42:01.288547 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ac3fa66-3687-4b8c-bdb6-e2a547347198 None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:42:01.288865 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ac3fa66-3687-4b8c-bdb6-e2a547347198 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:42:01.299982 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ac3fa66-3687-4b8c-bdb6-e2a547347198 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 07 10:42:01.305206 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application [None req-7ac3fa66-3687-4b8c-bdb6-e2a547347198 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 07 10:42:01.305206 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 07 10:42:01.305206 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 07 10:42:01.305206 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 07 10:42:01.305206 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 07 10:42:01.305206 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 07 10:42:01.305206 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 07 10:42:01.305206 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 07 10:42:01.305206 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 07 10:42:01.305206 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 07 10:42:01.305206 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 07 10:42:01.305206 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 07 10:42:01.305206 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 07 10:42:01.305206 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 07 10:42:01.305206 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 07 10:42:01.305206 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 07 10:42:01.306074 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 07 10:42:01.306074 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 07 10:42:01.306074 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 07 10:42:01.306074 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 07 10:42:01.306074 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 07 10:42:01.306074 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 07 10:42:01.306074 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 07 10:42:01.306074 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 07 10:42:01.306074 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 07 10:42:01.306074 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.get_or_create( Aug 07 10:42:01.306074 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 07 10:42:01.306074 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application with Lock( Aug 07 10:42:01.306074 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 07 10:42:01.306074 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self._enter() Aug 07 10:42:01.306074 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 07 10:42:01.306074 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 07 10:42:01.307539 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 07 10:42:01.307539 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.creator() Aug 07 10:42:01.307539 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 07 10:42:01.307539 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application created_value = creator( Aug 07 10:42:01.307539 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 07 10:42:01.307539 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 07 10:42:01.307539 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 07 10:42:01.307539 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 07 10:42:01.307539 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 07 10:42:01.307539 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 07 10:42:01.307539 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 07 10:42:01.307539 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application Aug 07 10:42:01.307539 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 266/548] 10.222.0.25 () {56 vars in 1102 bytes} [Sun Aug 7 10:42:00 2022] GET /identity/v3/roles/service => generated 85 bytes in 323 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 07 10:42:01.314258 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ac3fa66-3687-4b8c-bdb6-e2a547347198 None admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 10:42:01.397334 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d0de83d-bed4-40ba-a1fe-e0d610bd94c3 None admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 10:42:01.423420 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=100 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 10:42:01.426279 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=100 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 10:42:01.443417 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e658cfcc-5e57-440e-b92f-ac1d9fc085fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 10:42:01.444071 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e658cfcc-5e57-440e-b92f-ac1d9fc085fd None admin] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:42:01.444354 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e658cfcc-5e57-440e-b92f-ac1d9fc085fd None admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:42:01.444629 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e658cfcc-5e57-440e-b92f-ac1d9fc085fd None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:42:01.460942 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e658cfcc-5e57-440e-b92f-ac1d9fc085fd None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 10:42:01.462868 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e658cfcc-5e57-440e-b92f-ac1d9fc085fd None admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 10:42:01.481839 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application [None req-e658cfcc-5e57-440e-b92f-ac1d9fc085fd None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 07 10:42:01.481839 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 07 10:42:01.481839 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 07 10:42:01.481839 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 07 10:42:01.481839 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 07 10:42:01.481839 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 07 10:42:01.481839 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 07 10:42:01.481839 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 07 10:42:01.481839 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 07 10:42:01.481839 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 07 10:42:01.481839 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 07 10:42:01.481839 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 07 10:42:01.481839 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 07 10:42:01.481839 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 07 10:42:01.481839 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 07 10:42:01.481839 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 07 10:42:01.482750 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 07 10:42:01.482750 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 07 10:42:01.482750 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 07 10:42:01.482750 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 07 10:42:01.482750 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 07 10:42:01.482750 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self.get_or_create( Aug 07 10:42:01.482750 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 07 10:42:01.482750 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application with Lock( Aug 07 10:42:01.482750 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 07 10:42:01.482750 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self._enter() Aug 07 10:42:01.482750 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 07 10:42:01.482750 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 07 10:42:01.482750 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 07 10:42:01.482750 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self.creator() Aug 07 10:42:01.482750 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 07 10:42:01.482750 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application created_value = creator( Aug 07 10:42:01.482750 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 07 10:42:01.484304 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 07 10:42:01.484304 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 07 10:42:01.484304 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application Aug 07 10:42:01.486584 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 283/549] 10.222.0.25 () {56 vars in 1108 bytes} [Sun Aug 7 10:42:01 2022] GET /identity/v3/domains/Default => generated 87 bytes in 222 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 07 10:42:01.496095 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e658cfcc-5e57-440e-b92f-ac1d9fc085fd None admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 10:42:01.538957 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ac3fa66-3687-4b8c-bdb6-e2a547347198 None admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 10:42:01.602067 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e658cfcc-5e57-440e-b92f-ac1d9fc085fd None admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 10:42:01.620424 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ebca08c-af82-4ebb-8b86-1792c7f33c13 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 10:42:01.620846 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ebca08c-af82-4ebb-8b86-1792c7f33c13 None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:42:01.621130 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ebca08c-af82-4ebb-8b86-1792c7f33c13 None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:42:01.621407 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ebca08c-af82-4ebb-8b86-1792c7f33c13 None admin] PATH_INFO: `/v3/roles` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:42:01.622089 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ebca08c-af82-4ebb-8b86-1792c7f33c13 None admin] RBAC: Adding query filter params (name=service) {{(pid=80143) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 10:42:01.622394 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ebca08c-af82-4ebb-8b86-1792c7f33c13 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 10:42:01.623798 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ebca08c-af82-4ebb-8b86-1792c7f33c13 None admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 10:42:01.643019 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 267/550] 10.222.0.25 () {56 vars in 1103 bytes} [Sun Aug 7 10:42:01 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 325 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 10:42:01.646834 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ebca08c-af82-4ebb-8b86-1792c7f33c13 None admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 10:42:01.647960 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e10967c9-df9a-4406-9889-f211154f0b08 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 10:42:01.649068 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e10967c9-df9a-4406-9889-f211154f0b08 None admin] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:42:01.651846 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e10967c9-df9a-4406-9889-f211154f0b08 None admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:42:01.653110 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e10967c9-df9a-4406-9889-f211154f0b08 None admin] PATH_INFO: `/v3/domains` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:42:01.653909 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e10967c9-df9a-4406-9889-f211154f0b08 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80144) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 10:42:01.655410 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e10967c9-df9a-4406-9889-f211154f0b08 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 10:42:01.656854 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e10967c9-df9a-4406-9889-f211154f0b08 None admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 10:42:01.677302 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 284/551] 10.222.0.25 () {56 vars in 1109 bytes} [Sun Aug 7 10:42:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 10:42:01.683865 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e10967c9-df9a-4406-9889-f211154f0b08 None admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 10:42:01.759265 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=100 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 10:42:01.834776 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ebca08c-af82-4ebb-8b86-1792c7f33c13 None admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 10:42:01.841235 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=100 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 10:42:01.865695 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e10967c9-df9a-4406-9889-f211154f0b08 None admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 10:42:01.905653 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-feb0de89-486f-4214-9e30-776b4e1d481b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 10:42:01.906585 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feb0de89-486f-4214-9e30-776b4e1d481b None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:42:01.907035 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feb0de89-486f-4214-9e30-776b4e1d481b None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:42:01.907451 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feb0de89-486f-4214-9e30-776b4e1d481b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:42:01.927649 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-feb0de89-486f-4214-9e30-776b4e1d481b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 10:42:01.929465 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-feb0de89-486f-4214-9e30-776b4e1d481b None admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 10:42:01.936321 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ace261b3-6b77-4091-a234-3c8f3ca11d5d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 10:42:01.937156 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ace261b3-6b77-4091-a234-3c8f3ca11d5d None admin] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:42:01.937580 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ace261b3-6b77-4091-a234-3c8f3ca11d5d None admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:42:01.938012 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ace261b3-6b77-4091-a234-3c8f3ca11d5d None admin] PATH_INFO: `/v3/users/swift` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:42:01.951411 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ace261b3-6b77-4091-a234-3c8f3ca11d5d None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 10:42:01.954010 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ace261b3-6b77-4091-a234-3c8f3ca11d5d None admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 10:42:01.959624 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application [None req-feb0de89-486f-4214-9e30-776b4e1d481b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 07 10:42:01.959624 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 07 10:42:01.959624 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 07 10:42:01.959624 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 07 10:42:01.959624 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 07 10:42:01.959624 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 07 10:42:01.959624 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 07 10:42:01.959624 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 07 10:42:01.959624 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 07 10:42:01.959624 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 07 10:42:01.959624 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 07 10:42:01.959624 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 07 10:42:01.959624 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 07 10:42:01.959624 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 07 10:42:01.959624 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 07 10:42:01.959624 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 07 10:42:01.960743 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 07 10:42:01.960743 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 07 10:42:01.960743 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 07 10:42:01.960743 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 07 10:42:01.960743 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 07 10:42:01.960743 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.get_or_create( Aug 07 10:42:01.960743 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 07 10:42:01.960743 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application with Lock( Aug 07 10:42:01.960743 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 07 10:42:01.960743 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self._enter() Aug 07 10:42:01.960743 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 07 10:42:01.960743 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 07 10:42:01.960743 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 07 10:42:01.960743 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.creator() Aug 07 10:42:01.960743 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 07 10:42:01.960743 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application created_value = creator( Aug 07 10:42:01.960743 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 07 10:42:01.961926 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 07 10:42:01.961926 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 07 10:42:01.961926 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application Aug 07 10:42:01.965754 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 268/552] 10.222.0.25 () {56 vars in 1108 bytes} [Sun Aug 7 10:42:01 2022] GET /identity/v3/domains/Default => generated 87 bytes in 320 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 07 10:42:01.974909 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-feb0de89-486f-4214-9e30-776b4e1d481b None admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 10:42:01.982242 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application [None req-ace261b3-6b77-4091-a234-3c8f3ca11d5d None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Aug 07 10:42:01.982242 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 07 10:42:01.982242 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 07 10:42:01.982242 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 07 10:42:01.982242 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 07 10:42:01.982242 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 07 10:42:01.982242 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 07 10:42:01.982242 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 07 10:42:01.982242 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 07 10:42:01.982242 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 07 10:42:01.982242 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 07 10:42:01.982242 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 07 10:42:01.982242 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 07 10:42:01.982242 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 07 10:42:01.982242 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 07 10:42:01.982242 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 07 10:42:01.983368 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 07 10:42:01.983368 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 07 10:42:01.983368 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 07 10:42:01.983368 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 07 10:42:01.983368 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 07 10:42:01.983368 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 07 10:42:01.983368 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 07 10:42:01.983368 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 07 10:42:01.983368 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 07 10:42:01.983368 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self.get_or_create( Aug 07 10:42:01.983368 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 07 10:42:01.983368 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application with Lock( Aug 07 10:42:01.983368 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 07 10:42:01.983368 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self._enter() Aug 07 10:42:01.983368 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 07 10:42:01.983368 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 07 10:42:01.983368 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 07 10:42:01.983368 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self.creator() Aug 07 10:42:01.984538 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 07 10:42:01.984538 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application created_value = creator( Aug 07 10:42:01.984538 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 07 10:42:01.984538 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 07 10:42:01.984538 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 07 10:42:01.984538 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 07 10:42:01.984538 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 07 10:42:01.984538 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 07 10:42:01.984538 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Aug 07 10:42:01.984538 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application Aug 07 10:42:01.996421 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 285/553] 10.222.0.25 () {56 vars in 1096 bytes} [Sun Aug 7 10:42:01 2022] GET /identity/v3/users/swift => generated 83 bytes in 310 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 07 10:42:01.999740 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ace261b3-6b77-4091-a234-3c8f3ca11d5d None admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 10:42:02.136361 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=100 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 10:42:02.139123 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ace261b3-6b77-4091-a234-3c8f3ca11d5d None admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 10:42:02.167647 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-feb0de89-486f-4214-9e30-776b4e1d481b None admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 10:42:02.214754 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96fbb511-94de-4a16-bff9-96790b804121 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 10:42:02.215213 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-705051e5-ccf7-45e7-aebf-192a8b5bd8e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 10:42:02.215529 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96fbb511-94de-4a16-bff9-96790b804121 None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:42:02.215820 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96fbb511-94de-4a16-bff9-96790b804121 None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:42:02.216093 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96fbb511-94de-4a16-bff9-96790b804121 None admin] PATH_INFO: `/v3/domains` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:42:02.216759 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-705051e5-ccf7-45e7-aebf-192a8b5bd8e5 None admin] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:42:02.216862 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96fbb511-94de-4a16-bff9-96790b804121 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80143) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 10:42:02.217080 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96fbb511-94de-4a16-bff9-96790b804121 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 10:42:02.217524 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-705051e5-ccf7-45e7-aebf-192a8b5bd8e5 None admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:42:02.217988 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-705051e5-ccf7-45e7-aebf-192a8b5bd8e5 None admin] PATH_INFO: `/v3/users` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:42:02.218443 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96fbb511-94de-4a16-bff9-96790b804121 None admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 10:42:02.219156 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-705051e5-ccf7-45e7-aebf-192a8b5bd8e5 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80144) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 10:42:02.219673 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-705051e5-ccf7-45e7-aebf-192a8b5bd8e5 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 10:42:02.221303 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-705051e5-ccf7-45e7-aebf-192a8b5bd8e5 None admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 10:42:02.233441 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 269/554] 10.222.0.25 () {56 vars in 1109 bytes} [Sun Aug 7 10:42:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 259 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 10:42:02.240070 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96fbb511-94de-4a16-bff9-96790b804121 None admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 10:42:02.253900 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 286/555] 10.222.0.25 () {56 vars in 1135 bytes} [Sun Aug 7 10:42:01 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 250 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 10:42:02.255459 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-705051e5-ccf7-45e7-aebf-192a8b5bd8e5 None admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 10:42:02.270612 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=100 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 10:42:02.387501 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-705051e5-ccf7-45e7-aebf-192a8b5bd8e5 None admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 10:42:02.425020 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ed2e9fb-c0c3-441e-9de5-ceab332501d3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 10:42:02.425632 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ed2e9fb-c0c3-441e-9de5-ceab332501d3 None admin] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:42:02.425911 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ed2e9fb-c0c3-441e-9de5-ceab332501d3 None admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:42:02.426177 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ed2e9fb-c0c3-441e-9de5-ceab332501d3 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:42:02.441976 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96fbb511-94de-4a16-bff9-96790b804121 None admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 10:42:02.495879 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=100 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 10:42:02.500749 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1742cb09-0782-4d80-9741-8d821a378510 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 10:42:02.501487 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1742cb09-0782-4d80-9741-8d821a378510 None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:42:02.501768 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1742cb09-0782-4d80-9741-8d821a378510 None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:42:02.502035 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1742cb09-0782-4d80-9741-8d821a378510 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:42:02.509390 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1742cb09-0782-4d80-9741-8d821a378510 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 10:42:02.510893 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1742cb09-0782-4d80-9741-8d821a378510 None admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 10:42:02.522665 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application [None req-1742cb09-0782-4d80-9741-8d821a378510 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Aug 07 10:42:02.522665 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 07 10:42:02.522665 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 07 10:42:02.522665 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 07 10:42:02.522665 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 07 10:42:02.522665 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 07 10:42:02.522665 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 07 10:42:02.522665 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 07 10:42:02.522665 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 07 10:42:02.522665 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 07 10:42:02.522665 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 07 10:42:02.522665 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 07 10:42:02.522665 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 07 10:42:02.522665 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 07 10:42:02.522665 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 07 10:42:02.522665 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 07 10:42:02.524212 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 07 10:42:02.524212 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 07 10:42:02.524212 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 07 10:42:02.524212 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 07 10:42:02.524212 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 07 10:42:02.524212 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 07 10:42:02.524212 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 07 10:42:02.524212 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 07 10:42:02.524212 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 07 10:42:02.524212 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.get_or_create( Aug 07 10:42:02.524212 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 07 10:42:02.524212 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application with Lock( Aug 07 10:42:02.524212 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 07 10:42:02.524212 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self._enter() Aug 07 10:42:02.524212 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 07 10:42:02.524212 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 07 10:42:02.524212 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 07 10:42:02.524212 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.creator() Aug 07 10:42:02.525128 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 07 10:42:02.525128 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application created_value = creator( Aug 07 10:42:02.525128 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 07 10:42:02.525128 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 07 10:42:02.525128 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 07 10:42:02.525128 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 07 10:42:02.525128 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 07 10:42:02.525128 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 07 10:42:02.525128 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Aug 07 10:42:02.525128 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application Aug 07 10:42:02.525128 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 270/556] 10.222.0.25 () {56 vars in 1093 bytes} [Sun Aug 7 10:42:02 2022] GET /identity/v3/users/nova => generated 82 bytes in 284 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 07 10:42:02.525936 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7486f5f5-51f6-4301-b83a-7225cbdf339a None None] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:42:02.525936 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7486f5f5-51f6-4301-b83a-7225cbdf339a None None] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:42:02.526056 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7486f5f5-51f6-4301-b83a-7225cbdf339a None None] PATH_INFO: `/` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:42:02.527578 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 271/557] 10.222.0.25 () {54 vars in 927 bytes} [Sun Aug 7 10:42:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 07 10:42:02.528760 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54e8935f-2824-40d6-bc6c-9b87fd4a2e1a None None] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:42:02.528957 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54e8935f-2824-40d6-bc6c-9b87fd4a2e1a None None] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:42:02.529231 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54e8935f-2824-40d6-bc6c-9b87fd4a2e1a None None] PATH_INFO: `/` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:42:02.530967 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 272/558] 10.222.0.25 () {54 vars in 927 bytes} [Sun Aug 7 10:42:02 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 07 10:42:02.533825 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f0fc5a4-052d-47a9-bf9d-2a839d8d7270 None None] REQUEST_METHOD: `POST` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:42:02.534328 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f0fc5a4-052d-47a9-bf9d-2a839d8d7270 None None] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:42:02.534732 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f0fc5a4-052d-47a9-bf9d-2a839d8d7270 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:42:02.535426 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ed2e9fb-c0c3-441e-9de5-ceab332501d3 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 10:42:02.536884 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ed2e9fb-c0c3-441e-9de5-ceab332501d3 None admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 10:42:02.552741 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.auth.core [None req-2f0fc5a4-052d-47a9-bf9d-2a839d8d7270 None None] MFA Rules not processed for user `bac4e6b76fe64217ac8fa44e71cffe6a`. Rule list: `[]` (Enabled: `True`). {{(pid=80143) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 10:42:02.553929 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-2f0fc5a4-052d-47a9-bf9d-2a839d8d7270 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 10:42:02.624549 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 273/559] 10.222.0.25 () {58 vars in 1024 bytes} [Sun Aug 7 10:42:02 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 07 10:42:02.635574 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f0fc5a4-052d-47a9-bf9d-2a839d8d7270 None None] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 10:42:02.635574 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=100 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 10:42:02.654754 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 287/560] 10.222.0.25 () {58 vars in 1282 bytes} [Sun Aug 7 10:42:02 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 399 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Aug 07 10:42:02.656475 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f133712-29b2-42fb-8af0-b40c1d5a794d None None] REQUEST_METHOD: `POST` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:42:02.656816 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f133712-29b2-42fb-8af0-b40c1d5a794d None None] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:42:02.657096 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f133712-29b2-42fb-8af0-b40c1d5a794d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:42:02.678935 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.auth.core [None req-5f133712-29b2-42fb-8af0-b40c1d5a794d None None] MFA Rules not processed for user `bac4e6b76fe64217ac8fa44e71cffe6a`. Rule list: `[]` (Enabled: `True`). {{(pid=80144) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 10:42:02.680146 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.fernet_utils [None req-5f133712-29b2-42fb-8af0-b40c1d5a794d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80144) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 10:42:02.729519 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f0fc5a4-052d-47a9-bf9d-2a839d8d7270 None None] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 10:42:02.737530 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=100 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 10:42:02.753104 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 288/561] 10.222.0.25 () {58 vars in 1024 bytes} [Sun Aug 7 10:42:02 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 07 10:42:02.759112 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f133712-29b2-42fb-8af0-b40c1d5a794d None None] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 10:42:02.772171 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c8cac8a-49ac-44ad-be22-59183311d864 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 10:42:02.772974 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c8cac8a-49ac-44ad-be22-59183311d864 None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:42:02.773322 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c8cac8a-49ac-44ad-be22-59183311d864 None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:42:02.773665 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c8cac8a-49ac-44ad-be22-59183311d864 None admin] PATH_INFO: `/v3/users` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:42:02.774629 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c8cac8a-49ac-44ad-be22-59183311d864 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80143) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 10:42:02.775042 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c8cac8a-49ac-44ad-be22-59183311d864 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 10:42:02.777232 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c8cac8a-49ac-44ad-be22-59183311d864 None admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 10:42:02.798428 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 274/562] 10.222.0.25 () {56 vars in 1133 bytes} [Sun Aug 7 10:42:02 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 10:42:02.799300 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c8cac8a-49ac-44ad-be22-59183311d864 None admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 10:42:02.864189 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=100 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 10:42:02.879144 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f133712-29b2-42fb-8af0-b40c1d5a794d None None] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 10:42:02.905272 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c8cac8a-49ac-44ad-be22-59183311d864 None admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 10:42:02.919768 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9c12685-634f-4313-bb47-f3d83ff2b17d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 10:42:02.920476 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9c12685-634f-4313-bb47-f3d83ff2b17d None admin] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:42:02.920760 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9c12685-634f-4313-bb47-f3d83ff2b17d None admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:42:02.921034 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9c12685-634f-4313-bb47-f3d83ff2b17d None admin] PATH_INFO: `/v3/roles/service` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:42:02.931802 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9c12685-634f-4313-bb47-f3d83ff2b17d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 07 10:42:02.936761 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application [None req-a9c12685-634f-4313-bb47-f3d83ff2b17d None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 07 10:42:02.936761 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 07 10:42:02.936761 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 07 10:42:02.936761 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 07 10:42:02.936761 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 07 10:42:02.936761 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 07 10:42:02.936761 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 07 10:42:02.936761 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 07 10:42:02.936761 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 07 10:42:02.936761 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 07 10:42:02.936761 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 07 10:42:02.936761 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 07 10:42:02.936761 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 07 10:42:02.936761 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 07 10:42:02.936761 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 07 10:42:02.936761 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 07 10:42:02.937673 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 07 10:42:02.937673 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 07 10:42:02.937673 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 07 10:42:02.937673 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 07 10:42:02.937673 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 07 10:42:02.937673 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 07 10:42:02.937673 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 07 10:42:02.937673 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 07 10:42:02.937673 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 07 10:42:02.937673 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self.get_or_create( Aug 07 10:42:02.937673 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 07 10:42:02.937673 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application with Lock( Aug 07 10:42:02.937673 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 07 10:42:02.937673 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self._enter() Aug 07 10:42:02.937673 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 07 10:42:02.937673 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 07 10:42:02.938541 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 07 10:42:02.938541 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self.creator() Aug 07 10:42:02.938541 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 07 10:42:02.938541 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application created_value = creator( Aug 07 10:42:02.938541 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 07 10:42:02.938541 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 07 10:42:02.938541 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 07 10:42:02.938541 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 07 10:42:02.938541 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 07 10:42:02.938541 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 07 10:42:02.938541 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 07 10:42:02.938541 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application Aug 07 10:42:02.938541 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 289/563] 10.222.0.25 () {56 vars in 1102 bytes} [Sun Aug 7 10:42:02 2022] GET /identity/v3/roles/service => generated 85 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 07 10:42:02.939353 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9c12685-634f-4313-bb47-f3d83ff2b17d None admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 10:42:02.945128 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9ebb020-e636-4fb8-b04a-e915a0773dda None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 10:42:02.945769 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9ebb020-e636-4fb8-b04a-e915a0773dda None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:42:02.946050 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9ebb020-e636-4fb8-b04a-e915a0773dda None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:42:02.946320 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9ebb020-e636-4fb8-b04a-e915a0773dda None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:42:02.956797 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9ebb020-e636-4fb8-b04a-e915a0773dda None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 10:42:02.958235 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9ebb020-e636-4fb8-b04a-e915a0773dda None admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 10:42:02.973165 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application [None req-c9ebb020-e636-4fb8-b04a-e915a0773dda None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 07 10:42:02.973165 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 07 10:42:02.973165 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 07 10:42:02.973165 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 07 10:42:02.973165 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 07 10:42:02.973165 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 07 10:42:02.973165 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 07 10:42:02.973165 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 07 10:42:02.973165 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 07 10:42:02.973165 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 07 10:42:02.973165 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 07 10:42:02.973165 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 07 10:42:02.973165 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 07 10:42:02.973165 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 07 10:42:02.973165 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 07 10:42:02.973165 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 07 10:42:02.974200 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 07 10:42:02.974200 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 07 10:42:02.974200 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 07 10:42:02.974200 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 07 10:42:02.974200 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 07 10:42:02.974200 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.get_or_create( Aug 07 10:42:02.974200 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 07 10:42:02.974200 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application with Lock( Aug 07 10:42:02.974200 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 07 10:42:02.974200 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self._enter() Aug 07 10:42:02.974200 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 07 10:42:02.974200 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 07 10:42:02.974200 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 07 10:42:02.974200 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.creator() Aug 07 10:42:02.974200 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 07 10:42:02.974200 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application created_value = creator( Aug 07 10:42:02.974200 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 07 10:42:02.975515 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 07 10:42:02.975515 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 07 10:42:02.975515 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application Aug 07 10:42:02.975515 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 275/564] 10.222.0.25 () {56 vars in 1108 bytes} [Sun Aug 7 10:42:02 2022] GET /identity/v3/domains/Default => generated 87 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 07 10:42:02.975515 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9ebb020-e636-4fb8-b04a-e915a0773dda None admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 10:42:02.994407 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=100 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 10:42:03.044425 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9c12685-634f-4313-bb47-f3d83ff2b17d None admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 10:42:03.076173 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9ebb020-e636-4fb8-b04a-e915a0773dda None admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 10:42:03.080463 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31f8a65b-4d44-411d-a7b1-f0f8a6afd95c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 10:42:03.081081 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31f8a65b-4d44-411d-a7b1-f0f8a6afd95c None admin] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:42:03.081357 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31f8a65b-4d44-411d-a7b1-f0f8a6afd95c None admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:42:03.081623 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31f8a65b-4d44-411d-a7b1-f0f8a6afd95c None admin] PATH_INFO: `/v3/roles/service` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:42:03.092176 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31f8a65b-4d44-411d-a7b1-f0f8a6afd95c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 07 10:42:03.092630 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=100 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 10:42:03.096921 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application [None req-31f8a65b-4d44-411d-a7b1-f0f8a6afd95c None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 07 10:42:03.096921 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 07 10:42:03.096921 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 07 10:42:03.096921 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 07 10:42:03.096921 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 07 10:42:03.096921 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 07 10:42:03.096921 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 07 10:42:03.096921 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 07 10:42:03.096921 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 07 10:42:03.096921 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 07 10:42:03.096921 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 07 10:42:03.096921 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 07 10:42:03.096921 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 07 10:42:03.096921 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 07 10:42:03.096921 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 07 10:42:03.096921 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 07 10:42:03.097875 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 07 10:42:03.097875 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 07 10:42:03.097875 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 07 10:42:03.097875 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 07 10:42:03.097875 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 07 10:42:03.097875 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 07 10:42:03.097875 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 07 10:42:03.097875 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 07 10:42:03.097875 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 07 10:42:03.097875 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self.get_or_create( Aug 07 10:42:03.097875 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 07 10:42:03.097875 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application with Lock( Aug 07 10:42:03.097875 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 07 10:42:03.097875 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self._enter() Aug 07 10:42:03.097875 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 07 10:42:03.097875 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 07 10:42:03.098683 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 07 10:42:03.098683 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self.creator() Aug 07 10:42:03.098683 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 07 10:42:03.098683 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application created_value = creator( Aug 07 10:42:03.098683 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 07 10:42:03.098683 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 07 10:42:03.098683 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 07 10:42:03.098683 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 07 10:42:03.098683 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 07 10:42:03.098683 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 07 10:42:03.098683 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 07 10:42:03.098683 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application Aug 07 10:42:03.098683 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 290/565] 10.222.0.25 () {56 vars in 1102 bytes} [Sun Aug 7 10:42:02 2022] GET /identity/v3/roles/service => generated 85 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 07 10:42:03.099400 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31f8a65b-4d44-411d-a7b1-f0f8a6afd95c None admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 10:42:03.113756 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67696c92-96d4-4e47-9b8d-a8ba4353c119 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 10:42:03.114417 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67696c92-96d4-4e47-9b8d-a8ba4353c119 None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:42:03.114695 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67696c92-96d4-4e47-9b8d-a8ba4353c119 None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:42:03.114980 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67696c92-96d4-4e47-9b8d-a8ba4353c119 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:42:03.125622 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67696c92-96d4-4e47-9b8d-a8ba4353c119 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 10:42:03.127093 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67696c92-96d4-4e47-9b8d-a8ba4353c119 None admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 10:42:03.141775 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application [None req-67696c92-96d4-4e47-9b8d-a8ba4353c119 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 07 10:42:03.141775 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 07 10:42:03.141775 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 07 10:42:03.141775 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 07 10:42:03.141775 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 07 10:42:03.141775 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 07 10:42:03.141775 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 07 10:42:03.141775 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 07 10:42:03.141775 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 07 10:42:03.141775 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 07 10:42:03.141775 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 07 10:42:03.141775 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 07 10:42:03.141775 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 07 10:42:03.141775 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 07 10:42:03.141775 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 07 10:42:03.141775 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 07 10:42:03.142885 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 07 10:42:03.142885 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 07 10:42:03.142885 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 07 10:42:03.142885 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 07 10:42:03.142885 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 07 10:42:03.142885 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.get_or_create( Aug 07 10:42:03.142885 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 07 10:42:03.142885 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application with Lock( Aug 07 10:42:03.142885 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 07 10:42:03.142885 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self._enter() Aug 07 10:42:03.142885 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 07 10:42:03.142885 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 07 10:42:03.142885 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 07 10:42:03.142885 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.creator() Aug 07 10:42:03.142885 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 07 10:42:03.142885 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application created_value = creator( Aug 07 10:42:03.142885 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 07 10:42:03.143803 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 07 10:42:03.143803 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 07 10:42:03.143803 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application Aug 07 10:42:03.143803 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 276/566] 10.222.0.25 () {56 vars in 1108 bytes} [Sun Aug 7 10:42:02 2022] GET /identity/v3/domains/Default => generated 87 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 07 10:42:03.144035 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54eee7f3-212a-497c-8c5f-4e1fb3459f26 None None] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:42:03.144588 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54eee7f3-212a-497c-8c5f-4e1fb3459f26 None None] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:42:03.144588 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54eee7f3-212a-497c-8c5f-4e1fb3459f26 None None] PATH_INFO: `/` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:42:03.145742 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 277/567] 10.222.0.25 () {54 vars in 927 bytes} [Sun Aug 7 10:42:03 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 07 10:42:03.146432 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54eee7f3-212a-497c-8c5f-4e1fb3459f26 None None] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 10:42:03.204558 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31f8a65b-4d44-411d-a7b1-f0f8a6afd95c None admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 10:42:03.213656 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=100 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 10:42:03.239218 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b481e538-95f7-4359-8883-d1acbd5c7f29 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 10:42:03.239826 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b481e538-95f7-4359-8883-d1acbd5c7f29 None admin] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:42:03.240110 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b481e538-95f7-4359-8883-d1acbd5c7f29 None admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:42:03.240386 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b481e538-95f7-4359-8883-d1acbd5c7f29 None admin] PATH_INFO: `/v3/roles` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:42:03.241040 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b481e538-95f7-4359-8883-d1acbd5c7f29 None admin] RBAC: Adding query filter params (name=service) {{(pid=80144) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 10:42:03.241347 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b481e538-95f7-4359-8883-d1acbd5c7f29 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 10:42:03.242695 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b481e538-95f7-4359-8883-d1acbd5c7f29 None admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 10:42:03.247143 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54eee7f3-212a-497c-8c5f-4e1fb3459f26 None None] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 10:42:03.250964 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 291/568] 10.222.0.25 () {56 vars in 1103 bytes} [Sun Aug 7 10:42:03 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 10:42:03.251751 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b481e538-95f7-4359-8883-d1acbd5c7f29 None admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 10:42:03.281898 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a27934e-e8b2-42e9-b6e9-9426243a315c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 10:42:03.282571 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a27934e-e8b2-42e9-b6e9-9426243a315c None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:42:03.282764 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a27934e-e8b2-42e9-b6e9-9426243a315c None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:42:03.283060 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a27934e-e8b2-42e9-b6e9-9426243a315c None admin] PATH_INFO: `/v3/domains` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:42:03.283696 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a27934e-e8b2-42e9-b6e9-9426243a315c None admin] RBAC: Adding query filter params (name=Default) {{(pid=80143) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 10:42:03.284103 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a27934e-e8b2-42e9-b6e9-9426243a315c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 10:42:03.285348 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a27934e-e8b2-42e9-b6e9-9426243a315c None admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 10:42:03.297457 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 278/569] 10.222.0.25 () {56 vars in 1109 bytes} [Sun Aug 7 10:42:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 10:42:03.298260 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a27934e-e8b2-42e9-b6e9-9426243a315c None admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 10:42:03.326567 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=100 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 10:42:03.355606 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b481e538-95f7-4359-8883-d1acbd5c7f29 None admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 10:42:03.388669 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0eaec16-a9cb-4f27-b97b-1874703ed0d7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 10:42:03.389273 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0eaec16-a9cb-4f27-b97b-1874703ed0d7 None admin] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:42:03.389556 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0eaec16-a9cb-4f27-b97b-1874703ed0d7 None admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:42:03.389836 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0eaec16-a9cb-4f27-b97b-1874703ed0d7 None admin] PATH_INFO: `/v3/roles` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:42:03.390495 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0eaec16-a9cb-4f27-b97b-1874703ed0d7 None admin] RBAC: Adding query filter params (name=service) {{(pid=80144) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 10:42:03.390802 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0eaec16-a9cb-4f27-b97b-1874703ed0d7 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 10:42:03.392181 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0eaec16-a9cb-4f27-b97b-1874703ed0d7 None admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 10:42:03.397764 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a27934e-e8b2-42e9-b6e9-9426243a315c None admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 10:42:03.399886 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 292/570] 10.222.0.25 () {56 vars in 1103 bytes} [Sun Aug 7 10:42:03 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 10:42:03.401143 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a731f768-234e-4896-afd0-16e0aba29c01 None None] REQUEST_METHOD: `POST` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:42:03.401464 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a731f768-234e-4896-afd0-16e0aba29c01 None None] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:42:03.401739 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a731f768-234e-4896-afd0-16e0aba29c01 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:42:03.419298 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.auth.core [None req-a731f768-234e-4896-afd0-16e0aba29c01 None None] MFA Rules not processed for user `bac4e6b76fe64217ac8fa44e71cffe6a`. Rule list: `[]` (Enabled: `True`). {{(pid=80144) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 10:42:03.420475 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.fernet_utils [None req-a731f768-234e-4896-afd0-16e0aba29c01 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80144) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 10:42:03.435470 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8431a6c-4e2c-4d27-875d-52f076ee78a5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 10:42:03.436073 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8431a6c-4e2c-4d27-875d-52f076ee78a5 None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:42:03.436348 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8431a6c-4e2c-4d27-875d-52f076ee78a5 None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:42:03.436617 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8431a6c-4e2c-4d27-875d-52f076ee78a5 None admin] PATH_INFO: `/v3/domains` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:42:03.437251 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8431a6c-4e2c-4d27-875d-52f076ee78a5 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80143) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 10:42:03.437550 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8431a6c-4e2c-4d27-875d-52f076ee78a5 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 10:42:03.438895 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8431a6c-4e2c-4d27-875d-52f076ee78a5 None admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 10:42:03.445837 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=100 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 10:42:03.451096 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 279/571] 10.222.0.25 () {56 vars in 1109 bytes} [Sun Aug 7 10:42:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 10:42:03.451906 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8431a6c-4e2c-4d27-875d-52f076ee78a5 None admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 10:42:03.492829 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 293/572] 10.222.0.25 () {58 vars in 1024 bytes} [Sun Aug 7 10:42:03 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 07 10:42:03.493587 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a731f768-234e-4896-afd0-16e0aba29c01 None None] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 10:42:03.537384 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=100 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 10:42:03.550191 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8431a6c-4e2c-4d27-875d-52f076ee78a5 None admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 10:42:03.582997 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62c6fee7-2a85-42d4-8e58-7e14f9e286bd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 10:42:03.583627 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62c6fee7-2a85-42d4-8e58-7e14f9e286bd None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:42:03.583905 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62c6fee7-2a85-42d4-8e58-7e14f9e286bd None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:42:03.584224 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62c6fee7-2a85-42d4-8e58-7e14f9e286bd None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:42:03.595361 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62c6fee7-2a85-42d4-8e58-7e14f9e286bd None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 10:42:03.595504 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a731f768-234e-4896-afd0-16e0aba29c01 None None] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 10:42:03.596802 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62c6fee7-2a85-42d4-8e58-7e14f9e286bd None admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 10:42:03.611567 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application [None req-62c6fee7-2a85-42d4-8e58-7e14f9e286bd None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 07 10:42:03.611567 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 07 10:42:03.611567 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 07 10:42:03.611567 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 07 10:42:03.611567 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 07 10:42:03.611567 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 07 10:42:03.611567 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 07 10:42:03.611567 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 07 10:42:03.611567 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 07 10:42:03.611567 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 07 10:42:03.611567 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 07 10:42:03.611567 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 07 10:42:03.611567 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 07 10:42:03.611567 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 07 10:42:03.611567 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 07 10:42:03.611567 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 07 10:42:03.617361 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 07 10:42:03.617361 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 07 10:42:03.617361 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 07 10:42:03.617361 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 07 10:42:03.617361 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 07 10:42:03.617361 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.get_or_create( Aug 07 10:42:03.617361 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 07 10:42:03.617361 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application with Lock( Aug 07 10:42:03.617361 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 07 10:42:03.617361 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self._enter() Aug 07 10:42:03.617361 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 07 10:42:03.617361 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 07 10:42:03.617361 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 07 10:42:03.617361 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.creator() Aug 07 10:42:03.617361 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 07 10:42:03.617361 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application created_value = creator( Aug 07 10:42:03.617361 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 07 10:42:03.618591 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 07 10:42:03.618591 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 07 10:42:03.618591 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application Aug 07 10:42:03.618591 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 280/573] 10.222.0.25 () {56 vars in 1108 bytes} [Sun Aug 7 10:42:03 2022] GET /identity/v3/domains/Default => generated 87 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 07 10:42:03.618591 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62c6fee7-2a85-42d4-8e58-7e14f9e286bd None admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 10:42:03.630441 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6185a959-9872-4c16-a0d8-bd71058e8130 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 10:42:03.631102 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6185a959-9872-4c16-a0d8-bd71058e8130 None admin] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:42:03.631385 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6185a959-9872-4c16-a0d8-bd71058e8130 None admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:42:03.631657 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6185a959-9872-4c16-a0d8-bd71058e8130 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:42:03.638649 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6185a959-9872-4c16-a0d8-bd71058e8130 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 10:42:03.640126 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6185a959-9872-4c16-a0d8-bd71058e8130 None admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 10:42:03.652498 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application [None req-6185a959-9872-4c16-a0d8-bd71058e8130 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 07 10:42:03.652498 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 07 10:42:03.652498 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 07 10:42:03.652498 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 07 10:42:03.652498 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 07 10:42:03.652498 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 07 10:42:03.652498 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 07 10:42:03.652498 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 07 10:42:03.652498 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 07 10:42:03.652498 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 07 10:42:03.652498 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 07 10:42:03.652498 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 07 10:42:03.652498 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 07 10:42:03.652498 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 07 10:42:03.652498 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 07 10:42:03.653448 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 07 10:42:03.653448 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 07 10:42:03.653448 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 07 10:42:03.653448 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 07 10:42:03.653448 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 07 10:42:03.653448 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 07 10:42:03.653448 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self.get_or_create( Aug 07 10:42:03.653448 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 07 10:42:03.653448 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application with Lock( Aug 07 10:42:03.653448 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 07 10:42:03.653448 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self._enter() Aug 07 10:42:03.653448 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 07 10:42:03.653448 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 07 10:42:03.653448 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 07 10:42:03.653448 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self.creator() Aug 07 10:42:03.653448 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 07 10:42:03.653448 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application created_value = creator( Aug 07 10:42:03.654363 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 07 10:42:03.654363 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 07 10:42:03.654363 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 07 10:42:03.654363 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 07 10:42:03.654363 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 07 10:42:03.654363 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 07 10:42:03.654363 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 07 10:42:03.654363 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application Aug 07 10:42:03.654363 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 294/574] 10.222.0.25 () {56 vars in 1111 bytes} [Sun Aug 7 10:42:03 2022] GET /identity/v3/projects/service => generated 88 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 07 10:42:03.654849 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6185a959-9872-4c16-a0d8-bd71058e8130 None admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 10:42:03.681644 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=100 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 10:42:03.715128 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62c6fee7-2a85-42d4-8e58-7e14f9e286bd None admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 10:42:03.748795 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13b371a2-f5c2-437b-bca1-60537770770d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 10:42:03.749356 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13b371a2-f5c2-437b-bca1-60537770770d None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:42:03.749637 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13b371a2-f5c2-437b-bca1-60537770770d None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:42:03.749913 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13b371a2-f5c2-437b-bca1-60537770770d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:42:03.756551 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6185a959-9872-4c16-a0d8-bd71058e8130 None admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 10:42:03.759189 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=100 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 10:42:03.761656 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13b371a2-f5c2-437b-bca1-60537770770d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 10:42:03.763492 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13b371a2-f5c2-437b-bca1-60537770770d None admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 10:42:03.778208 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application [None req-13b371a2-f5c2-437b-bca1-60537770770d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 07 10:42:03.778208 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 07 10:42:03.778208 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 07 10:42:03.778208 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 07 10:42:03.778208 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 07 10:42:03.778208 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 07 10:42:03.778208 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 07 10:42:03.778208 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 07 10:42:03.778208 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 07 10:42:03.778208 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 07 10:42:03.778208 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 07 10:42:03.778208 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 07 10:42:03.778208 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 07 10:42:03.778208 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 07 10:42:03.778208 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 07 10:42:03.778208 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 07 10:42:03.779238 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 07 10:42:03.779238 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 07 10:42:03.779238 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 07 10:42:03.779238 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 07 10:42:03.779238 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 07 10:42:03.779238 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.get_or_create( Aug 07 10:42:03.779238 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 07 10:42:03.779238 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application with Lock( Aug 07 10:42:03.779238 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 07 10:42:03.779238 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self._enter() Aug 07 10:42:03.779238 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 07 10:42:03.779238 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 07 10:42:03.779238 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 07 10:42:03.779238 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.creator() Aug 07 10:42:03.779238 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 07 10:42:03.779238 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application created_value = creator( Aug 07 10:42:03.779238 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 07 10:42:03.780114 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 07 10:42:03.780114 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 07 10:42:03.780114 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application Aug 07 10:42:03.780114 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 281/575] 10.222.0.25 () {56 vars in 1108 bytes} [Sun Aug 7 10:42:03 2022] GET /identity/v3/domains/Default => generated 87 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 07 10:42:03.780345 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13b371a2-f5c2-437b-bca1-60537770770d None admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 10:42:03.792002 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af7b09f5-ff56-4a85-ab43-225b13ecc40a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 10:42:03.792629 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af7b09f5-ff56-4a85-ab43-225b13ecc40a None admin] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:42:03.792905 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af7b09f5-ff56-4a85-ab43-225b13ecc40a None admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:42:03.793172 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af7b09f5-ff56-4a85-ab43-225b13ecc40a None admin] PATH_INFO: `/v3/projects/service` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:42:03.799952 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af7b09f5-ff56-4a85-ab43-225b13ecc40a None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 10:42:03.801418 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af7b09f5-ff56-4a85-ab43-225b13ecc40a None admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 10:42:03.812478 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application [None req-af7b09f5-ff56-4a85-ab43-225b13ecc40a None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 07 10:42:03.812478 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 07 10:42:03.812478 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 07 10:42:03.812478 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 07 10:42:03.812478 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 07 10:42:03.812478 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 07 10:42:03.812478 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 07 10:42:03.812478 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 07 10:42:03.812478 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 07 10:42:03.812478 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 07 10:42:03.812478 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 07 10:42:03.812478 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 07 10:42:03.812478 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 07 10:42:03.812478 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 07 10:42:03.812478 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 07 10:42:03.813488 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 07 10:42:03.813488 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 07 10:42:03.813488 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 07 10:42:03.813488 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 07 10:42:03.813488 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 07 10:42:03.813488 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 07 10:42:03.813488 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self.get_or_create( Aug 07 10:42:03.813488 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 07 10:42:03.813488 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application with Lock( Aug 07 10:42:03.813488 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 07 10:42:03.813488 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self._enter() Aug 07 10:42:03.813488 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 07 10:42:03.813488 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 07 10:42:03.813488 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 07 10:42:03.813488 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self.creator() Aug 07 10:42:03.813488 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 07 10:42:03.813488 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application created_value = creator( Aug 07 10:42:03.814526 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 07 10:42:03.814526 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 07 10:42:03.814526 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 07 10:42:03.814526 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 07 10:42:03.814526 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 07 10:42:03.814526 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 07 10:42:03.814526 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 07 10:42:03.814526 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application Aug 07 10:42:03.815185 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 295/576] 10.222.0.25 () {56 vars in 1111 bytes} [Sun Aug 7 10:42:03 2022] GET /identity/v3/projects/service => generated 88 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 07 10:42:03.815862 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af7b09f5-ff56-4a85-ab43-225b13ecc40a None admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 10:42:03.881310 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13b371a2-f5c2-437b-bca1-60537770770d None admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 10:42:03.896156 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=100 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 10:42:03.914788 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af7b09f5-ff56-4a85-ab43-225b13ecc40a None admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 10:42:03.915808 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f68864d8-f2be-40d6-836e-90842ac4eff8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 10:42:03.917004 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f68864d8-f2be-40d6-836e-90842ac4eff8 None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:42:03.917285 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f68864d8-f2be-40d6-836e-90842ac4eff8 None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:42:03.917562 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f68864d8-f2be-40d6-836e-90842ac4eff8 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:42:03.927537 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f68864d8-f2be-40d6-836e-90842ac4eff8 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 07 10:42:03.932742 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application [None req-f68864d8-f2be-40d6-836e-90842ac4eff8 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 07 10:42:03.932742 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 07 10:42:03.932742 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 07 10:42:03.932742 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 07 10:42:03.932742 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 07 10:42:03.932742 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 07 10:42:03.932742 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 07 10:42:03.932742 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 07 10:42:03.932742 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 07 10:42:03.932742 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 07 10:42:03.932742 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 07 10:42:03.932742 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 07 10:42:03.932742 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 07 10:42:03.932742 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 07 10:42:03.932742 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 07 10:42:03.932742 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 07 10:42:03.933694 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 07 10:42:03.933694 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 07 10:42:03.933694 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 07 10:42:03.933694 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 07 10:42:03.933694 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 07 10:42:03.933694 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 07 10:42:03.933694 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 07 10:42:03.933694 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 07 10:42:03.933694 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 07 10:42:03.933694 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.get_or_create( Aug 07 10:42:03.933694 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 07 10:42:03.933694 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application with Lock( Aug 07 10:42:03.933694 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 07 10:42:03.933694 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self._enter() Aug 07 10:42:03.933694 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 07 10:42:03.933694 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 07 10:42:03.934506 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 07 10:42:03.934506 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.creator() Aug 07 10:42:03.934506 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 07 10:42:03.934506 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application created_value = creator( Aug 07 10:42:03.934506 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 07 10:42:03.934506 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 07 10:42:03.934506 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 07 10:42:03.934506 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 07 10:42:03.934506 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 07 10:42:03.934506 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 07 10:42:03.934506 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 07 10:42:03.934506 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application Aug 07 10:42:03.934506 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 282/577] 10.222.0.25 () {56 vars in 1102 bytes} [Sun Aug 7 10:42:03 2022] GET /identity/v3/roles/service => generated 85 bytes in 154 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 07 10:42:03.935229 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f68864d8-f2be-40d6-836e-90842ac4eff8 None admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 10:42:03.951562 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18f2d623-69bc-49d1-989b-8bff99a5e69e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 10:42:03.952252 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18f2d623-69bc-49d1-989b-8bff99a5e69e None admin] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:42:03.952447 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18f2d623-69bc-49d1-989b-8bff99a5e69e None admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:42:03.952722 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18f2d623-69bc-49d1-989b-8bff99a5e69e None admin] PATH_INFO: `/v3/domains` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:42:03.953361 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18f2d623-69bc-49d1-989b-8bff99a5e69e None admin] RBAC: Adding query filter params (name=Default) {{(pid=80144) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 10:42:03.953664 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18f2d623-69bc-49d1-989b-8bff99a5e69e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 10:42:03.955036 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18f2d623-69bc-49d1-989b-8bff99a5e69e None admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 10:42:03.972146 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 296/578] 10.222.0.25 () {56 vars in 1109 bytes} [Sun Aug 7 10:42:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 10:42:03.972146 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18f2d623-69bc-49d1-989b-8bff99a5e69e None admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 10:42:03.994899 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=100 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 10:42:04.043091 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f68864d8-f2be-40d6-836e-90842ac4eff8 None admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 10:42:04.063802 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18f2d623-69bc-49d1-989b-8bff99a5e69e None admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 10:42:04.077130 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2be04a72-a23b-40ff-8880-90b9961f9384 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 10:42:04.077847 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2be04a72-a23b-40ff-8880-90b9961f9384 None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:42:04.078135 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2be04a72-a23b-40ff-8880-90b9961f9384 None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:42:04.078417 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2be04a72-a23b-40ff-8880-90b9961f9384 None admin] PATH_INFO: `/v3/projects` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:42:04.079137 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2be04a72-a23b-40ff-8880-90b9961f9384 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80143) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 10:42:04.079446 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2be04a72-a23b-40ff-8880-90b9961f9384 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 10:42:04.080869 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2be04a72-a23b-40ff-8880-90b9961f9384 None admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 10:42:04.093573 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 283/579] 10.222.0.25 () {56 vars in 1148 bytes} [Sun Aug 7 10:42:03 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 10:42:04.094536 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2be04a72-a23b-40ff-8880-90b9961f9384 None admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 10:42:04.104211 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f69595ca-8205-4ffe-9a41-118219377a92 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 10:42:04.104822 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f69595ca-8205-4ffe-9a41-118219377a92 None admin] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:42:04.105102 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f69595ca-8205-4ffe-9a41-118219377a92 None admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:42:04.105373 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f69595ca-8205-4ffe-9a41-118219377a92 None admin] PATH_INFO: `/v3/domains` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:42:04.106015 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f69595ca-8205-4ffe-9a41-118219377a92 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80144) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 10:42:04.106318 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f69595ca-8205-4ffe-9a41-118219377a92 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 10:42:04.107704 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f69595ca-8205-4ffe-9a41-118219377a92 None admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 10:42:04.118985 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 297/580] 10.222.0.25 () {56 vars in 1109 bytes} [Sun Aug 7 10:42:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 10:42:04.119794 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f69595ca-8205-4ffe-9a41-118219377a92 None admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 10:42:04.122738 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=100 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 10:42:04.191728 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2be04a72-a23b-40ff-8880-90b9961f9384 None admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 10:42:04.204972 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=100 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 10:42:04.220099 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f69595ca-8205-4ffe-9a41-118219377a92 None admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 10:42:04.226937 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8f6e366-a623-4ea4-86f7-9dca1fb88153 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 10:42:04.227563 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8f6e366-a623-4ea4-86f7-9dca1fb88153 None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:42:04.227844 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8f6e366-a623-4ea4-86f7-9dca1fb88153 None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:42:04.228120 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8f6e366-a623-4ea4-86f7-9dca1fb88153 None admin] PATH_INFO: `/v3/projects` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:42:04.229316 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8f6e366-a623-4ea4-86f7-9dca1fb88153 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80143) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 10:42:04.229624 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8f6e366-a623-4ea4-86f7-9dca1fb88153 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 10:42:04.231073 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8f6e366-a623-4ea4-86f7-9dca1fb88153 None admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 10:42:04.242773 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 284/581] 10.222.0.25 () {56 vars in 1148 bytes} [Sun Aug 7 10:42:04 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 10:42:04.243594 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8f6e366-a623-4ea4-86f7-9dca1fb88153 None admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 10:42:04.258605 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67807572-32a1-47ea-829a-87dcb43decbb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 10:42:04.259241 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67807572-32a1-47ea-829a-87dcb43decbb None admin] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:42:04.259519 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67807572-32a1-47ea-829a-87dcb43decbb None admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:42:04.259830 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67807572-32a1-47ea-829a-87dcb43decbb None admin] PATH_INFO: `/v3/roles` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:42:04.261082 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67807572-32a1-47ea-829a-87dcb43decbb None admin] RBAC: Adding query filter params (name=service) {{(pid=80144) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 10:42:04.261082 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67807572-32a1-47ea-829a-87dcb43decbb None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 10:42:04.262375 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67807572-32a1-47ea-829a-87dcb43decbb None admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 10:42:04.270151 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 298/582] 10.222.0.25 () {56 vars in 1103 bytes} [Sun Aug 7 10:42:04 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 10:42:04.271025 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67807572-32a1-47ea-829a-87dcb43decbb None admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 10:42:04.331907 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=100 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 10:42:04.344597 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8f6e366-a623-4ea4-86f7-9dca1fb88153 None admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 10:42:04.370860 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67807572-32a1-47ea-829a-87dcb43decbb None admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 10:42:04.378518 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec9744a7-70d5-47c1-a0d9-4112cf99e5c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 10:42:04.379176 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec9744a7-70d5-47c1-a0d9-4112cf99e5c9 None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:42:04.379460 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec9744a7-70d5-47c1-a0d9-4112cf99e5c9 None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:42:04.379737 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec9744a7-70d5-47c1-a0d9-4112cf99e5c9 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:42:04.387504 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec9744a7-70d5-47c1-a0d9-4112cf99e5c9 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 10:42:04.388933 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec9744a7-70d5-47c1-a0d9-4112cf99e5c9 None admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 10:42:04.399668 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application [None req-ec9744a7-70d5-47c1-a0d9-4112cf99e5c9 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Aug 07 10:42:04.399668 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 07 10:42:04.399668 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 07 10:42:04.399668 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 07 10:42:04.399668 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 07 10:42:04.399668 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 07 10:42:04.399668 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 07 10:42:04.399668 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 07 10:42:04.399668 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 07 10:42:04.399668 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 07 10:42:04.399668 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 07 10:42:04.399668 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 07 10:42:04.399668 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 07 10:42:04.399668 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 07 10:42:04.399668 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 07 10:42:04.399668 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 07 10:42:04.400828 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 07 10:42:04.400828 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 07 10:42:04.400828 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 07 10:42:04.400828 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 07 10:42:04.400828 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 07 10:42:04.400828 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 07 10:42:04.400828 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 07 10:42:04.400828 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 07 10:42:04.400828 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 07 10:42:04.400828 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.get_or_create( Aug 07 10:42:04.400828 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 07 10:42:04.400828 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application with Lock( Aug 07 10:42:04.400828 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 07 10:42:04.400828 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self._enter() Aug 07 10:42:04.400828 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 07 10:42:04.400828 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 07 10:42:04.400828 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 07 10:42:04.400828 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.creator() Aug 07 10:42:04.402379 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 07 10:42:04.402379 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application created_value = creator( Aug 07 10:42:04.402379 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 07 10:42:04.402379 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 07 10:42:04.402379 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 07 10:42:04.402379 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 07 10:42:04.402379 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 07 10:42:04.402379 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 07 10:42:04.402379 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Aug 07 10:42:04.402379 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application Aug 07 10:42:04.402379 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 285/583] 10.222.0.25 () {56 vars in 1099 bytes} [Sun Aug 7 10:42:04 2022] GET /identity/v3/users/cinder => generated 84 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 07 10:42:04.402379 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec9744a7-70d5-47c1-a0d9-4112cf99e5c9 None admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 10:42:04.409417 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39d25b86-3973-4c48-ae9c-e1ab1748cbe4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 10:42:04.410027 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39d25b86-3973-4c48-ae9c-e1ab1748cbe4 None admin] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:42:04.410314 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39d25b86-3973-4c48-ae9c-e1ab1748cbe4 None admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:42:04.410588 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39d25b86-3973-4c48-ae9c-e1ab1748cbe4 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:42:04.411345 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39d25b86-3973-4c48-ae9c-e1ab1748cbe4 None admin] RBAC: Adding query filter params (role.id=fbf375e9760340c9aa9529333eeb25f9, scope.project.id=6de1f3cac02c4bcdb05dd2337176033d, user.id=69490ea2a9a340bf91aca38bf5655b45) {{(pid=80144) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 10:42:04.411653 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39d25b86-3973-4c48-ae9c-e1ab1748cbe4 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 10:42:04.413063 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39d25b86-3973-4c48-ae9c-e1ab1748cbe4 None admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 10:42:04.422285 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 299/584] 10.222.0.25 () {56 vars in 1374 bytes} [Sun Aug 7 10:42:04 2022] GET /identity/v3/role_assignments?user.id=69490ea2a9a340bf91aca38bf5655b45&scope.project.id=6de1f3cac02c4bcdb05dd2337176033d&role.id=fbf375e9760340c9aa9529333eeb25f9 => generated 259 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 10:42:04.423108 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39d25b86-3973-4c48-ae9c-e1ab1748cbe4 None admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 10:42:04.444691 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=100 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 10:42:04.524897 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec9744a7-70d5-47c1-a0d9-4112cf99e5c9 None admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 10:42:04.537821 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39d25b86-3973-4c48-ae9c-e1ab1748cbe4 None admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 10:42:04.567244 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9d8ba64-adaf-45dc-b87d-6c3b33f00810 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 10:42:04.568282 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9d8ba64-adaf-45dc-b87d-6c3b33f00810 None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:42:04.568622 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9d8ba64-adaf-45dc-b87d-6c3b33f00810 None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:42:04.568743 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=100 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 10:42:04.568968 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9d8ba64-adaf-45dc-b87d-6c3b33f00810 None admin] PATH_INFO: `/v3/users/glance` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:42:04.578011 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9d8ba64-adaf-45dc-b87d-6c3b33f00810 None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 10:42:04.579097 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ad77848-4e26-4333-a0a1-fcf66f8df8ca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 10:42:04.579490 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9d8ba64-adaf-45dc-b87d-6c3b33f00810 None admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 10:42:04.579760 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ad77848-4e26-4333-a0a1-fcf66f8df8ca None admin] REQUEST_METHOD: `PUT` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:42:04.580653 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ad77848-4e26-4333-a0a1-fcf66f8df8ca None admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:42:04.580653 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ad77848-4e26-4333-a0a1-fcf66f8df8ca None admin] PATH_INFO: `/v3/projects/6de1f3cac02c4bcdb05dd2337176033d/users/31d16d1b96104b4eb690398432b9829e/roles/fbf375e9760340c9aa9529333eeb25f9` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:42:04.592566 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application [None req-a9d8ba64-adaf-45dc-b87d-6c3b33f00810 None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Aug 07 10:42:04.592566 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 07 10:42:04.592566 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 07 10:42:04.592566 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 07 10:42:04.592566 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 07 10:42:04.592566 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 07 10:42:04.592566 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 07 10:42:04.592566 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 07 10:42:04.592566 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 07 10:42:04.592566 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 07 10:42:04.592566 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 07 10:42:04.592566 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 07 10:42:04.592566 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 07 10:42:04.592566 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 07 10:42:04.592566 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 07 10:42:04.592566 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 07 10:42:04.593508 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 07 10:42:04.593508 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 07 10:42:04.593508 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 07 10:42:04.593508 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 07 10:42:04.593508 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 07 10:42:04.593508 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 07 10:42:04.593508 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 07 10:42:04.593508 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 07 10:42:04.593508 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 07 10:42:04.593508 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.get_or_create( Aug 07 10:42:04.593508 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 07 10:42:04.593508 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application with Lock( Aug 07 10:42:04.593508 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 07 10:42:04.593508 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self._enter() Aug 07 10:42:04.593508 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 07 10:42:04.593508 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 07 10:42:04.593508 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 07 10:42:04.593508 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.creator() Aug 07 10:42:04.594425 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 07 10:42:04.594425 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application created_value = creator( Aug 07 10:42:04.594425 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 07 10:42:04.594425 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 07 10:42:04.594425 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 07 10:42:04.594425 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 07 10:42:04.594425 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 07 10:42:04.594425 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 07 10:42:04.594425 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Aug 07 10:42:04.594425 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application Aug 07 10:42:04.594425 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 286/585] 10.222.0.25 () {56 vars in 1099 bytes} [Sun Aug 7 10:42:04 2022] GET /identity/v3/users/glance => generated 84 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 07 10:42:04.595056 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9d8ba64-adaf-45dc-b87d-6c3b33f00810 None admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 10:42:04.601234 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ad77848-4e26-4333-a0a1-fcf66f8df8ca None admin] RBAC: Authorizing `identity:create_grant(project_id=6de1f3cac02c4bcdb05dd2337176033d, user_id=31d16d1b96104b4eb690398432b9829e, role_id=fbf375e9760340c9aa9529333eeb25f9)` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 10:42:04.602662 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ad77848-4e26-4333-a0a1-fcf66f8df8ca None admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 10:42:04.626965 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 300/586] 10.222.0.25 () {58 vars in 1439 bytes} [Sun Aug 7 10:42:04 2022] PUT /identity/v3/projects/6de1f3cac02c4bcdb05dd2337176033d/users/31d16d1b96104b4eb690398432b9829e/roles/fbf375e9760340c9aa9529333eeb25f9 => generated 0 bytes in 194 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 07 10:42:04.626965 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ad77848-4e26-4333-a0a1-fcf66f8df8ca None admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 10:42:04.709034 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=100 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 10:42:04.721778 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9d8ba64-adaf-45dc-b87d-6c3b33f00810 None admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 10:42:04.760335 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ad77848-4e26-4333-a0a1-fcf66f8df8ca None admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 10:42:04.765802 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1fcc14e-7db2-4386-85a5-6ffdfe3a8ecb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 10:42:04.766459 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1fcc14e-7db2-4386-85a5-6ffdfe3a8ecb None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:42:04.766740 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1fcc14e-7db2-4386-85a5-6ffdfe3a8ecb None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:42:04.767088 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1fcc14e-7db2-4386-85a5-6ffdfe3a8ecb None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:42:04.780677 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1fcc14e-7db2-4386-85a5-6ffdfe3a8ecb None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 10:42:04.780994 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1fcc14e-7db2-4386-85a5-6ffdfe3a8ecb None admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 10:42:04.797085 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application [None req-a1fcc14e-7db2-4386-85a5-6ffdfe3a8ecb None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 07 10:42:04.797085 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 07 10:42:04.797085 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 07 10:42:04.797085 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 07 10:42:04.797085 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 07 10:42:04.797085 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 07 10:42:04.797085 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 07 10:42:04.797085 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 07 10:42:04.797085 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 07 10:42:04.797085 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 07 10:42:04.797085 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 07 10:42:04.797085 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 07 10:42:04.797085 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 07 10:42:04.797085 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 07 10:42:04.797085 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 07 10:42:04.797085 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 07 10:42:04.798078 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 07 10:42:04.798078 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 07 10:42:04.798078 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 07 10:42:04.798078 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 07 10:42:04.798078 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 07 10:42:04.798078 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.get_or_create( Aug 07 10:42:04.798078 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 07 10:42:04.798078 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application with Lock( Aug 07 10:42:04.798078 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 07 10:42:04.798078 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self._enter() Aug 07 10:42:04.798078 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 07 10:42:04.798078 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 07 10:42:04.798078 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 07 10:42:04.798078 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.creator() Aug 07 10:42:04.798078 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 07 10:42:04.798078 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application created_value = creator( Aug 07 10:42:04.798078 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 07 10:42:04.798986 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 07 10:42:04.798986 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 07 10:42:04.798986 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application Aug 07 10:42:04.798986 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 287/587] 10.222.0.25 () {56 vars in 1108 bytes} [Sun Aug 7 10:42:04 2022] GET /identity/v3/domains/Default => generated 87 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 07 10:42:04.799835 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1fcc14e-7db2-4386-85a5-6ffdfe3a8ecb None admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 10:42:04.802458 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72a25ab6-5f11-4fd0-b0ff-f29db169a4bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 10:42:04.803520 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72a25ab6-5f11-4fd0-b0ff-f29db169a4bf None admin] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:42:04.804599 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72a25ab6-5f11-4fd0-b0ff-f29db169a4bf None admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:42:04.804924 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72a25ab6-5f11-4fd0-b0ff-f29db169a4bf None admin] PATH_INFO: `/v3/users` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:42:04.805677 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72a25ab6-5f11-4fd0-b0ff-f29db169a4bf None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=80144) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 10:42:04.805987 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72a25ab6-5f11-4fd0-b0ff-f29db169a4bf None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 10:42:04.808004 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72a25ab6-5f11-4fd0-b0ff-f29db169a4bf None admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 10:42:04.827879 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 301/588] 10.222.0.25 () {56 vars in 1137 bytes} [Sun Aug 7 10:42:04 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 10:42:04.829042 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72a25ab6-5f11-4fd0-b0ff-f29db169a4bf None admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 10:42:04.875540 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 10:42:04.929757 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1fcc14e-7db2-4386-85a5-6ffdfe3a8ecb None admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 10:42:04.959220 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72a25ab6-5f11-4fd0-b0ff-f29db169a4bf None admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 10:42:04.975926 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dc5ab2a-41ee-4194-9be8-2ac54b09bf35 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 10:42:04.976550 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dc5ab2a-41ee-4194-9be8-2ac54b09bf35 None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:42:04.976833 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dc5ab2a-41ee-4194-9be8-2ac54b09bf35 None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:42:04.977110 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dc5ab2a-41ee-4194-9be8-2ac54b09bf35 None admin] PATH_INFO: `/v3/users` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:42:04.978006 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dc5ab2a-41ee-4194-9be8-2ac54b09bf35 None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=80143) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 10:42:04.979064 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dc5ab2a-41ee-4194-9be8-2ac54b09bf35 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 10:42:04.980549 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dc5ab2a-41ee-4194-9be8-2ac54b09bf35 None admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 10:42:04.996538 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e09da4d-44b7-40e3-8f07-02a156cc1c53 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 10:42:04.997289 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e09da4d-44b7-40e3-8f07-02a156cc1c53 None admin] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:42:04.997702 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e09da4d-44b7-40e3-8f07-02a156cc1c53 None admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:42:04.998076 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e09da4d-44b7-40e3-8f07-02a156cc1c53 None admin] PATH_INFO: `/v3/domains` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:42:04.998825 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e09da4d-44b7-40e3-8f07-02a156cc1c53 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80144) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 10:42:04.999259 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e09da4d-44b7-40e3-8f07-02a156cc1c53 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 10:42:05.000700 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e09da4d-44b7-40e3-8f07-02a156cc1c53 None admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 10:42:05.002821 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 288/589] 10.222.0.25 () {56 vars in 1137 bytes} [Sun Aug 7 10:42:04 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 10:42:05.004157 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dc5ab2a-41ee-4194-9be8-2ac54b09bf35 None admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 10:42:05.006759 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=100 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 10:42:05.019617 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 302/590] 10.222.0.25 () {56 vars in 1109 bytes} [Sun Aug 7 10:42:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 10:42:05.020641 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e09da4d-44b7-40e3-8f07-02a156cc1c53 None admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 10:42:05.124562 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dc5ab2a-41ee-4194-9be8-2ac54b09bf35 None admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 10:42:05.155387 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=100 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 10:42:05.166356 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e09da4d-44b7-40e3-8f07-02a156cc1c53 None admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 10:42:05.179713 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b8d3b08-7755-4bb4-bfc4-b57105c809ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 10:42:05.181072 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b8d3b08-7755-4bb4-bfc4-b57105c809ee None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:42:05.182217 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b8d3b08-7755-4bb4-bfc4-b57105c809ee None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:42:05.183155 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b8d3b08-7755-4bb4-bfc4-b57105c809ee None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:42:05.201370 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b8d3b08-7755-4bb4-bfc4-b57105c809ee None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 10:42:05.203941 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b8d3b08-7755-4bb4-bfc4-b57105c809ee None admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 10:42:05.215928 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ffad88d-cc69-4158-8a86-fd6e55d26a25 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 10:42:05.217069 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ffad88d-cc69-4158-8a86-fd6e55d26a25 None admin] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:42:05.217357 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ffad88d-cc69-4158-8a86-fd6e55d26a25 None admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:42:05.217633 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ffad88d-cc69-4158-8a86-fd6e55d26a25 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:42:05.224012 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application [None req-3b8d3b08-7755-4bb4-bfc4-b57105c809ee None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 07 10:42:05.224012 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 07 10:42:05.224012 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 07 10:42:05.224012 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 07 10:42:05.224012 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 07 10:42:05.224012 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 07 10:42:05.224012 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 07 10:42:05.224012 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 07 10:42:05.224012 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 07 10:42:05.224012 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 07 10:42:05.224012 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 07 10:42:05.224012 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 07 10:42:05.224012 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 07 10:42:05.224012 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 07 10:42:05.224012 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 07 10:42:05.224012 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 07 10:42:05.224993 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 07 10:42:05.224993 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 07 10:42:05.224993 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 07 10:42:05.224993 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 07 10:42:05.224993 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 07 10:42:05.224993 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.get_or_create( Aug 07 10:42:05.224993 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 07 10:42:05.224993 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application with Lock( Aug 07 10:42:05.224993 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 07 10:42:05.224993 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self._enter() Aug 07 10:42:05.224993 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 07 10:42:05.224993 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 07 10:42:05.224993 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 07 10:42:05.224993 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.creator() Aug 07 10:42:05.224993 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 07 10:42:05.224993 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application created_value = creator( Aug 07 10:42:05.224993 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 07 10:42:05.225868 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 07 10:42:05.225868 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 07 10:42:05.225868 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application Aug 07 10:42:05.228298 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 289/591] 10.222.0.25 () {56 vars in 1108 bytes} [Sun Aug 7 10:42:05 2022] GET /identity/v3/domains/Default => generated 87 bytes in 225 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 07 10:42:05.229306 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b8d3b08-7755-4bb4-bfc4-b57105c809ee None admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 10:42:05.239797 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ffad88d-cc69-4158-8a86-fd6e55d26a25 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 10:42:05.241460 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ffad88d-cc69-4158-8a86-fd6e55d26a25 None admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 10:42:05.265607 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application [None req-9ffad88d-cc69-4158-8a86-fd6e55d26a25 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 07 10:42:05.265607 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 07 10:42:05.265607 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 07 10:42:05.265607 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 07 10:42:05.265607 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 07 10:42:05.265607 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 07 10:42:05.265607 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 07 10:42:05.265607 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 07 10:42:05.265607 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 07 10:42:05.265607 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 07 10:42:05.265607 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 07 10:42:05.265607 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 07 10:42:05.265607 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 07 10:42:05.265607 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 07 10:42:05.265607 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 07 10:42:05.265607 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 07 10:42:05.266783 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 07 10:42:05.266783 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 07 10:42:05.266783 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 07 10:42:05.266783 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 07 10:42:05.266783 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 07 10:42:05.266783 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self.get_or_create( Aug 07 10:42:05.266783 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 07 10:42:05.266783 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application with Lock( Aug 07 10:42:05.266783 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 07 10:42:05.266783 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self._enter() Aug 07 10:42:05.266783 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 07 10:42:05.266783 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 07 10:42:05.266783 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 07 10:42:05.266783 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self.creator() Aug 07 10:42:05.266783 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 07 10:42:05.266783 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application created_value = creator( Aug 07 10:42:05.266783 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 07 10:42:05.267733 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 07 10:42:05.267733 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 07 10:42:05.267733 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application Aug 07 10:42:05.273126 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 303/592] 10.222.0.25 () {56 vars in 1108 bytes} [Sun Aug 7 10:42:05 2022] GET /identity/v3/domains/Default => generated 87 bytes in 248 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 07 10:42:05.278490 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ffad88d-cc69-4158-8a86-fd6e55d26a25 None admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 10:42:05.303358 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=100 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 10:42:05.350202 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b8d3b08-7755-4bb4-bfc4-b57105c809ee None admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 10:42:05.391687 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc9758cb-7101-4753-beb5-19b52cd179bd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 10:42:05.392762 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc9758cb-7101-4753-beb5-19b52cd179bd None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:42:05.393249 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc9758cb-7101-4753-beb5-19b52cd179bd None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:42:05.393750 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc9758cb-7101-4753-beb5-19b52cd179bd None admin] PATH_INFO: `/v3/users/neutron` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:42:05.403169 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc9758cb-7101-4753-beb5-19b52cd179bd None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 10:42:05.405140 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc9758cb-7101-4753-beb5-19b52cd179bd None admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 10:42:05.408801 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ffad88d-cc69-4158-8a86-fd6e55d26a25 None admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 10:42:05.419244 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application [None req-bc9758cb-7101-4753-beb5-19b52cd179bd None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Aug 07 10:42:05.419244 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 07 10:42:05.419244 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 07 10:42:05.419244 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 07 10:42:05.419244 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 07 10:42:05.419244 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 07 10:42:05.419244 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 07 10:42:05.419244 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 07 10:42:05.419244 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 07 10:42:05.419244 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 07 10:42:05.419244 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 07 10:42:05.419244 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 07 10:42:05.419244 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 07 10:42:05.419244 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 07 10:42:05.419244 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 07 10:42:05.419244 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 07 10:42:05.420247 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 07 10:42:05.420247 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 07 10:42:05.420247 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 07 10:42:05.420247 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 07 10:42:05.420247 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 07 10:42:05.420247 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 07 10:42:05.420247 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 07 10:42:05.420247 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 07 10:42:05.420247 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 07 10:42:05.420247 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.get_or_create( Aug 07 10:42:05.420247 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 07 10:42:05.420247 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application with Lock( Aug 07 10:42:05.420247 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 07 10:42:05.420247 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self._enter() Aug 07 10:42:05.420247 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 07 10:42:05.420247 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 07 10:42:05.420247 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 07 10:42:05.420247 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.creator() Aug 07 10:42:05.421191 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 07 10:42:05.421191 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application created_value = creator( Aug 07 10:42:05.421191 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 07 10:42:05.421191 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 07 10:42:05.421191 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 07 10:42:05.421191 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 07 10:42:05.421191 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 07 10:42:05.421191 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 07 10:42:05.421191 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Aug 07 10:42:05.421191 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application Aug 07 10:42:05.423856 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 290/593] 10.222.0.25 () {56 vars in 1102 bytes} [Sun Aug 7 10:42:05 2022] GET /identity/v3/users/neutron => generated 85 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 07 10:42:05.424998 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc9758cb-7101-4753-beb5-19b52cd179bd None admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 10:42:05.456957 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15ee4f2f-6193-4e65-a7fb-caf957bb58a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 10:42:05.458109 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15ee4f2f-6193-4e65-a7fb-caf957bb58a6 None admin] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:42:05.458411 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15ee4f2f-6193-4e65-a7fb-caf957bb58a6 None admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:42:05.458731 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15ee4f2f-6193-4e65-a7fb-caf957bb58a6 None admin] PATH_INFO: `/v3/domains` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:42:05.459617 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15ee4f2f-6193-4e65-a7fb-caf957bb58a6 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80144) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 10:42:05.460039 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15ee4f2f-6193-4e65-a7fb-caf957bb58a6 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 10:42:05.462307 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15ee4f2f-6193-4e65-a7fb-caf957bb58a6 None admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 10:42:05.482586 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 304/594] 10.222.0.25 () {56 vars in 1109 bytes} [Sun Aug 7 10:42:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 10:42:05.483556 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15ee4f2f-6193-4e65-a7fb-caf957bb58a6 None admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 10:42:05.496995 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=100 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 10:42:05.550692 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc9758cb-7101-4753-beb5-19b52cd179bd None admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 10:42:05.569217 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=100 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 10:42:05.594971 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9ae2765-cfb4-417f-a0dc-257ca13df14b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 10:42:05.595600 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9ae2765-cfb4-417f-a0dc-257ca13df14b None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:42:05.595879 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9ae2765-cfb4-417f-a0dc-257ca13df14b None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:42:05.596151 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9ae2765-cfb4-417f-a0dc-257ca13df14b None admin] PATH_INFO: `/v3/domains` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:42:05.596802 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9ae2765-cfb4-417f-a0dc-257ca13df14b None admin] RBAC: Adding query filter params (name=Default) {{(pid=80143) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 10:42:05.597103 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9ae2765-cfb4-417f-a0dc-257ca13df14b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 10:42:05.598529 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9ae2765-cfb4-417f-a0dc-257ca13df14b None admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 10:42:05.613442 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 291/595] 10.222.0.25 () {56 vars in 1109 bytes} [Sun Aug 7 10:42:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 10:42:05.614351 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9ae2765-cfb4-417f-a0dc-257ca13df14b None admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 10:42:05.622002 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15ee4f2f-6193-4e65-a7fb-caf957bb58a6 None admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 10:42:05.663297 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0816170-f224-47ba-a823-f1b1a9cabb83 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 10:42:05.664361 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0816170-f224-47ba-a823-f1b1a9cabb83 None admin] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:42:05.664897 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0816170-f224-47ba-a823-f1b1a9cabb83 None admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:42:05.665372 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0816170-f224-47ba-a823-f1b1a9cabb83 None admin] PATH_INFO: `/v3/users` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:42:05.666448 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0816170-f224-47ba-a823-f1b1a9cabb83 None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=80144) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 10:42:05.667096 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0816170-f224-47ba-a823-f1b1a9cabb83 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 10:42:05.668782 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0816170-f224-47ba-a823-f1b1a9cabb83 None admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 10:42:05.689258 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 305/596] 10.222.0.25 () {56 vars in 1139 bytes} [Sun Aug 7 10:42:05 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 10:42:05.690491 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0816170-f224-47ba-a823-f1b1a9cabb83 None admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 10:42:05.731275 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9ae2765-cfb4-417f-a0dc-257ca13df14b None admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 10:42:05.770089 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=100 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 10:42:05.771094 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49a35aca-9430-4711-a048-e5ef0d67964e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 10:42:05.771899 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49a35aca-9430-4711-a048-e5ef0d67964e None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:42:05.772217 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49a35aca-9430-4711-a048-e5ef0d67964e None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:42:05.773024 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49a35aca-9430-4711-a048-e5ef0d67964e None admin] PATH_INFO: `/v3/projects/service` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:42:05.781295 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49a35aca-9430-4711-a048-e5ef0d67964e None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 10:42:05.782742 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49a35aca-9430-4711-a048-e5ef0d67964e None admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 10:42:05.796548 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application [None req-49a35aca-9430-4711-a048-e5ef0d67964e None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 07 10:42:05.796548 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 07 10:42:05.796548 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 07 10:42:05.796548 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 07 10:42:05.796548 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 07 10:42:05.796548 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 07 10:42:05.796548 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 07 10:42:05.796548 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 07 10:42:05.796548 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 07 10:42:05.796548 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 07 10:42:05.796548 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 07 10:42:05.796548 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 07 10:42:05.796548 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 07 10:42:05.796548 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 07 10:42:05.796548 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 07 10:42:05.797488 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 07 10:42:05.797488 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 07 10:42:05.797488 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 07 10:42:05.797488 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 07 10:42:05.797488 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 07 10:42:05.797488 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 07 10:42:05.797488 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.get_or_create( Aug 07 10:42:05.797488 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 07 10:42:05.797488 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application with Lock( Aug 07 10:42:05.797488 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 07 10:42:05.797488 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self._enter() Aug 07 10:42:05.797488 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 07 10:42:05.797488 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 07 10:42:05.797488 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 07 10:42:05.797488 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.creator() Aug 07 10:42:05.797488 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 07 10:42:05.797488 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application created_value = creator( Aug 07 10:42:05.798357 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 07 10:42:05.798357 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 07 10:42:05.798357 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 07 10:42:05.798357 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 07 10:42:05.798357 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 07 10:42:05.798357 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 07 10:42:05.798357 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 07 10:42:05.798357 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application Aug 07 10:42:05.798781 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 292/597] 10.222.0.25 () {56 vars in 1111 bytes} [Sun Aug 7 10:42:05 2022] GET /identity/v3/projects/service => generated 88 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 07 10:42:05.800130 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49a35aca-9430-4711-a048-e5ef0d67964e None admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 10:42:05.816624 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0816170-f224-47ba-a823-f1b1a9cabb83 None admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 10:42:05.838145 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=100 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 10:42:05.858128 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90845cf0-a738-4f59-a42c-04a80ff55027 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 10:42:05.859259 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90845cf0-a738-4f59-a42c-04a80ff55027 None admin] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:42:05.859797 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90845cf0-a738-4f59-a42c-04a80ff55027 None admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:42:05.860343 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90845cf0-a738-4f59-a42c-04a80ff55027 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:42:05.870332 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90845cf0-a738-4f59-a42c-04a80ff55027 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 10:42:05.872211 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90845cf0-a738-4f59-a42c-04a80ff55027 None admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 10:42:05.884942 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application [None req-90845cf0-a738-4f59-a42c-04a80ff55027 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 07 10:42:05.884942 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 07 10:42:05.884942 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 07 10:42:05.884942 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 07 10:42:05.884942 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 07 10:42:05.884942 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 07 10:42:05.884942 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 07 10:42:05.884942 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 07 10:42:05.884942 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 07 10:42:05.884942 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 07 10:42:05.884942 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 07 10:42:05.884942 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 07 10:42:05.884942 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 07 10:42:05.884942 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 07 10:42:05.884942 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 07 10:42:05.885948 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 07 10:42:05.885948 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 07 10:42:05.885948 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 07 10:42:05.885948 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 07 10:42:05.885948 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 07 10:42:05.885948 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 07 10:42:05.885948 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self.get_or_create( Aug 07 10:42:05.885948 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 07 10:42:05.885948 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application with Lock( Aug 07 10:42:05.885948 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 07 10:42:05.885948 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self._enter() Aug 07 10:42:05.885948 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 07 10:42:05.885948 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 07 10:42:05.885948 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 07 10:42:05.885948 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self.creator() Aug 07 10:42:05.885948 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 07 10:42:05.885948 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application created_value = creator( Aug 07 10:42:05.886824 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 07 10:42:05.886824 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 07 10:42:05.886824 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 07 10:42:05.886824 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 07 10:42:05.886824 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 07 10:42:05.886824 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 07 10:42:05.886824 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 07 10:42:05.886824 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application Aug 07 10:42:05.889348 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 306/598] 10.222.0.25 () {56 vars in 1111 bytes} [Sun Aug 7 10:42:05 2022] GET /identity/v3/projects/service => generated 88 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 07 10:42:05.890477 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90845cf0-a738-4f59-a42c-04a80ff55027 None admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 10:42:05.913528 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49a35aca-9430-4711-a048-e5ef0d67964e None admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 10:42:05.950123 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b366ba5-9922-444b-b48a-9b0c1be9ce50 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 10:42:05.950780 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b366ba5-9922-444b-b48a-9b0c1be9ce50 None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:42:05.951111 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b366ba5-9922-444b-b48a-9b0c1be9ce50 None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:42:05.951438 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b366ba5-9922-444b-b48a-9b0c1be9ce50 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:42:05.963420 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b366ba5-9922-444b-b48a-9b0c1be9ce50 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 10:42:05.965007 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b366ba5-9922-444b-b48a-9b0c1be9ce50 None admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 10:42:05.981638 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application [None req-0b366ba5-9922-444b-b48a-9b0c1be9ce50 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 07 10:42:05.981638 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 07 10:42:05.981638 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 07 10:42:05.981638 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 07 10:42:05.981638 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 07 10:42:05.981638 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 07 10:42:05.981638 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 07 10:42:05.981638 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 07 10:42:05.981638 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 07 10:42:05.981638 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 07 10:42:05.981638 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 07 10:42:05.981638 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 07 10:42:05.981638 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 07 10:42:05.981638 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 07 10:42:05.981638 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 07 10:42:05.981638 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 07 10:42:05.982655 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 07 10:42:05.982655 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 07 10:42:05.982655 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 07 10:42:05.982655 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 07 10:42:05.982655 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 07 10:42:05.982655 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.get_or_create( Aug 07 10:42:05.982655 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 07 10:42:05.982655 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application with Lock( Aug 07 10:42:05.982655 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 07 10:42:05.982655 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self._enter() Aug 07 10:42:05.982655 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 07 10:42:05.982655 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 07 10:42:05.982655 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 07 10:42:05.982655 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.creator() Aug 07 10:42:05.982655 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 07 10:42:05.982655 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application created_value = creator( Aug 07 10:42:05.982655 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 07 10:42:05.983595 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 07 10:42:05.983595 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 07 10:42:05.983595 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application Aug 07 10:42:05.983595 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 293/599] 10.222.0.25 () {56 vars in 1108 bytes} [Sun Aug 7 10:42:05 2022] GET /identity/v3/domains/Default => generated 87 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 07 10:42:05.984357 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b366ba5-9922-444b-b48a-9b0c1be9ce50 None admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 10:42:06.009751 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90845cf0-a738-4f59-a42c-04a80ff55027 None admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 10:42:06.038942 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=100 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 10:42:06.049582 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca5b08d0-ec87-4176-aeaa-dea99f15a240 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 10:42:06.050496 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca5b08d0-ec87-4176-aeaa-dea99f15a240 None admin] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:42:06.051033 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca5b08d0-ec87-4176-aeaa-dea99f15a240 None admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:42:06.051542 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca5b08d0-ec87-4176-aeaa-dea99f15a240 None admin] PATH_INFO: `/v3/projects` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:42:06.052443 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca5b08d0-ec87-4176-aeaa-dea99f15a240 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80144) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 10:42:06.052954 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca5b08d0-ec87-4176-aeaa-dea99f15a240 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 10:42:06.054603 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca5b08d0-ec87-4176-aeaa-dea99f15a240 None admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 10:42:06.068448 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 307/600] 10.222.0.25 () {56 vars in 1148 bytes} [Sun Aug 7 10:42:05 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 10:42:06.069590 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca5b08d0-ec87-4176-aeaa-dea99f15a240 None admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 10:42:06.087319 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=100 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 10:42:06.091625 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b366ba5-9922-444b-b48a-9b0c1be9ce50 None admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 10:42:06.127408 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85969a61-6742-4551-b5b0-604b3a1683d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 10:42:06.128051 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85969a61-6742-4551-b5b0-604b3a1683d0 None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:42:06.128332 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85969a61-6742-4551-b5b0-604b3a1683d0 None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:42:06.128602 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85969a61-6742-4551-b5b0-604b3a1683d0 None admin] PATH_INFO: `/v3/projects` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:42:06.129297 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85969a61-6742-4551-b5b0-604b3a1683d0 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80143) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 10:42:06.129598 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85969a61-6742-4551-b5b0-604b3a1683d0 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 10:42:06.131141 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85969a61-6742-4551-b5b0-604b3a1683d0 None admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 10:42:06.144396 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 294/601] 10.222.0.25 () {56 vars in 1148 bytes} [Sun Aug 7 10:42:05 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 10:42:06.145351 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85969a61-6742-4551-b5b0-604b3a1683d0 None admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 10:42:06.182572 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca5b08d0-ec87-4176-aeaa-dea99f15a240 None admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 10:42:06.220200 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30c84a44-003f-43df-b9f7-e22649296d05 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 10:42:06.221159 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30c84a44-003f-43df-b9f7-e22649296d05 None admin] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:42:06.221644 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30c84a44-003f-43df-b9f7-e22649296d05 None admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:42:06.222158 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30c84a44-003f-43df-b9f7-e22649296d05 None admin] PATH_INFO: `/v3/domains` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:42:06.223048 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30c84a44-003f-43df-b9f7-e22649296d05 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80144) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 10:42:06.223557 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30c84a44-003f-43df-b9f7-e22649296d05 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 10:42:06.225279 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30c84a44-003f-43df-b9f7-e22649296d05 None admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 10:42:06.238495 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 308/602] 10.222.0.25 () {56 vars in 1109 bytes} [Sun Aug 7 10:42:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 10:42:06.239756 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30c84a44-003f-43df-b9f7-e22649296d05 None admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 10:42:06.249560 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85969a61-6742-4551-b5b0-604b3a1683d0 None admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 10:42:06.284420 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38c7f4c1-1dba-47f7-8ae1-608db49b8a25 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 10:42:06.285112 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38c7f4c1-1dba-47f7-8ae1-608db49b8a25 None admin] REQUEST_METHOD: `PUT` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:42:06.285451 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38c7f4c1-1dba-47f7-8ae1-608db49b8a25 None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:42:06.285727 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38c7f4c1-1dba-47f7-8ae1-608db49b8a25 None admin] PATH_INFO: `/v3/projects/6de1f3cac02c4bcdb05dd2337176033d/users/88dbd6dfd1164994a3e16da553561910/roles/fbf375e9760340c9aa9529333eeb25f9` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:42:06.292373 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=100 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 10:42:06.301733 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38c7f4c1-1dba-47f7-8ae1-608db49b8a25 None admin] RBAC: Authorizing `identity:create_grant(project_id=6de1f3cac02c4bcdb05dd2337176033d, user_id=88dbd6dfd1164994a3e16da553561910, role_id=fbf375e9760340c9aa9529333eeb25f9)` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 10:42:06.303300 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38c7f4c1-1dba-47f7-8ae1-608db49b8a25 None admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 10:42:06.311267 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 10:42:06.327605 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 295/603] 10.222.0.25 () {58 vars in 1439 bytes} [Sun Aug 7 10:42:06 2022] PUT /identity/v3/projects/6de1f3cac02c4bcdb05dd2337176033d/users/88dbd6dfd1164994a3e16da553561910/roles/fbf375e9760340c9aa9529333eeb25f9 => generated 0 bytes in 174 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 07 10:42:06.328682 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38c7f4c1-1dba-47f7-8ae1-608db49b8a25 None admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 10:42:06.357260 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30c84a44-003f-43df-b9f7-e22649296d05 None admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 10:42:06.396088 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20fc7d66-a610-4b5f-87a9-d3128455d13d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 10:42:06.397128 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20fc7d66-a610-4b5f-87a9-d3128455d13d None admin] REQUEST_METHOD: `PUT` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:42:06.397630 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20fc7d66-a610-4b5f-87a9-d3128455d13d None admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:42:06.398109 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20fc7d66-a610-4b5f-87a9-d3128455d13d None admin] PATH_INFO: `/v3/projects/6de1f3cac02c4bcdb05dd2337176033d/users/a11186d9d0d6488b953ede2f1628148d/roles/fbf375e9760340c9aa9529333eeb25f9` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:42:06.416471 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20fc7d66-a610-4b5f-87a9-d3128455d13d None admin] RBAC: Authorizing `identity:create_grant(project_id=6de1f3cac02c4bcdb05dd2337176033d, user_id=a11186d9d0d6488b953ede2f1628148d, role_id=fbf375e9760340c9aa9529333eeb25f9)` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 10:42:06.418333 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20fc7d66-a610-4b5f-87a9-d3128455d13d None admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 10:42:06.451497 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 309/604] 10.222.0.25 () {58 vars in 1439 bytes} [Sun Aug 7 10:42:06 2022] PUT /identity/v3/projects/6de1f3cac02c4bcdb05dd2337176033d/users/a11186d9d0d6488b953ede2f1628148d/roles/fbf375e9760340c9aa9529333eeb25f9 => generated 0 bytes in 212 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 07 10:42:06.543072 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38c7f4c1-1dba-47f7-8ae1-608db49b8a25 None admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 10:42:06.592558 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b577d38c-01c5-4fdf-8570-ea1f3057f4fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 10:42:06.593211 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b577d38c-01c5-4fdf-8570-ea1f3057f4fb None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:42:06.593486 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b577d38c-01c5-4fdf-8570-ea1f3057f4fb None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:42:06.593753 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b577d38c-01c5-4fdf-8570-ea1f3057f4fb None admin] PATH_INFO: `/v3/projects/service` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:42:06.601950 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b577d38c-01c5-4fdf-8570-ea1f3057f4fb None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 10:42:06.603500 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b577d38c-01c5-4fdf-8570-ea1f3057f4fb None admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 10:42:06.616329 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application [None req-b577d38c-01c5-4fdf-8570-ea1f3057f4fb None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 07 10:42:06.616329 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 07 10:42:06.616329 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 07 10:42:06.616329 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 07 10:42:06.616329 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 07 10:42:06.616329 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 07 10:42:06.616329 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 07 10:42:06.616329 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 07 10:42:06.616329 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 07 10:42:06.616329 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 07 10:42:06.616329 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 07 10:42:06.616329 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 07 10:42:06.616329 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 07 10:42:06.616329 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 07 10:42:06.616329 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 07 10:42:06.617299 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 07 10:42:06.617299 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 07 10:42:06.617299 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 07 10:42:06.617299 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 07 10:42:06.617299 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 07 10:42:06.617299 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 07 10:42:06.617299 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.get_or_create( Aug 07 10:42:06.617299 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 07 10:42:06.617299 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application with Lock( Aug 07 10:42:06.617299 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 07 10:42:06.617299 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self._enter() Aug 07 10:42:06.617299 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 07 10:42:06.617299 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 07 10:42:06.617299 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 07 10:42:06.617299 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.creator() Aug 07 10:42:06.617299 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 07 10:42:06.617299 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application created_value = creator( Aug 07 10:42:06.618174 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 07 10:42:06.618174 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 07 10:42:06.618174 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 07 10:42:06.618174 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 07 10:42:06.618174 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 07 10:42:06.618174 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 07 10:42:06.618174 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 07 10:42:06.618174 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application Aug 07 10:42:06.620478 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 296/605] 10.222.0.25 () {56 vars in 1111 bytes} [Sun Aug 7 10:42:06 2022] GET /identity/v3/projects/service => generated 88 bytes in 290 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 07 10:42:06.624541 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20fc7d66-a610-4b5f-87a9-d3128455d13d None admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 10:42:06.662834 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22cf1324-6f34-4ed1-a5c2-fd16154e9fc9 None None] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:42:06.663157 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22cf1324-6f34-4ed1-a5c2-fd16154e9fc9 None None] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:42:06.663439 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22cf1324-6f34-4ed1-a5c2-fd16154e9fc9 None None] PATH_INFO: `/` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:42:06.665029 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 297/606] 10.222.0.25 () {54 vars in 927 bytes} [Sun Aug 7 10:42:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 07 10:42:06.670234 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a70e4fa0-0cbd-485d-abb7-b80cc6c56d16 None None] REQUEST_METHOD: `POST` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:42:06.670518 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a70e4fa0-0cbd-485d-abb7-b80cc6c56d16 None None] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:42:06.670792 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a70e4fa0-0cbd-485d-abb7-b80cc6c56d16 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:42:06.693563 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.auth.core [None req-a70e4fa0-0cbd-485d-abb7-b80cc6c56d16 None None] MFA Rules not processed for user `bac4e6b76fe64217ac8fa44e71cffe6a`. Rule list: `[]` (Enabled: `True`). {{(pid=80143) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 10:42:06.695148 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-a70e4fa0-0cbd-485d-abb7-b80cc6c56d16 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 10:42:06.750274 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 10:42:06.759612 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20fc7d66-a610-4b5f-87a9-d3128455d13d None admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 10:42:06.765672 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=100 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 10:42:06.800830 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 298/607] 10.222.0.25 () {58 vars in 1024 bytes} [Sun Aug 7 10:42:06 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 07 10:42:06.815763 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed457f9a-82d8-4eb7-8c47-c0f73a78ab8a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 10:42:06.816374 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed457f9a-82d8-4eb7-8c47-c0f73a78ab8a None admin] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:42:06.816685 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed457f9a-82d8-4eb7-8c47-c0f73a78ab8a None admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:42:06.816921 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed457f9a-82d8-4eb7-8c47-c0f73a78ab8a None admin] PATH_INFO: `/v3/projects` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:42:06.817618 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed457f9a-82d8-4eb7-8c47-c0f73a78ab8a None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80144) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 10:42:06.817917 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed457f9a-82d8-4eb7-8c47-c0f73a78ab8a None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 10:42:06.819370 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed457f9a-82d8-4eb7-8c47-c0f73a78ab8a None admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 10:42:06.822998 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a70e4fa0-0cbd-485d-abb7-b80cc6c56d16 None None] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 10:42:06.852022 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 310/608] 10.222.0.25 () {56 vars in 1148 bytes} [Sun Aug 7 10:42:06 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 227 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 10:42:06.856105 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed457f9a-82d8-4eb7-8c47-c0f73a78ab8a None admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 10:42:06.990302 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a70e4fa0-0cbd-485d-abb7-b80cc6c56d16 None None] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 10:42:06.999422 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed457f9a-82d8-4eb7-8c47-c0f73a78ab8a None admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 10:42:07.028888 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c70ad4f-fe67-4b9b-8440-df6039528d1c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 10:42:07.030117 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c70ad4f-fe67-4b9b-8440-df6039528d1c None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:42:07.030408 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c70ad4f-fe67-4b9b-8440-df6039528d1c None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:42:07.030686 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c70ad4f-fe67-4b9b-8440-df6039528d1c None admin] PATH_INFO: `/v3/roles/service` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:42:07.037070 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cde22d3-7c55-49b8-aea2-5f0f5cc53b99 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 10:42:07.037821 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cde22d3-7c55-49b8-aea2-5f0f5cc53b99 None admin] REQUEST_METHOD: `PUT` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:42:07.038146 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cde22d3-7c55-49b8-aea2-5f0f5cc53b99 None admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:42:07.038489 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cde22d3-7c55-49b8-aea2-5f0f5cc53b99 None admin] PATH_INFO: `/v3/projects/6de1f3cac02c4bcdb05dd2337176033d/users/be1283c9bf47424c9aa3ad27cab03027/roles/fbf375e9760340c9aa9529333eeb25f9` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:42:07.047877 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c70ad4f-fe67-4b9b-8440-df6039528d1c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 07 10:42:07.054804 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application [None req-0c70ad4f-fe67-4b9b-8440-df6039528d1c None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 07 10:42:07.054804 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 07 10:42:07.054804 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 07 10:42:07.054804 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 07 10:42:07.054804 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 07 10:42:07.054804 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 07 10:42:07.054804 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 07 10:42:07.054804 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 07 10:42:07.054804 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 07 10:42:07.054804 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 07 10:42:07.054804 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 07 10:42:07.054804 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 07 10:42:07.054804 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 07 10:42:07.054804 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 07 10:42:07.054804 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 07 10:42:07.054804 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 07 10:42:07.057630 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 07 10:42:07.057630 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 07 10:42:07.057630 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 07 10:42:07.057630 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 07 10:42:07.057630 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 07 10:42:07.057630 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 07 10:42:07.057630 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 07 10:42:07.057630 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 07 10:42:07.057630 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 07 10:42:07.057630 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.get_or_create( Aug 07 10:42:07.057630 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 07 10:42:07.057630 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application with Lock( Aug 07 10:42:07.057630 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 07 10:42:07.057630 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self._enter() Aug 07 10:42:07.057630 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 07 10:42:07.057630 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 07 10:42:07.059244 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 07 10:42:07.059244 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.creator() Aug 07 10:42:07.059244 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 07 10:42:07.059244 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application created_value = creator( Aug 07 10:42:07.059244 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 07 10:42:07.059244 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 07 10:42:07.059244 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 07 10:42:07.059244 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 07 10:42:07.059244 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 07 10:42:07.059244 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 07 10:42:07.059244 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 07 10:42:07.059244 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application Aug 07 10:42:07.059879 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cde22d3-7c55-49b8-aea2-5f0f5cc53b99 None admin] RBAC: Authorizing `identity:create_grant(project_id=6de1f3cac02c4bcdb05dd2337176033d, user_id=be1283c9bf47424c9aa3ad27cab03027, role_id=fbf375e9760340c9aa9529333eeb25f9)` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 10:42:07.059954 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 299/609] 10.222.0.25 () {56 vars in 1102 bytes} [Sun Aug 7 10:42:06 2022] GET /identity/v3/roles/service => generated 85 bytes in 234 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 07 10:42:07.060032 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cde22d3-7c55-49b8-aea2-5f0f5cc53b99 None admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 10:42:07.067907 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c70ad4f-fe67-4b9b-8440-df6039528d1c None admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 10:42:07.076654 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 311/610] 10.222.0.25 () {58 vars in 1439 bytes} [Sun Aug 7 10:42:06 2022] PUT /identity/v3/projects/6de1f3cac02c4bcdb05dd2337176033d/users/be1283c9bf47424c9aa3ad27cab03027/roles/fbf375e9760340c9aa9529333eeb25f9 => generated 0 bytes in 221 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 07 10:42:07.099066 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4445ece-7af8-4589-8f8e-928508551c65 None None] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:42:07.099651 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4445ece-7af8-4589-8f8e-928508551c65 None None] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:42:07.100027 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4445ece-7af8-4589-8f8e-928508551c65 None None] PATH_INFO: `/` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:42:07.102007 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 312/611] 10.222.0.25 () {54 vars in 927 bytes} [Sun Aug 7 10:42:07 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 07 10:42:07.110155 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16a9d258-d7e8-4140-887d-59f2a06c01c9 None None] REQUEST_METHOD: `POST` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:42:07.110479 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16a9d258-d7e8-4140-887d-59f2a06c01c9 None None] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:42:07.110704 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16a9d258-d7e8-4140-887d-59f2a06c01c9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:42:07.125872 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=100 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 10:42:07.128661 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 10:42:07.133325 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.auth.core [None req-16a9d258-d7e8-4140-887d-59f2a06c01c9 None None] MFA Rules not processed for user `bac4e6b76fe64217ac8fa44e71cffe6a`. Rule list: `[]` (Enabled: `True`). {{(pid=80144) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 10:42:07.139790 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.fernet_utils [None req-16a9d258-d7e8-4140-887d-59f2a06c01c9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80144) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 10:42:07.249268 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 313/612] 10.222.0.25 () {58 vars in 1024 bytes} [Sun Aug 7 10:42:07 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 141 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 07 10:42:07.257762 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c70ad4f-fe67-4b9b-8440-df6039528d1c None admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 10:42:07.273937 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16a9d258-d7e8-4140-887d-59f2a06c01c9 None None] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 10:42:07.313024 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-695903e5-1d30-4016-a3c0-6ff4709eee77 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 10:42:07.313214 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-695903e5-1d30-4016-a3c0-6ff4709eee77 None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:42:07.313443 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-695903e5-1d30-4016-a3c0-6ff4709eee77 None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:42:07.313716 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-695903e5-1d30-4016-a3c0-6ff4709eee77 None admin] PATH_INFO: `/v3/roles` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:42:07.314380 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-695903e5-1d30-4016-a3c0-6ff4709eee77 None admin] RBAC: Adding query filter params (name=service) {{(pid=80143) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 10:42:07.314679 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-695903e5-1d30-4016-a3c0-6ff4709eee77 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 10:42:07.316108 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-695903e5-1d30-4016-a3c0-6ff4709eee77 None admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 10:42:07.325059 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 300/613] 10.222.0.25 () {56 vars in 1103 bytes} [Sun Aug 7 10:42:07 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 257 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 10:42:07.330053 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-695903e5-1d30-4016-a3c0-6ff4709eee77 None admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 10:42:07.385566 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16a9d258-d7e8-4140-887d-59f2a06c01c9 None None] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 10:42:07.422985 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3a1b88d-0358-4cb9-9f00-6356352ce008 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 10:42:07.423861 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3a1b88d-0358-4cb9-9f00-6356352ce008 None admin] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:42:07.424316 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3a1b88d-0358-4cb9-9f00-6356352ce008 None admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:42:07.424804 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3a1b88d-0358-4cb9-9f00-6356352ce008 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:42:07.427851 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=100 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 10:42:07.438958 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3a1b88d-0358-4cb9-9f00-6356352ce008 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 07 10:42:07.443946 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application [None req-f3a1b88d-0358-4cb9-9f00-6356352ce008 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 07 10:42:07.443946 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 07 10:42:07.443946 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 07 10:42:07.443946 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 07 10:42:07.443946 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 07 10:42:07.443946 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 07 10:42:07.443946 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 07 10:42:07.443946 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 07 10:42:07.443946 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 07 10:42:07.443946 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 07 10:42:07.443946 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 07 10:42:07.443946 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 07 10:42:07.443946 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 07 10:42:07.443946 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 07 10:42:07.443946 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 07 10:42:07.443946 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 07 10:42:07.444849 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 07 10:42:07.444849 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 07 10:42:07.444849 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 07 10:42:07.444849 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 07 10:42:07.444849 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 07 10:42:07.444849 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 07 10:42:07.444849 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 07 10:42:07.444849 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 07 10:42:07.444849 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 07 10:42:07.444849 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self.get_or_create( Aug 07 10:42:07.444849 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 07 10:42:07.444849 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application with Lock( Aug 07 10:42:07.444849 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 07 10:42:07.444849 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self._enter() Aug 07 10:42:07.444849 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 07 10:42:07.444849 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 07 10:42:07.445717 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 07 10:42:07.445717 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self.creator() Aug 07 10:42:07.445717 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 07 10:42:07.445717 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application created_value = creator( Aug 07 10:42:07.445717 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 07 10:42:07.445717 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 07 10:42:07.445717 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 07 10:42:07.445717 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 07 10:42:07.445717 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 07 10:42:07.445717 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 07 10:42:07.445717 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 07 10:42:07.445717 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application Aug 07 10:42:07.447945 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 314/614] 10.222.0.25 () {56 vars in 1102 bytes} [Sun Aug 7 10:42:07 2022] GET /identity/v3/roles/service => generated 85 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 07 10:42:07.450985 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-695903e5-1d30-4016-a3c0-6ff4709eee77 None admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 10:42:07.453968 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3a1b88d-0358-4cb9-9f00-6356352ce008 None admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 10:42:07.460407 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=100 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 10:42:07.508607 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4b89c79-b2b3-481a-b899-f63bfbb1e90f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 10:42:07.509551 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4b89c79-b2b3-481a-b899-f63bfbb1e90f None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:42:07.510363 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4b89c79-b2b3-481a-b899-f63bfbb1e90f None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:42:07.510729 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4b89c79-b2b3-481a-b899-f63bfbb1e90f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:42:07.535705 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4b89c79-b2b3-481a-b899-f63bfbb1e90f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 10:42:07.537285 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4b89c79-b2b3-481a-b899-f63bfbb1e90f None admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 10:42:07.558144 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application [None req-a4b89c79-b2b3-481a-b899-f63bfbb1e90f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 07 10:42:07.558144 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 07 10:42:07.558144 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 07 10:42:07.558144 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 07 10:42:07.558144 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 07 10:42:07.558144 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 07 10:42:07.558144 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 07 10:42:07.558144 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 07 10:42:07.558144 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 07 10:42:07.558144 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 07 10:42:07.558144 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 07 10:42:07.558144 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 07 10:42:07.558144 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 07 10:42:07.558144 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 07 10:42:07.558144 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 07 10:42:07.558144 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 07 10:42:07.559259 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 07 10:42:07.559259 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 07 10:42:07.559259 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 07 10:42:07.559259 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 07 10:42:07.559259 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 07 10:42:07.559259 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.get_or_create( Aug 07 10:42:07.559259 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 07 10:42:07.559259 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application with Lock( Aug 07 10:42:07.559259 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 07 10:42:07.559259 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self._enter() Aug 07 10:42:07.559259 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 07 10:42:07.559259 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 07 10:42:07.559259 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 07 10:42:07.559259 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.creator() Aug 07 10:42:07.559259 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 07 10:42:07.559259 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application created_value = creator( Aug 07 10:42:07.559259 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 07 10:42:07.560163 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 07 10:42:07.560163 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 07 10:42:07.560163 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application Aug 07 10:42:07.564468 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 301/615] 10.222.0.25 () {56 vars in 1108 bytes} [Sun Aug 7 10:42:07 2022] GET /identity/v3/domains/Default => generated 87 bytes in 230 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 07 10:42:07.567221 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3a1b88d-0358-4cb9-9f00-6356352ce008 None admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 10:42:07.567831 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4b89c79-b2b3-481a-b899-f63bfbb1e90f None admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 10:42:07.604272 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95b03abf-95b1-4795-a741-91e3eff50d43 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 10:42:07.605244 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95b03abf-95b1-4795-a741-91e3eff50d43 None admin] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:42:07.605736 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95b03abf-95b1-4795-a741-91e3eff50d43 None admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:42:07.606234 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95b03abf-95b1-4795-a741-91e3eff50d43 None admin] PATH_INFO: `/v3/roles` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:42:07.607154 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95b03abf-95b1-4795-a741-91e3eff50d43 None admin] RBAC: Adding query filter params (name=service) {{(pid=80144) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 10:42:07.607661 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95b03abf-95b1-4795-a741-91e3eff50d43 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 10:42:07.609291 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95b03abf-95b1-4795-a741-91e3eff50d43 None admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 10:42:07.618716 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 315/616] 10.222.0.25 () {56 vars in 1103 bytes} [Sun Aug 7 10:42:07 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 07 10:42:07.625130 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95b03abf-95b1-4795-a741-91e3eff50d43 None admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 10:42:07.692202 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=100 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 10:42:07.744797 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95b03abf-95b1-4795-a741-91e3eff50d43 None admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 10:42:07.787766 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4b89c79-b2b3-481a-b899-f63bfbb1e90f None admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 10:42:07.788480 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e98f8b1-609a-4f35-8dac-5d38ce221dff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 10:42:07.789597 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e98f8b1-609a-4f35-8dac-5d38ce221dff None admin] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:42:07.791439 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e98f8b1-609a-4f35-8dac-5d38ce221dff None admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:42:07.791937 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e98f8b1-609a-4f35-8dac-5d38ce221dff None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:42:07.811190 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e98f8b1-609a-4f35-8dac-5d38ce221dff None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 10:42:07.813453 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e98f8b1-609a-4f35-8dac-5d38ce221dff None admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 10:42:07.840355 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application [None req-7e98f8b1-609a-4f35-8dac-5d38ce221dff None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 07 10:42:07.840355 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 07 10:42:07.840355 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 07 10:42:07.840355 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 07 10:42:07.840355 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 07 10:42:07.840355 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 07 10:42:07.840355 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 07 10:42:07.840355 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 07 10:42:07.840355 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 07 10:42:07.840355 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 07 10:42:07.840355 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 07 10:42:07.840355 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 07 10:42:07.840355 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 07 10:42:07.840355 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 07 10:42:07.840355 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 07 10:42:07.840355 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 07 10:42:07.841318 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 07 10:42:07.841318 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 07 10:42:07.841318 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 07 10:42:07.841318 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 07 10:42:07.841318 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 07 10:42:07.841318 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self.get_or_create( Aug 07 10:42:07.841318 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 07 10:42:07.841318 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application with Lock( Aug 07 10:42:07.841318 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 07 10:42:07.841318 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self._enter() Aug 07 10:42:07.841318 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 07 10:42:07.841318 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 07 10:42:07.841318 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 07 10:42:07.841318 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self.creator() Aug 07 10:42:07.841318 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 07 10:42:07.841318 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application created_value = creator( Aug 07 10:42:07.841318 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 07 10:42:07.842191 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 07 10:42:07.842191 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 07 10:42:07.842191 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application Aug 07 10:42:07.842191 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 316/617] 10.222.0.25 () {56 vars in 1108 bytes} [Sun Aug 7 10:42:07 2022] GET /identity/v3/domains/Default => generated 87 bytes in 217 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 07 10:42:07.850867 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e98f8b1-609a-4f35-8dac-5d38ce221dff None admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 10:42:07.856256 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-847cb1b0-cfff-46c0-8372-1fbdc0abf468 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 10:42:07.856871 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-847cb1b0-cfff-46c0-8372-1fbdc0abf468 None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:42:07.857147 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-847cb1b0-cfff-46c0-8372-1fbdc0abf468 None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:42:07.857420 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-847cb1b0-cfff-46c0-8372-1fbdc0abf468 None admin] PATH_INFO: `/v3/domains` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:42:07.858066 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-847cb1b0-cfff-46c0-8372-1fbdc0abf468 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80143) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 10:42:07.858366 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-847cb1b0-cfff-46c0-8372-1fbdc0abf468 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 10:42:07.859757 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-847cb1b0-cfff-46c0-8372-1fbdc0abf468 None admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 10:42:07.881023 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 302/618] 10.222.0.25 () {56 vars in 1109 bytes} [Sun Aug 7 10:42:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 313 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 10:42:07.889739 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-847cb1b0-cfff-46c0-8372-1fbdc0abf468 None admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 10:42:07.915622 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=100 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 10:42:08.028549 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e98f8b1-609a-4f35-8dac-5d38ce221dff None admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 10:42:08.043948 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=100 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 10:42:08.062541 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-847cb1b0-cfff-46c0-8372-1fbdc0abf468 None admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 10:42:08.091541 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a50071f-3309-4f4a-9149-c87732a58710 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 10:42:08.092078 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a50071f-3309-4f4a-9149-c87732a58710 None admin] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:42:08.092358 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a50071f-3309-4f4a-9149-c87732a58710 None admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:42:08.092630 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a50071f-3309-4f4a-9149-c87732a58710 None admin] PATH_INFO: `/v3/domains` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:42:08.093269 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a50071f-3309-4f4a-9149-c87732a58710 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80144) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 10:42:08.093573 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a50071f-3309-4f4a-9149-c87732a58710 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 10:42:08.094933 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a50071f-3309-4f4a-9149-c87732a58710 None admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 10:42:08.113746 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 317/619] 10.222.0.25 () {56 vars in 1109 bytes} [Sun Aug 7 10:42:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 263 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 10:42:08.121206 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a50071f-3309-4f4a-9149-c87732a58710 None admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 10:42:08.133911 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-215a51b2-c7cb-431b-8120-c8b8e8fac647 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 10:42:08.134544 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-215a51b2-c7cb-431b-8120-c8b8e8fac647 None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:42:08.135389 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-215a51b2-c7cb-431b-8120-c8b8e8fac647 None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:42:08.135665 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-215a51b2-c7cb-431b-8120-c8b8e8fac647 None admin] PATH_INFO: `/v3/users/swift` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:42:08.145919 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-215a51b2-c7cb-431b-8120-c8b8e8fac647 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 10:42:08.147400 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-215a51b2-c7cb-431b-8120-c8b8e8fac647 None admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 10:42:08.163272 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application [None req-215a51b2-c7cb-431b-8120-c8b8e8fac647 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Aug 07 10:42:08.163272 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 07 10:42:08.163272 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 07 10:42:08.163272 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 07 10:42:08.163272 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 07 10:42:08.163272 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 07 10:42:08.163272 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 07 10:42:08.163272 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 07 10:42:08.163272 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 07 10:42:08.163272 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 07 10:42:08.163272 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 07 10:42:08.163272 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 07 10:42:08.163272 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 07 10:42:08.163272 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 07 10:42:08.163272 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 07 10:42:08.163272 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 07 10:42:08.164377 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 07 10:42:08.164377 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 07 10:42:08.164377 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 07 10:42:08.164377 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 07 10:42:08.164377 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 07 10:42:08.164377 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 07 10:42:08.164377 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 07 10:42:08.164377 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 07 10:42:08.164377 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 07 10:42:08.164377 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.get_or_create( Aug 07 10:42:08.164377 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 07 10:42:08.164377 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application with Lock( Aug 07 10:42:08.164377 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 07 10:42:08.164377 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self._enter() Aug 07 10:42:08.164377 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 07 10:42:08.164377 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 07 10:42:08.164377 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 07 10:42:08.164377 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.creator() Aug 07 10:42:08.165306 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 07 10:42:08.165306 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application created_value = creator( Aug 07 10:42:08.165306 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 07 10:42:08.165306 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 07 10:42:08.165306 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 07 10:42:08.165306 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 07 10:42:08.165306 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 07 10:42:08.165306 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 07 10:42:08.165306 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Aug 07 10:42:08.165306 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application Aug 07 10:42:08.165306 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 303/620] 10.222.0.25 () {56 vars in 1096 bytes} [Sun Aug 7 10:42:07 2022] GET /identity/v3/users/swift => generated 83 bytes in 275 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 07 10:42:08.171839 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-215a51b2-c7cb-431b-8120-c8b8e8fac647 None admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 10:42:08.305890 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=100 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 10:42:08.307403 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a50071f-3309-4f4a-9149-c87732a58710 None admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 10:42:08.348338 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5649d9b0-946e-4938-b4ce-4cf1fac8fb57 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 10:42:08.348937 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5649d9b0-946e-4938-b4ce-4cf1fac8fb57 None admin] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:42:08.349452 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5649d9b0-946e-4938-b4ce-4cf1fac8fb57 None admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:42:08.349744 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5649d9b0-946e-4938-b4ce-4cf1fac8fb57 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:42:08.357747 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-215a51b2-c7cb-431b-8120-c8b8e8fac647 None admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 10:42:08.357886 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5649d9b0-946e-4938-b4ce-4cf1fac8fb57 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 10:42:08.359778 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5649d9b0-946e-4938-b4ce-4cf1fac8fb57 None admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 10:42:08.376316 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application [None req-5649d9b0-946e-4938-b4ce-4cf1fac8fb57 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Aug 07 10:42:08.376316 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 07 10:42:08.376316 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 07 10:42:08.376316 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 07 10:42:08.376316 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 07 10:42:08.376316 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 07 10:42:08.376316 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 07 10:42:08.376316 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 07 10:42:08.376316 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 07 10:42:08.376316 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 07 10:42:08.376316 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 07 10:42:08.376316 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 07 10:42:08.376316 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 07 10:42:08.376316 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 07 10:42:08.376316 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 07 10:42:08.376316 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 07 10:42:08.377283 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 07 10:42:08.377283 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 07 10:42:08.377283 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 07 10:42:08.377283 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 07 10:42:08.377283 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 07 10:42:08.377283 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 07 10:42:08.377283 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 07 10:42:08.377283 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 07 10:42:08.377283 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 07 10:42:08.377283 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self.get_or_create( Aug 07 10:42:08.377283 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 07 10:42:08.377283 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application with Lock( Aug 07 10:42:08.377283 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 07 10:42:08.377283 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self._enter() Aug 07 10:42:08.377283 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 07 10:42:08.377283 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 07 10:42:08.377283 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 07 10:42:08.377283 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self.creator() Aug 07 10:42:08.378207 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 07 10:42:08.378207 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application created_value = creator( Aug 07 10:42:08.378207 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 07 10:42:08.378207 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 07 10:42:08.378207 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 07 10:42:08.378207 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 07 10:42:08.378207 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 07 10:42:08.378207 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 07 10:42:08.378207 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Aug 07 10:42:08.378207 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application Aug 07 10:42:08.378207 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 318/621] 10.222.0.25 () {56 vars in 1093 bytes} [Sun Aug 7 10:42:08 2022] GET /identity/v3/users/nova => generated 82 bytes in 259 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 07 10:42:08.387971 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5649d9b0-946e-4938-b4ce-4cf1fac8fb57 None admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 10:42:08.404393 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-669edc8a-94ce-44a7-982e-1174032ae474 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 10:42:08.405008 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-669edc8a-94ce-44a7-982e-1174032ae474 None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:42:08.405288 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-669edc8a-94ce-44a7-982e-1174032ae474 None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:42:08.405560 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-669edc8a-94ce-44a7-982e-1174032ae474 None admin] PATH_INFO: `/v3/users` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:42:08.406294 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-669edc8a-94ce-44a7-982e-1174032ae474 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80143) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 10:42:08.406594 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-669edc8a-94ce-44a7-982e-1174032ae474 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 10:42:08.413420 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-669edc8a-94ce-44a7-982e-1174032ae474 None admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 10:42:08.437577 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 304/622] 10.222.0.25 () {56 vars in 1135 bytes} [Sun Aug 7 10:42:08 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 266 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 10:42:08.443146 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-669edc8a-94ce-44a7-982e-1174032ae474 None admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 10:42:08.449264 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=100 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 10:42:08.576255 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5649d9b0-946e-4938-b4ce-4cf1fac8fb57 None admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 10:42:08.608909 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-669edc8a-94ce-44a7-982e-1174032ae474 None admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 10:42:08.620754 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeb0a2a3-cb27-4ce4-a14a-e786d43071f9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 10:42:08.621368 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeb0a2a3-cb27-4ce4-a14a-e786d43071f9 None admin] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:42:08.621649 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeb0a2a3-cb27-4ce4-a14a-e786d43071f9 None admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:42:08.621923 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeb0a2a3-cb27-4ce4-a14a-e786d43071f9 None admin] PATH_INFO: `/v3/users` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:42:08.622734 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eeb0a2a3-cb27-4ce4-a14a-e786d43071f9 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80144) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 10:42:08.623074 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eeb0a2a3-cb27-4ce4-a14a-e786d43071f9 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 10:42:08.624593 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eeb0a2a3-cb27-4ce4-a14a-e786d43071f9 None admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 10:42:08.644509 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb0e0338-c87e-47e8-92a2-967da265f9f3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 10:42:08.644734 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 319/623] 10.222.0.25 () {56 vars in 1133 bytes} [Sun Aug 7 10:42:08 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 257 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 10:42:08.645141 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb0e0338-c87e-47e8-92a2-967da265f9f3 None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:42:08.645421 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb0e0338-c87e-47e8-92a2-967da265f9f3 None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:42:08.645693 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb0e0338-c87e-47e8-92a2-967da265f9f3 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:42:08.655707 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=100 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 10:42:08.657101 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeb0a2a3-cb27-4ce4-a14a-e786d43071f9 None admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 10:42:08.657282 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb0e0338-c87e-47e8-92a2-967da265f9f3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 10:42:08.658891 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb0e0338-c87e-47e8-92a2-967da265f9f3 None admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 10:42:08.674938 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application [None req-eb0e0338-c87e-47e8-92a2-967da265f9f3 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 07 10:42:08.674938 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 07 10:42:08.674938 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 07 10:42:08.674938 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 07 10:42:08.674938 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 07 10:42:08.674938 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 07 10:42:08.674938 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 07 10:42:08.674938 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 07 10:42:08.674938 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 07 10:42:08.674938 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 07 10:42:08.674938 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 07 10:42:08.674938 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 07 10:42:08.674938 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 07 10:42:08.674938 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 07 10:42:08.674938 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 07 10:42:08.674938 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 07 10:42:08.675925 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 07 10:42:08.675925 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 07 10:42:08.675925 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 07 10:42:08.675925 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 07 10:42:08.675925 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 07 10:42:08.675925 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.get_or_create( Aug 07 10:42:08.675925 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 07 10:42:08.675925 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application with Lock( Aug 07 10:42:08.675925 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 07 10:42:08.675925 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self._enter() Aug 07 10:42:08.675925 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 07 10:42:08.675925 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 07 10:42:08.675925 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 07 10:42:08.675925 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.creator() Aug 07 10:42:08.675925 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 07 10:42:08.675925 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application created_value = creator( Aug 07 10:42:08.675925 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 07 10:42:08.677210 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 07 10:42:08.677210 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 07 10:42:08.677210 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application Aug 07 10:42:08.677210 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 305/624] 10.222.0.25 () {56 vars in 1108 bytes} [Sun Aug 7 10:42:08 2022] GET /identity/v3/domains/Default => generated 87 bytes in 234 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 07 10:42:08.682976 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb0e0338-c87e-47e8-92a2-967da265f9f3 None admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 10:42:08.801866 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb0e0338-c87e-47e8-92a2-967da265f9f3 None admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 10:42:08.860826 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7de5cb50-7cdf-4992-b66a-7737201ba123 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 10:42:08.861799 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7de5cb50-7cdf-4992-b66a-7737201ba123 None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:42:08.862478 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7de5cb50-7cdf-4992-b66a-7737201ba123 None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:42:08.862917 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7de5cb50-7cdf-4992-b66a-7737201ba123 None admin] PATH_INFO: `/v3/domains` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:42:08.863944 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7de5cb50-7cdf-4992-b66a-7737201ba123 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80143) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 10:42:08.864397 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7de5cb50-7cdf-4992-b66a-7737201ba123 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 10:42:08.866439 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7de5cb50-7cdf-4992-b66a-7737201ba123 None admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 10:42:08.881029 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeb0a2a3-cb27-4ce4-a14a-e786d43071f9 None admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 10:42:08.887908 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 306/625] 10.222.0.25 () {56 vars in 1109 bytes} [Sun Aug 7 10:42:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 07 10:42:08.890971 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c900747-f152-46a8-8e6d-e6586f6077b7 None None] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:42:08.894575 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c900747-f152-46a8-8e6d-e6586f6077b7 None None] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:42:08.894865 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c900747-f152-46a8-8e6d-e6586f6077b7 None None] PATH_INFO: `/` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:42:08.895281 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=100 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 10:42:08.900178 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 307/626] 10.222.0.25 () {54 vars in 927 bytes} [Sun Aug 7 10:42:08 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 07 10:42:08.900178 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c900747-f152-46a8-8e6d-e6586f6077b7 None None] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 10:42:08.924671 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61cb4ccf-8b6f-4ce1-b606-f113277a99bb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 10:42:08.925744 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61cb4ccf-8b6f-4ce1-b606-f113277a99bb None admin] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:42:08.926291 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61cb4ccf-8b6f-4ce1-b606-f113277a99bb None admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:42:08.926780 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61cb4ccf-8b6f-4ce1-b606-f113277a99bb None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:42:08.953237 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=100 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 10:42:09.002894 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61cb4ccf-8b6f-4ce1-b606-f113277a99bb None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 10:42:09.004375 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61cb4ccf-8b6f-4ce1-b606-f113277a99bb None admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 10:42:09.007553 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c900747-f152-46a8-8e6d-e6586f6077b7 None None] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 10:42:09.042348 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dd887c9-7a5b-42c6-bd62-e808f25dd10f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 10:42:09.043012 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dd887c9-7a5b-42c6-bd62-e808f25dd10f None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:42:09.043296 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dd887c9-7a5b-42c6-bd62-e808f25dd10f None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:42:09.043571 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dd887c9-7a5b-42c6-bd62-e808f25dd10f None admin] PATH_INFO: `/v3/projects/service` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:42:09.050265 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dd887c9-7a5b-42c6-bd62-e808f25dd10f None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 10:42:09.051741 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dd887c9-7a5b-42c6-bd62-e808f25dd10f None admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 10:42:09.063738 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application [None req-7dd887c9-7a5b-42c6-bd62-e808f25dd10f None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 07 10:42:09.063738 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 07 10:42:09.063738 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 07 10:42:09.063738 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 07 10:42:09.063738 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 07 10:42:09.063738 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 07 10:42:09.063738 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 07 10:42:09.063738 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 07 10:42:09.063738 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 07 10:42:09.063738 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 07 10:42:09.063738 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 07 10:42:09.063738 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 07 10:42:09.063738 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 07 10:42:09.063738 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 07 10:42:09.063738 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 07 10:42:09.065057 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 07 10:42:09.065057 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 07 10:42:09.065057 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 07 10:42:09.065057 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 07 10:42:09.065057 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 07 10:42:09.065057 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 07 10:42:09.065057 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.get_or_create( Aug 07 10:42:09.065057 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 07 10:42:09.065057 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application with Lock( Aug 07 10:42:09.065057 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 07 10:42:09.065057 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self._enter() Aug 07 10:42:09.065057 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 07 10:42:09.065057 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 07 10:42:09.065057 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 07 10:42:09.065057 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.creator() Aug 07 10:42:09.065057 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 07 10:42:09.065057 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application created_value = creator( Aug 07 10:42:09.066025 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 07 10:42:09.066025 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 07 10:42:09.066025 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 07 10:42:09.066025 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 07 10:42:09.066025 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 07 10:42:09.066025 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 07 10:42:09.066025 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 07 10:42:09.066025 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application Aug 07 10:42:09.069633 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 308/627] 10.222.0.25 () {56 vars in 1111 bytes} [Sun Aug 7 10:42:08 2022] GET /identity/v3/projects/service => generated 88 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 07 10:42:09.070859 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89e0ecff-7c4c-439d-939a-798c7cf4a4ba None None] REQUEST_METHOD: `POST` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:42:09.071167 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89e0ecff-7c4c-439d-939a-798c7cf4a4ba None None] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:42:09.071446 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89e0ecff-7c4c-439d-939a-798c7cf4a4ba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:42:09.091919 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.auth.core [None req-89e0ecff-7c4c-439d-939a-798c7cf4a4ba None None] MFA Rules not processed for user `bac4e6b76fe64217ac8fa44e71cffe6a`. Rule list: `[]` (Enabled: `True`). {{(pid=80143) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 10:42:09.093113 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-89e0ecff-7c4c-439d-939a-798c7cf4a4ba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 10:42:09.097167 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=100 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 10:42:09.120711 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 320/628] 10.222.0.25 () {58 vars in 1282 bytes} [Sun Aug 7 10:42:08 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 457 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 07 10:42:09.120711 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c8f1f3c-3435-4654-ae85-6ec53f20e543 None None] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:42:09.120711 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c8f1f3c-3435-4654-ae85-6ec53f20e543 None None] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:42:09.120711 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c8f1f3c-3435-4654-ae85-6ec53f20e543 None None] PATH_INFO: `/` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:42:09.120711 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 321/629] 10.222.0.25 () {54 vars in 927 bytes} [Sun Aug 7 10:42:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 07 10:42:09.120711 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c8f1f3c-3435-4654-ae85-6ec53f20e543 None None] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 10:42:09.162275 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 309/630] 10.222.0.25 () {58 vars in 1024 bytes} [Sun Aug 7 10:42:09 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 07 10:42:09.162987 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89e0ecff-7c4c-439d-939a-798c7cf4a4ba None None] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 10:42:09.210200 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=100 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 10:42:09.241311 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c8f1f3c-3435-4654-ae85-6ec53f20e543 None None] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 10:42:09.268659 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89e0ecff-7c4c-439d-939a-798c7cf4a4ba None None] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 10:42:09.277450 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e389a83-ea89-4665-ab8a-c42a246a6116 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 10:42:09.278231 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e389a83-ea89-4665-ab8a-c42a246a6116 None admin] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:42:09.278515 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e389a83-ea89-4665-ab8a-c42a246a6116 None admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:42:09.278839 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e389a83-ea89-4665-ab8a-c42a246a6116 None admin] PATH_INFO: `/v3/projects` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:42:09.279727 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e389a83-ea89-4665-ab8a-c42a246a6116 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80144) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 10:42:09.280046 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e389a83-ea89-4665-ab8a-c42a246a6116 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 10:42:09.281513 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e389a83-ea89-4665-ab8a-c42a246a6116 None admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 10:42:09.294428 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 322/631] 10.222.0.25 () {56 vars in 1148 bytes} [Sun Aug 7 10:42:09 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 10:42:09.295655 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bafaf78b-79e5-4712-8383-2469bfa53268 None None] REQUEST_METHOD: `POST` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:42:09.295946 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bafaf78b-79e5-4712-8383-2469bfa53268 None None] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:42:09.296224 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bafaf78b-79e5-4712-8383-2469bfa53268 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:42:09.308821 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f93122a-12ff-4f9f-8a31-cc51ac066564 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 10:42:09.309458 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f93122a-12ff-4f9f-8a31-cc51ac066564 None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:42:09.309744 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f93122a-12ff-4f9f-8a31-cc51ac066564 None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:42:09.310022 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f93122a-12ff-4f9f-8a31-cc51ac066564 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:42:09.316967 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.auth.core [None req-bafaf78b-79e5-4712-8383-2469bfa53268 None None] MFA Rules not processed for user `bac4e6b76fe64217ac8fa44e71cffe6a`. Rule list: `[]` (Enabled: `True`). {{(pid=80144) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 10:42:09.318537 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.fernet_utils [None req-bafaf78b-79e5-4712-8383-2469bfa53268 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80144) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 10:42:09.320251 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f93122a-12ff-4f9f-8a31-cc51ac066564 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 10:42:09.321698 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f93122a-12ff-4f9f-8a31-cc51ac066564 None admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 10:42:09.336450 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application [None req-0f93122a-12ff-4f9f-8a31-cc51ac066564 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 07 10:42:09.336450 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 07 10:42:09.336450 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 07 10:42:09.336450 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 07 10:42:09.336450 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 07 10:42:09.336450 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 07 10:42:09.336450 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 07 10:42:09.336450 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 07 10:42:09.336450 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 07 10:42:09.336450 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 07 10:42:09.336450 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 07 10:42:09.336450 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 07 10:42:09.336450 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 07 10:42:09.336450 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 07 10:42:09.336450 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 07 10:42:09.336450 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 07 10:42:09.337378 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 07 10:42:09.337378 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 07 10:42:09.337378 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 07 10:42:09.337378 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 07 10:42:09.337378 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 07 10:42:09.337378 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.get_or_create( Aug 07 10:42:09.337378 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 07 10:42:09.337378 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application with Lock( Aug 07 10:42:09.337378 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 07 10:42:09.337378 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self._enter() Aug 07 10:42:09.337378 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 07 10:42:09.337378 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 07 10:42:09.337378 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 07 10:42:09.337378 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.creator() Aug 07 10:42:09.337378 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 07 10:42:09.337378 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application created_value = creator( Aug 07 10:42:09.337378 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 07 10:42:09.338358 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 07 10:42:09.338358 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 07 10:42:09.338358 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application Aug 07 10:42:09.338358 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 310/632] 10.222.0.25 () {56 vars in 1108 bytes} [Sun Aug 7 10:42:09 2022] GET /identity/v3/domains/Default => generated 87 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 07 10:42:09.339221 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f93122a-12ff-4f9f-8a31-cc51ac066564 None admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 10:42:09.367723 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=100 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 10:42:09.392902 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 323/633] 10.222.0.25 () {58 vars in 1024 bytes} [Sun Aug 7 10:42:09 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 07 10:42:09.393755 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bafaf78b-79e5-4712-8383-2469bfa53268 None None] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 10:42:09.456645 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=100 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 10:42:09.468499 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f93122a-12ff-4f9f-8a31-cc51ac066564 None admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 10:42:09.509486 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-949e28c1-3b7d-48f2-99b1-9542ed0dfe53 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 10:42:09.510659 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-949e28c1-3b7d-48f2-99b1-9542ed0dfe53 None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:42:09.510947 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-949e28c1-3b7d-48f2-99b1-9542ed0dfe53 None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:42:09.511231 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-949e28c1-3b7d-48f2-99b1-9542ed0dfe53 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:42:09.512030 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bafaf78b-79e5-4712-8383-2469bfa53268 None None] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 10:42:09.521770 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-949e28c1-3b7d-48f2-99b1-9542ed0dfe53 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 07 10:42:09.526905 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application [None req-949e28c1-3b7d-48f2-99b1-9542ed0dfe53 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 07 10:42:09.526905 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 07 10:42:09.526905 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 07 10:42:09.526905 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 07 10:42:09.526905 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 07 10:42:09.526905 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 07 10:42:09.526905 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 07 10:42:09.526905 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 07 10:42:09.526905 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 07 10:42:09.526905 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 07 10:42:09.526905 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 07 10:42:09.526905 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 07 10:42:09.526905 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 07 10:42:09.526905 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 07 10:42:09.526905 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 07 10:42:09.526905 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 07 10:42:09.527824 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 07 10:42:09.527824 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 07 10:42:09.527824 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 07 10:42:09.527824 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 07 10:42:09.527824 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 07 10:42:09.527824 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 07 10:42:09.527824 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 07 10:42:09.527824 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 07 10:42:09.527824 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 07 10:42:09.527824 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.get_or_create( Aug 07 10:42:09.527824 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 07 10:42:09.527824 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application with Lock( Aug 07 10:42:09.527824 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 07 10:42:09.527824 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self._enter() Aug 07 10:42:09.527824 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 07 10:42:09.527824 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 07 10:42:09.528642 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 07 10:42:09.528642 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.creator() Aug 07 10:42:09.528642 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 07 10:42:09.528642 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application created_value = creator( Aug 07 10:42:09.528642 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 07 10:42:09.528642 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 07 10:42:09.528642 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 07 10:42:09.528642 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 07 10:42:09.528642 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 07 10:42:09.528642 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 07 10:42:09.528642 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 07 10:42:09.528642 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application Aug 07 10:42:09.528642 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 311/634] 10.222.0.25 () {56 vars in 1102 bytes} [Sun Aug 7 10:42:09 2022] GET /identity/v3/roles/service => generated 85 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 07 10:42:09.529329 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-949e28c1-3b7d-48f2-99b1-9542ed0dfe53 None admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 10:42:09.553115 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c4d15ef-8457-4f4d-97f9-55cfa6fb66d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 10:42:09.553774 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c4d15ef-8457-4f4d-97f9-55cfa6fb66d4 None admin] REQUEST_METHOD: `PUT` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:42:09.554050 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c4d15ef-8457-4f4d-97f9-55cfa6fb66d4 None admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:42:09.554324 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c4d15ef-8457-4f4d-97f9-55cfa6fb66d4 None admin] PATH_INFO: `/v3/projects/6de1f3cac02c4bcdb05dd2337176033d/users/69490ea2a9a340bf91aca38bf5655b45/roles/fbf375e9760340c9aa9529333eeb25f9` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:42:09.568996 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c4d15ef-8457-4f4d-97f9-55cfa6fb66d4 None admin] RBAC: Authorizing `identity:create_grant(project_id=6de1f3cac02c4bcdb05dd2337176033d, user_id=69490ea2a9a340bf91aca38bf5655b45, role_id=fbf375e9760340c9aa9529333eeb25f9)` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 10:42:09.570409 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c4d15ef-8457-4f4d-97f9-55cfa6fb66d4 None admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 10:42:09.585057 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 324/635] 10.222.0.25 () {58 vars in 1439 bytes} [Sun Aug 7 10:42:09 2022] PUT /identity/v3/projects/6de1f3cac02c4bcdb05dd2337176033d/users/69490ea2a9a340bf91aca38bf5655b45/roles/fbf375e9760340c9aa9529333eeb25f9 => generated 0 bytes in 192 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 07 10:42:09.599195 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c4d15ef-8457-4f4d-97f9-55cfa6fb66d4 None admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 10:42:09.616718 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 10:42:09.640346 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-949e28c1-3b7d-48f2-99b1-9542ed0dfe53 None admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 10:42:09.680000 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bed61f06-0abe-4bf1-8e7d-241c359b9994 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 10:42:09.680660 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bed61f06-0abe-4bf1-8e7d-241c359b9994 None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:42:09.680944 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bed61f06-0abe-4bf1-8e7d-241c359b9994 None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:42:09.681335 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bed61f06-0abe-4bf1-8e7d-241c359b9994 None admin] PATH_INFO: `/v3/domains` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:42:09.682668 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bed61f06-0abe-4bf1-8e7d-241c359b9994 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80143) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 10:42:09.683107 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bed61f06-0abe-4bf1-8e7d-241c359b9994 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 10:42:09.684534 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bed61f06-0abe-4bf1-8e7d-241c359b9994 None admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 10:42:09.697070 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c4d15ef-8457-4f4d-97f9-55cfa6fb66d4 None admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 10:42:09.703001 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 312/636] 10.222.0.25 () {56 vars in 1109 bytes} [Sun Aug 7 10:42:09 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 10:42:09.703001 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bed61f06-0abe-4bf1-8e7d-241c359b9994 None admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 10:42:09.718190 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=100 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 10:42:09.747395 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5904504a-4f93-471e-989f-0a90133a76d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 10:42:09.748381 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5904504a-4f93-471e-989f-0a90133a76d4 None admin] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:42:09.748801 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5904504a-4f93-471e-989f-0a90133a76d4 None admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:42:09.749281 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5904504a-4f93-471e-989f-0a90133a76d4 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:42:09.760402 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5904504a-4f93-471e-989f-0a90133a76d4 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 07 10:42:09.765379 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application [None req-5904504a-4f93-471e-989f-0a90133a76d4 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 07 10:42:09.765379 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 07 10:42:09.765379 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 07 10:42:09.765379 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 07 10:42:09.765379 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 07 10:42:09.765379 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 07 10:42:09.765379 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 07 10:42:09.765379 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 07 10:42:09.765379 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 07 10:42:09.765379 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 07 10:42:09.765379 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 07 10:42:09.765379 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 07 10:42:09.765379 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 07 10:42:09.765379 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 07 10:42:09.765379 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 07 10:42:09.765379 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 07 10:42:09.766537 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 07 10:42:09.766537 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 07 10:42:09.766537 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 07 10:42:09.766537 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 07 10:42:09.766537 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 07 10:42:09.766537 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 07 10:42:09.766537 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 07 10:42:09.766537 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 07 10:42:09.766537 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 07 10:42:09.766537 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self.get_or_create( Aug 07 10:42:09.766537 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 07 10:42:09.766537 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application with Lock( Aug 07 10:42:09.766537 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 07 10:42:09.766537 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self._enter() Aug 07 10:42:09.766537 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 07 10:42:09.766537 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 07 10:42:09.767609 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 07 10:42:09.767609 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self.creator() Aug 07 10:42:09.767609 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 07 10:42:09.767609 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application created_value = creator( Aug 07 10:42:09.767609 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 07 10:42:09.767609 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 07 10:42:09.767609 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 07 10:42:09.767609 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 07 10:42:09.767609 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 07 10:42:09.767609 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 07 10:42:09.767609 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 07 10:42:09.767609 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application Aug 07 10:42:09.769557 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 325/637] 10.222.0.25 () {56 vars in 1102 bytes} [Sun Aug 7 10:42:09 2022] GET /identity/v3/roles/service => generated 85 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 07 10:42:09.770855 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-842c732f-5b80-4b98-9a8a-ec8c4a3c1538 None None] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:42:09.771298 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-842c732f-5b80-4b98-9a8a-ec8c4a3c1538 None None] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:42:09.771710 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-842c732f-5b80-4b98-9a8a-ec8c4a3c1538 None None] PATH_INFO: `/` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:42:09.773295 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 326/638] 10.222.0.25 () {54 vars in 927 bytes} [Sun Aug 7 10:42:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 07 10:42:09.774133 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-842c732f-5b80-4b98-9a8a-ec8c4a3c1538 None None] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 10:42:09.822279 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bed61f06-0abe-4bf1-8e7d-241c359b9994 None admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 10:42:09.860778 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-137194d9-242b-45a7-9933-def70ccfb03b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 10:42:09.861622 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-137194d9-242b-45a7-9933-def70ccfb03b None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:42:09.862158 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-137194d9-242b-45a7-9933-def70ccfb03b None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:42:09.862841 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-137194d9-242b-45a7-9933-def70ccfb03b None admin] PATH_INFO: `/v3/roles` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:42:09.863711 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-137194d9-242b-45a7-9933-def70ccfb03b None admin] RBAC: Adding query filter params (name=service) {{(pid=80143) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 10:42:09.864153 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-137194d9-242b-45a7-9933-def70ccfb03b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 10:42:09.865863 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=100 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 10:42:09.866707 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-137194d9-242b-45a7-9933-def70ccfb03b None admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 10:42:09.876485 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 313/639] 10.222.0.25 () {56 vars in 1103 bytes} [Sun Aug 7 10:42:09 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 10:42:09.877476 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-137194d9-242b-45a7-9933-def70ccfb03b None admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 10:42:09.884249 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-842c732f-5b80-4b98-9a8a-ec8c4a3c1538 None None] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 10:42:09.920009 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfacc50f-9744-4e72-b6dc-ee747240a87f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 10:42:09.920639 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfacc50f-9744-4e72-b6dc-ee747240a87f None admin] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:42:09.920942 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfacc50f-9744-4e72-b6dc-ee747240a87f None admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:42:09.921182 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfacc50f-9744-4e72-b6dc-ee747240a87f None admin] PATH_INFO: `/v3/projects/service` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:42:09.928086 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfacc50f-9744-4e72-b6dc-ee747240a87f None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 10:42:09.929559 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfacc50f-9744-4e72-b6dc-ee747240a87f None admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 10:42:09.941189 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application [None req-bfacc50f-9744-4e72-b6dc-ee747240a87f None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 07 10:42:09.941189 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 07 10:42:09.941189 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 07 10:42:09.941189 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 07 10:42:09.941189 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 07 10:42:09.941189 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 07 10:42:09.941189 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 07 10:42:09.941189 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 07 10:42:09.941189 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 07 10:42:09.941189 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 07 10:42:09.941189 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 07 10:42:09.941189 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 07 10:42:09.941189 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 07 10:42:09.941189 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 07 10:42:09.941189 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 07 10:42:09.942073 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 07 10:42:09.942073 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 07 10:42:09.942073 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 07 10:42:09.942073 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 07 10:42:09.942073 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 07 10:42:09.942073 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 07 10:42:09.942073 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self.get_or_create( Aug 07 10:42:09.942073 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 07 10:42:09.942073 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application with Lock( Aug 07 10:42:09.942073 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 07 10:42:09.942073 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self._enter() Aug 07 10:42:09.942073 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 07 10:42:09.942073 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 07 10:42:09.942073 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 07 10:42:09.942073 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self.creator() Aug 07 10:42:09.942073 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 07 10:42:09.942073 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application created_value = creator( Aug 07 10:42:09.943000 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 07 10:42:09.943000 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 07 10:42:09.943000 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 07 10:42:09.943000 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 07 10:42:09.943000 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 07 10:42:09.943000 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 07 10:42:09.943000 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 07 10:42:09.943000 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application Aug 07 10:42:09.943000 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 327/640] 10.222.0.25 () {56 vars in 1111 bytes} [Sun Aug 7 10:42:09 2022] GET /identity/v3/projects/service => generated 88 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 07 10:42:09.943477 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ffe64b2-f276-4b4f-8043-0a62c30ad8cc None None] REQUEST_METHOD: `POST` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:42:09.943997 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ffe64b2-f276-4b4f-8043-0a62c30ad8cc None None] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:42:09.943997 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ffe64b2-f276-4b4f-8043-0a62c30ad8cc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:42:09.963721 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.auth.core [None req-1ffe64b2-f276-4b4f-8043-0a62c30ad8cc None None] MFA Rules not processed for user `bac4e6b76fe64217ac8fa44e71cffe6a`. Rule list: `[]` (Enabled: `True`). {{(pid=80144) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 10:42:09.964585 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=100 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 10:42:09.964901 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.fernet_utils [None req-1ffe64b2-f276-4b4f-8043-0a62c30ad8cc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80144) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 10:42:09.993659 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-137194d9-242b-45a7-9933-def70ccfb03b None admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 10:42:10.027816 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4079d40-e3b6-4ab5-a028-3a8c2cdeab5f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 10:42:10.028424 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4079d40-e3b6-4ab5-a028-3a8c2cdeab5f None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:42:10.028704 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4079d40-e3b6-4ab5-a028-3a8c2cdeab5f None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:42:10.028975 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4079d40-e3b6-4ab5-a028-3a8c2cdeab5f None admin] PATH_INFO: `/v3/roles` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:42:10.029632 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4079d40-e3b6-4ab5-a028-3a8c2cdeab5f None admin] RBAC: Adding query filter params (name=service) {{(pid=80143) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 10:42:10.029937 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4079d40-e3b6-4ab5-a028-3a8c2cdeab5f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 10:42:10.031358 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4079d40-e3b6-4ab5-a028-3a8c2cdeab5f None admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 10:42:10.036252 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 328/641] 10.222.0.25 () {58 vars in 1024 bytes} [Sun Aug 7 10:42:09 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 07 10:42:10.037239 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ffe64b2-f276-4b4f-8043-0a62c30ad8cc None None] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 10:42:10.055320 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 314/642] 10.222.0.25 () {56 vars in 1103 bytes} [Sun Aug 7 10:42:09 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 10:42:10.056322 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4079d40-e3b6-4ab5-a028-3a8c2cdeab5f None admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 10:42:10.133406 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=100 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 10:42:10.160021 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ffe64b2-f276-4b4f-8043-0a62c30ad8cc None None] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 10:42:10.166499 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4079d40-e3b6-4ab5-a028-3a8c2cdeab5f None admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 10:42:10.200352 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7900dc25-9543-4300-b076-e696ae4d1893 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 10:42:10.200543 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0040effa-119c-4a89-83c2-efc599f7296c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 10:42:10.200990 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0040effa-119c-4a89-83c2-efc599f7296c None admin] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:42:10.201268 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0040effa-119c-4a89-83c2-efc599f7296c None admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:42:10.201534 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0040effa-119c-4a89-83c2-efc599f7296c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:42:10.201669 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7900dc25-9543-4300-b076-e696ae4d1893 None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:42:10.201869 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7900dc25-9543-4300-b076-e696ae4d1893 None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:42:10.202138 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7900dc25-9543-4300-b076-e696ae4d1893 None admin] PATH_INFO: `/v3/projects` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:42:10.202826 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7900dc25-9543-4300-b076-e696ae4d1893 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80143) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 10:42:10.203156 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7900dc25-9543-4300-b076-e696ae4d1893 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 10:42:10.204565 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7900dc25-9543-4300-b076-e696ae4d1893 None admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 10:42:10.207517 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=100 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 10:42:10.212466 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0040effa-119c-4a89-83c2-efc599f7296c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 10:42:10.214271 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0040effa-119c-4a89-83c2-efc599f7296c None admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 10:42:10.222544 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 315/643] 10.222.0.25 () {56 vars in 1148 bytes} [Sun Aug 7 10:42:10 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 10:42:10.223327 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7900dc25-9543-4300-b076-e696ae4d1893 None admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 10:42:10.229606 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application [None req-0040effa-119c-4a89-83c2-efc599f7296c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 07 10:42:10.229606 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 07 10:42:10.229606 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 07 10:42:10.229606 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 07 10:42:10.229606 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 07 10:42:10.229606 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 07 10:42:10.229606 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 07 10:42:10.229606 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 07 10:42:10.229606 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 07 10:42:10.229606 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 07 10:42:10.229606 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 07 10:42:10.229606 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 07 10:42:10.229606 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 07 10:42:10.229606 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 07 10:42:10.229606 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 07 10:42:10.229606 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 07 10:42:10.230584 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 07 10:42:10.230584 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 07 10:42:10.230584 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 07 10:42:10.230584 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 07 10:42:10.230584 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 07 10:42:10.230584 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self.get_or_create( Aug 07 10:42:10.230584 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 07 10:42:10.230584 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application with Lock( Aug 07 10:42:10.230584 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 07 10:42:10.230584 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self._enter() Aug 07 10:42:10.230584 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 07 10:42:10.230584 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 07 10:42:10.230584 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 07 10:42:10.230584 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self.creator() Aug 07 10:42:10.230584 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 07 10:42:10.230584 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application created_value = creator( Aug 07 10:42:10.230584 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 07 10:42:10.236977 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 07 10:42:10.236977 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 07 10:42:10.236977 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application Aug 07 10:42:10.236977 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 329/644] 10.222.0.25 () {56 vars in 1108 bytes} [Sun Aug 7 10:42:10 2022] GET /identity/v3/domains/Default => generated 87 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 07 10:42:10.236977 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0040effa-119c-4a89-83c2-efc599f7296c None admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 10:42:10.332860 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7900dc25-9543-4300-b076-e696ae4d1893 None admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 10:42:10.343451 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0040effa-119c-4a89-83c2-efc599f7296c None admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 10:42:10.368036 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c7bf1d0-ceea-4f5b-acfb-30c810c1465f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 10:42:10.368662 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c7bf1d0-ceea-4f5b-acfb-30c810c1465f None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:42:10.368938 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c7bf1d0-ceea-4f5b-acfb-30c810c1465f None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:42:10.369886 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c7bf1d0-ceea-4f5b-acfb-30c810c1465f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:42:10.377133 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=100 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 10:42:10.380565 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c7bf1d0-ceea-4f5b-acfb-30c810c1465f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 10:42:10.381959 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c7bf1d0-ceea-4f5b-acfb-30c810c1465f None admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 10:42:10.384958 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be00d4e9-ea65-45ac-b507-0054110fa0f7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 10:42:10.385806 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be00d4e9-ea65-45ac-b507-0054110fa0f7 None admin] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:42:10.386271 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be00d4e9-ea65-45ac-b507-0054110fa0f7 None admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:42:10.386725 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be00d4e9-ea65-45ac-b507-0054110fa0f7 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:42:10.396062 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application [None req-5c7bf1d0-ceea-4f5b-acfb-30c810c1465f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 07 10:42:10.396062 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 07 10:42:10.396062 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 07 10:42:10.396062 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 07 10:42:10.396062 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 07 10:42:10.396062 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 07 10:42:10.396062 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 07 10:42:10.396062 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 07 10:42:10.396062 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 07 10:42:10.396062 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 07 10:42:10.396062 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 07 10:42:10.396062 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 07 10:42:10.396062 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 07 10:42:10.396062 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 07 10:42:10.396062 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 07 10:42:10.396062 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 07 10:42:10.396981 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 07 10:42:10.396981 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 07 10:42:10.396981 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 07 10:42:10.396981 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 07 10:42:10.396981 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 07 10:42:10.396981 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.get_or_create( Aug 07 10:42:10.396981 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 07 10:42:10.396981 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application with Lock( Aug 07 10:42:10.396981 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 07 10:42:10.396981 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self._enter() Aug 07 10:42:10.396981 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 07 10:42:10.396981 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 07 10:42:10.396981 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 07 10:42:10.396981 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.creator() Aug 07 10:42:10.396981 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 07 10:42:10.396981 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application created_value = creator( Aug 07 10:42:10.396981 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 07 10:42:10.398029 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 07 10:42:10.398029 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 07 10:42:10.398029 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application Aug 07 10:42:10.398029 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 316/645] 10.222.0.25 () {56 vars in 1108 bytes} [Sun Aug 7 10:42:10 2022] GET /identity/v3/domains/Default => generated 87 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 07 10:42:10.398029 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c7bf1d0-ceea-4f5b-acfb-30c810c1465f None admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 10:42:10.403385 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be00d4e9-ea65-45ac-b507-0054110fa0f7 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 07 10:42:10.408695 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application [None req-be00d4e9-ea65-45ac-b507-0054110fa0f7 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 07 10:42:10.408695 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 07 10:42:10.408695 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 07 10:42:10.408695 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 07 10:42:10.408695 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 07 10:42:10.408695 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 07 10:42:10.408695 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 07 10:42:10.408695 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 07 10:42:10.408695 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 07 10:42:10.408695 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 07 10:42:10.408695 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 07 10:42:10.408695 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 07 10:42:10.408695 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 07 10:42:10.408695 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 07 10:42:10.408695 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 07 10:42:10.408695 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 07 10:42:10.409615 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 07 10:42:10.409615 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 07 10:42:10.409615 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 07 10:42:10.409615 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 07 10:42:10.409615 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 07 10:42:10.409615 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 07 10:42:10.409615 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 07 10:42:10.409615 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 07 10:42:10.409615 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 07 10:42:10.409615 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self.get_or_create( Aug 07 10:42:10.409615 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 07 10:42:10.409615 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application with Lock( Aug 07 10:42:10.409615 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 07 10:42:10.409615 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self._enter() Aug 07 10:42:10.409615 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 07 10:42:10.409615 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 07 10:42:10.410432 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 07 10:42:10.410432 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self.creator() Aug 07 10:42:10.410432 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 07 10:42:10.410432 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application created_value = creator( Aug 07 10:42:10.410432 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 07 10:42:10.410432 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 07 10:42:10.410432 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 07 10:42:10.410432 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 07 10:42:10.410432 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 07 10:42:10.410432 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 07 10:42:10.410432 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 07 10:42:10.410432 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application Aug 07 10:42:10.414647 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 330/646] 10.222.0.25 () {56 vars in 1102 bytes} [Sun Aug 7 10:42:10 2022] GET /identity/v3/roles/service => generated 85 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 07 10:42:10.418346 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be00d4e9-ea65-45ac-b507-0054110fa0f7 None admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 10:42:10.454218 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=100 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 10:42:10.507858 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c7bf1d0-ceea-4f5b-acfb-30c810c1465f None admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 10:42:10.521031 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be00d4e9-ea65-45ac-b507-0054110fa0f7 None admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 10:42:10.543160 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ca74af6-e6bd-4ce3-b732-cd9629e346d3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 10:42:10.543763 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ca74af6-e6bd-4ce3-b732-cd9629e346d3 None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:42:10.544122 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ca74af6-e6bd-4ce3-b732-cd9629e346d3 None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:42:10.544365 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ca74af6-e6bd-4ce3-b732-cd9629e346d3 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:42:10.545017 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ca74af6-e6bd-4ce3-b732-cd9629e346d3 None admin] RBAC: Adding query filter params (role.id=fbf375e9760340c9aa9529333eeb25f9, scope.project.id=6de1f3cac02c4bcdb05dd2337176033d, user.id=31d16d1b96104b4eb690398432b9829e) {{(pid=80143) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 10:42:10.545316 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ca74af6-e6bd-4ce3-b732-cd9629e346d3 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 10:42:10.546739 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ca74af6-e6bd-4ce3-b732-cd9629e346d3 None admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 10:42:10.562959 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 317/647] 10.222.0.25 () {56 vars in 1374 bytes} [Sun Aug 7 10:42:10 2022] GET /identity/v3/role_assignments?user.id=31d16d1b96104b4eb690398432b9829e&scope.project.id=6de1f3cac02c4bcdb05dd2337176033d&role.id=fbf375e9760340c9aa9529333eeb25f9 => generated 608 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 10:42:10.562959 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ca74af6-e6bd-4ce3-b732-cd9629e346d3 None admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 10:42:10.570673 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cdac3d6-185f-45e0-9a07-34b3f5dc5d0b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 10:42:10.571929 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cdac3d6-185f-45e0-9a07-34b3f5dc5d0b None admin] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:42:10.572307 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cdac3d6-185f-45e0-9a07-34b3f5dc5d0b None admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:42:10.572879 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cdac3d6-185f-45e0-9a07-34b3f5dc5d0b None admin] PATH_INFO: `/v3/domains` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:42:10.573563 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cdac3d6-185f-45e0-9a07-34b3f5dc5d0b None admin] RBAC: Adding query filter params (name=Default) {{(pid=80144) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 10:42:10.573873 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cdac3d6-185f-45e0-9a07-34b3f5dc5d0b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 10:42:10.576090 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cdac3d6-185f-45e0-9a07-34b3f5dc5d0b None admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 10:42:10.592059 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 331/648] 10.222.0.25 () {56 vars in 1109 bytes} [Sun Aug 7 10:42:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 10:42:10.593185 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cdac3d6-185f-45e0-9a07-34b3f5dc5d0b None admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 10:42:10.660007 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=100 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 10:42:10.672622 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ca74af6-e6bd-4ce3-b732-cd9629e346d3 None admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 10:42:10.686557 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=100 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 10:42:10.712909 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4deecde5-7770-44bf-9d8f-284f368bb98d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 10:42:10.713491 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4deecde5-7770-44bf-9d8f-284f368bb98d None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:42:10.713976 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4deecde5-7770-44bf-9d8f-284f368bb98d None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:42:10.714438 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4deecde5-7770-44bf-9d8f-284f368bb98d None admin] PATH_INFO: `/v3/domains` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:42:10.715324 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4deecde5-7770-44bf-9d8f-284f368bb98d None admin] RBAC: Adding query filter params (name=Default) {{(pid=80143) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 10:42:10.715639 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4deecde5-7770-44bf-9d8f-284f368bb98d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 10:42:10.718002 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4deecde5-7770-44bf-9d8f-284f368bb98d None admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 10:42:10.726416 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cdac3d6-185f-45e0-9a07-34b3f5dc5d0b None admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 10:42:10.735736 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 318/649] 10.222.0.25 () {56 vars in 1109 bytes} [Sun Aug 7 10:42:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 10:42:10.736629 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4deecde5-7770-44bf-9d8f-284f368bb98d None admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 10:42:10.770042 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35e16e13-116e-4c96-aa7c-13cf6aa67b0d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 10:42:10.770735 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35e16e13-116e-4c96-aa7c-13cf6aa67b0d None admin] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:42:10.771065 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35e16e13-116e-4c96-aa7c-13cf6aa67b0d None admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:42:10.771452 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35e16e13-116e-4c96-aa7c-13cf6aa67b0d None admin] PATH_INFO: `/v3/roles` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:42:10.772045 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35e16e13-116e-4c96-aa7c-13cf6aa67b0d None admin] RBAC: Adding query filter params (name=service) {{(pid=80144) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 10:42:10.772349 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35e16e13-116e-4c96-aa7c-13cf6aa67b0d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 10:42:10.773711 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35e16e13-116e-4c96-aa7c-13cf6aa67b0d None admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 10:42:10.782913 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 332/650] 10.222.0.25 () {56 vars in 1103 bytes} [Sun Aug 7 10:42:10 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 10:42:10.783763 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35e16e13-116e-4c96-aa7c-13cf6aa67b0d None admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 10:42:10.857725 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4deecde5-7770-44bf-9d8f-284f368bb98d None admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 10:42:10.901450 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73f2e90c-7ead-490d-851d-aa0128e24e73 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 10:42:10.902105 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73f2e90c-7ead-490d-851d-aa0128e24e73 None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:42:10.902382 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73f2e90c-7ead-490d-851d-aa0128e24e73 None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:42:10.902660 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73f2e90c-7ead-490d-851d-aa0128e24e73 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:42:10.910024 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35e16e13-116e-4c96-aa7c-13cf6aa67b0d None admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 10:42:10.910366 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73f2e90c-7ead-490d-851d-aa0128e24e73 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 10:42:10.911861 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73f2e90c-7ead-490d-851d-aa0128e24e73 None admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 10:42:10.924137 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application [None req-73f2e90c-7ead-490d-851d-aa0128e24e73 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Aug 07 10:42:10.924137 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 07 10:42:10.924137 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 07 10:42:10.924137 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 07 10:42:10.924137 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 07 10:42:10.924137 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 07 10:42:10.924137 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 07 10:42:10.924137 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 07 10:42:10.924137 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 07 10:42:10.924137 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 07 10:42:10.924137 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 07 10:42:10.924137 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 07 10:42:10.924137 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 07 10:42:10.924137 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 07 10:42:10.924137 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 07 10:42:10.924137 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 07 10:42:10.925091 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 07 10:42:10.925091 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 07 10:42:10.925091 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 07 10:42:10.925091 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 07 10:42:10.925091 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 07 10:42:10.925091 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 07 10:42:10.925091 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 07 10:42:10.925091 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 07 10:42:10.925091 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 07 10:42:10.925091 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.get_or_create( Aug 07 10:42:10.925091 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 07 10:42:10.925091 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application with Lock( Aug 07 10:42:10.925091 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 07 10:42:10.925091 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self._enter() Aug 07 10:42:10.925091 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 07 10:42:10.925091 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 07 10:42:10.925091 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 07 10:42:10.925091 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.creator() Aug 07 10:42:10.926409 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 07 10:42:10.926409 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application created_value = creator( Aug 07 10:42:10.926409 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 07 10:42:10.926409 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 07 10:42:10.926409 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 07 10:42:10.926409 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 07 10:42:10.926409 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 07 10:42:10.926409 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 07 10:42:10.926409 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Aug 07 10:42:10.926409 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application Aug 07 10:42:10.926409 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 319/651] 10.222.0.25 () {56 vars in 1099 bytes} [Sun Aug 7 10:42:10 2022] GET /identity/v3/users/cinder => generated 84 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 07 10:42:10.926409 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73f2e90c-7ead-490d-851d-aa0128e24e73 None admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 10:42:10.932959 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=100 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 10:42:10.955345 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf7cb5ed-7068-4b4a-a6a6-db6f625627ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 10:42:10.955984 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf7cb5ed-7068-4b4a-a6a6-db6f625627ce None admin] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:42:10.956285 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf7cb5ed-7068-4b4a-a6a6-db6f625627ce None admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:42:10.956541 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf7cb5ed-7068-4b4a-a6a6-db6f625627ce None admin] PATH_INFO: `/v3/users/glance` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:42:10.965280 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=100 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 10:42:10.971304 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf7cb5ed-7068-4b4a-a6a6-db6f625627ce None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 10:42:10.972743 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf7cb5ed-7068-4b4a-a6a6-db6f625627ce None admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 10:42:10.990391 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application [None req-bf7cb5ed-7068-4b4a-a6a6-db6f625627ce None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Aug 07 10:42:10.990391 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 07 10:42:10.990391 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 07 10:42:10.990391 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 07 10:42:10.990391 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 07 10:42:10.990391 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 07 10:42:10.990391 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 07 10:42:10.990391 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 07 10:42:10.990391 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 07 10:42:10.990391 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 07 10:42:10.990391 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 07 10:42:10.990391 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 07 10:42:10.990391 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 07 10:42:10.990391 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 07 10:42:10.990391 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 07 10:42:10.990391 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 07 10:42:10.991404 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 07 10:42:10.991404 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 07 10:42:10.991404 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 07 10:42:10.991404 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 07 10:42:10.991404 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 07 10:42:10.991404 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 07 10:42:10.991404 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 07 10:42:10.991404 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 07 10:42:10.991404 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 07 10:42:10.991404 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self.get_or_create( Aug 07 10:42:10.991404 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 07 10:42:10.991404 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application with Lock( Aug 07 10:42:10.991404 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 07 10:42:10.991404 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self._enter() Aug 07 10:42:10.991404 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 07 10:42:10.991404 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 07 10:42:10.991404 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 07 10:42:10.991404 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self.creator() Aug 07 10:42:10.992332 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 07 10:42:10.992332 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application created_value = creator( Aug 07 10:42:10.992332 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 07 10:42:10.992332 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 07 10:42:10.992332 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 07 10:42:10.992332 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 07 10:42:10.992332 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 07 10:42:10.992332 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 07 10:42:10.992332 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Aug 07 10:42:10.992332 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application Aug 07 10:42:10.992332 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 333/652] 10.222.0.25 () {56 vars in 1099 bytes} [Sun Aug 7 10:42:10 2022] GET /identity/v3/users/glance => generated 84 bytes in 208 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 07 10:42:10.995684 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf7cb5ed-7068-4b4a-a6a6-db6f625627ce None admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 10:42:11.048417 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73f2e90c-7ead-490d-851d-aa0128e24e73 None admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 10:42:11.094440 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-370d201f-0e89-4eae-bd20-08d486567203 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 10:42:11.095111 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-370d201f-0e89-4eae-bd20-08d486567203 None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:42:11.095801 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-370d201f-0e89-4eae-bd20-08d486567203 None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:42:11.096081 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-370d201f-0e89-4eae-bd20-08d486567203 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:42:11.110906 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-370d201f-0e89-4eae-bd20-08d486567203 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 10:42:11.113442 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-370d201f-0e89-4eae-bd20-08d486567203 None admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 10:42:11.118369 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf7cb5ed-7068-4b4a-a6a6-db6f625627ce None admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 10:42:11.132666 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application [None req-370d201f-0e89-4eae-bd20-08d486567203 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 07 10:42:11.132666 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 07 10:42:11.132666 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 07 10:42:11.132666 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 07 10:42:11.132666 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 07 10:42:11.132666 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 07 10:42:11.132666 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 07 10:42:11.132666 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 07 10:42:11.132666 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 07 10:42:11.132666 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 07 10:42:11.132666 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 07 10:42:11.132666 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 07 10:42:11.132666 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 07 10:42:11.132666 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 07 10:42:11.132666 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 07 10:42:11.132666 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 07 10:42:11.134944 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 07 10:42:11.134944 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 07 10:42:11.134944 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 07 10:42:11.134944 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 07 10:42:11.134944 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 07 10:42:11.134944 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.get_or_create( Aug 07 10:42:11.134944 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 07 10:42:11.134944 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application with Lock( Aug 07 10:42:11.134944 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 07 10:42:11.134944 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self._enter() Aug 07 10:42:11.134944 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 07 10:42:11.134944 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 07 10:42:11.134944 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 07 10:42:11.134944 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.creator() Aug 07 10:42:11.134944 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 07 10:42:11.134944 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application created_value = creator( Aug 07 10:42:11.134944 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 07 10:42:11.137315 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 07 10:42:11.137315 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 07 10:42:11.137315 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application Aug 07 10:42:11.137315 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 320/653] 10.222.0.25 () {56 vars in 1108 bytes} [Sun Aug 7 10:42:10 2022] GET /identity/v3/domains/Default => generated 87 bytes in 209 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 07 10:42:11.137315 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-370d201f-0e89-4eae-bd20-08d486567203 None admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 10:42:11.165906 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce01513e-fcbb-4de9-95fe-c74e782c8ca5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 10:42:11.167027 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce01513e-fcbb-4de9-95fe-c74e782c8ca5 None admin] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:42:11.167311 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce01513e-fcbb-4de9-95fe-c74e782c8ca5 None admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:42:11.167632 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce01513e-fcbb-4de9-95fe-c74e782c8ca5 None admin] PATH_INFO: `/v3/users` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:42:11.170668 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce01513e-fcbb-4de9-95fe-c74e782c8ca5 None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=80144) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 10:42:11.171596 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce01513e-fcbb-4de9-95fe-c74e782c8ca5 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 10:42:11.173553 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce01513e-fcbb-4de9-95fe-c74e782c8ca5 None admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 10:42:11.199222 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 334/654] 10.222.0.25 () {56 vars in 1137 bytes} [Sun Aug 7 10:42:10 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 10:42:11.200195 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce01513e-fcbb-4de9-95fe-c74e782c8ca5 None admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 10:42:11.240376 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=100 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 10:42:11.245635 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=100 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 10:42:11.261297 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-370d201f-0e89-4eae-bd20-08d486567203 None admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 10:42:11.296813 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-712cd25b-ba69-4338-8223-7c3514e7fdbe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 10:42:11.297429 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-712cd25b-ba69-4338-8223-7c3514e7fdbe None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:42:11.297715 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-712cd25b-ba69-4338-8223-7c3514e7fdbe None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:42:11.297999 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-712cd25b-ba69-4338-8223-7c3514e7fdbe None admin] PATH_INFO: `/v3/users` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:42:11.298785 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-712cd25b-ba69-4338-8223-7c3514e7fdbe None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=80143) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 10:42:11.299132 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-712cd25b-ba69-4338-8223-7c3514e7fdbe None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 10:42:11.300696 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-712cd25b-ba69-4338-8223-7c3514e7fdbe None admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 10:42:11.320605 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 321/655] 10.222.0.25 () {56 vars in 1137 bytes} [Sun Aug 7 10:42:11 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 10:42:11.321679 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-712cd25b-ba69-4338-8223-7c3514e7fdbe None admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 10:42:11.335534 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce01513e-fcbb-4de9-95fe-c74e782c8ca5 None admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 10:42:11.386197 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1898cb8c-f202-4d5c-a8cb-d001dc0dd6b6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 10:42:11.387328 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1898cb8c-f202-4d5c-a8cb-d001dc0dd6b6 None admin] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:42:11.388116 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1898cb8c-f202-4d5c-a8cb-d001dc0dd6b6 None admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:42:11.388853 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1898cb8c-f202-4d5c-a8cb-d001dc0dd6b6 None admin] PATH_INFO: `/v3/domains` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:42:11.389656 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1898cb8c-f202-4d5c-a8cb-d001dc0dd6b6 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80144) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 10:42:11.390107 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1898cb8c-f202-4d5c-a8cb-d001dc0dd6b6 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 10:42:11.392431 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1898cb8c-f202-4d5c-a8cb-d001dc0dd6b6 None admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 10:42:11.405039 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 335/656] 10.222.0.25 () {56 vars in 1109 bytes} [Sun Aug 7 10:42:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 10:42:11.406091 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1898cb8c-f202-4d5c-a8cb-d001dc0dd6b6 None admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 10:42:11.437910 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-712cd25b-ba69-4338-8223-7c3514e7fdbe None admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 10:42:11.475583 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82bc0ed1-7acb-4816-8755-7037cd91cd3a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 10:42:11.476441 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82bc0ed1-7acb-4816-8755-7037cd91cd3a None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:42:11.476897 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82bc0ed1-7acb-4816-8755-7037cd91cd3a None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:42:11.477346 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82bc0ed1-7acb-4816-8755-7037cd91cd3a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:42:11.491470 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=100 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 10:42:11.540633 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=100 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 10:42:11.551118 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1898cb8c-f202-4d5c-a8cb-d001dc0dd6b6 None admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 10:42:11.561428 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82bc0ed1-7acb-4816-8755-7037cd91cd3a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 10:42:11.563276 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82bc0ed1-7acb-4816-8755-7037cd91cd3a None admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 10:42:11.589279 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab293f75-c715-4a67-a2e4-cd86233d98fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 10:42:11.590002 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab293f75-c715-4a67-a2e4-cd86233d98fc None admin] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:42:11.590358 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab293f75-c715-4a67-a2e4-cd86233d98fc None admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:42:11.590814 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab293f75-c715-4a67-a2e4-cd86233d98fc None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:42:11.672257 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 322/657] 10.222.0.25 () {58 vars in 1282 bytes} [Sun Aug 7 10:42:11 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 351 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 07 10:42:11.673364 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82bc0ed1-7acb-4816-8755-7037cd91cd3a None admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 10:42:11.682357 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab293f75-c715-4a67-a2e4-cd86233d98fc None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 10:42:11.683809 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab293f75-c715-4a67-a2e4-cd86233d98fc None admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 10:42:11.719158 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=100 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 10:42:11.778230 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=100 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 10:42:11.782326 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82bc0ed1-7acb-4816-8755-7037cd91cd3a None admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 10:42:11.795203 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 336/658] 10.222.0.25 () {58 vars in 1282 bytes} [Sun Aug 7 10:42:11 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 384 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 07 10:42:11.795203 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cff7ad78-8aaf-41ef-9832-9c517b5106ec None None] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:42:11.795203 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cff7ad78-8aaf-41ef-9832-9c517b5106ec None None] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:42:11.795203 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cff7ad78-8aaf-41ef-9832-9c517b5106ec None None] PATH_INFO: `/` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:42:11.795203 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 337/659] 10.222.0.25 () {54 vars in 927 bytes} [Sun Aug 7 10:42:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 07 10:42:11.795203 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cff7ad78-8aaf-41ef-9832-9c517b5106ec None None] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 10:42:11.825478 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b26e765-66db-41ec-8dbd-40700f1b90ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 10:42:11.826290 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b26e765-66db-41ec-8dbd-40700f1b90ef None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:42:11.826669 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b26e765-66db-41ec-8dbd-40700f1b90ef None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:42:11.827085 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b26e765-66db-41ec-8dbd-40700f1b90ef None admin] PATH_INFO: `/v3/users/neutron` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:42:11.834285 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b26e765-66db-41ec-8dbd-40700f1b90ef None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 10:42:11.835839 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b26e765-66db-41ec-8dbd-40700f1b90ef None admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 10:42:11.847972 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application [None req-6b26e765-66db-41ec-8dbd-40700f1b90ef None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Aug 07 10:42:11.847972 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 07 10:42:11.847972 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 07 10:42:11.847972 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 07 10:42:11.847972 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 07 10:42:11.847972 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 07 10:42:11.847972 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 07 10:42:11.847972 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 07 10:42:11.847972 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 07 10:42:11.847972 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 07 10:42:11.847972 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 07 10:42:11.847972 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 07 10:42:11.847972 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 07 10:42:11.847972 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 07 10:42:11.847972 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 07 10:42:11.847972 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 07 10:42:11.848944 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 07 10:42:11.848944 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 07 10:42:11.848944 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 07 10:42:11.848944 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 07 10:42:11.848944 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 07 10:42:11.848944 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 07 10:42:11.848944 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 07 10:42:11.848944 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 07 10:42:11.848944 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 07 10:42:11.848944 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.get_or_create( Aug 07 10:42:11.848944 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 07 10:42:11.848944 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application with Lock( Aug 07 10:42:11.848944 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 07 10:42:11.848944 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self._enter() Aug 07 10:42:11.848944 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 07 10:42:11.848944 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 07 10:42:11.848944 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 07 10:42:11.848944 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.creator() Aug 07 10:42:11.849894 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 07 10:42:11.849894 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application created_value = creator( Aug 07 10:42:11.849894 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 07 10:42:11.849894 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 07 10:42:11.849894 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 07 10:42:11.849894 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 07 10:42:11.849894 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 07 10:42:11.849894 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 07 10:42:11.849894 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Aug 07 10:42:11.849894 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application Aug 07 10:42:11.849894 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 323/660] 10.222.0.25 () {56 vars in 1102 bytes} [Sun Aug 7 10:42:11 2022] GET /identity/v3/users/neutron => generated 85 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 07 10:42:11.849894 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b26e765-66db-41ec-8dbd-40700f1b90ef None admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 10:42:11.914266 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cff7ad78-8aaf-41ef-9832-9c517b5106ec None None] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 10:42:11.957299 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-557717cc-51ba-4ad8-b761-c0da89aa2b5e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 10:42:11.957956 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-557717cc-51ba-4ad8-b761-c0da89aa2b5e None admin] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:42:11.958241 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-557717cc-51ba-4ad8-b761-c0da89aa2b5e None admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:42:11.958518 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-557717cc-51ba-4ad8-b761-c0da89aa2b5e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:42:11.958827 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b26e765-66db-41ec-8dbd-40700f1b90ef None admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 10:42:11.960797 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=100 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 10:42:11.971625 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-557717cc-51ba-4ad8-b761-c0da89aa2b5e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 10:42:11.972906 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-557717cc-51ba-4ad8-b761-c0da89aa2b5e None admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 10:42:11.988531 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application [None req-557717cc-51ba-4ad8-b761-c0da89aa2b5e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 07 10:42:11.988531 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 07 10:42:11.988531 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 07 10:42:11.988531 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 07 10:42:11.988531 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 07 10:42:11.988531 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 07 10:42:11.988531 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 07 10:42:11.988531 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 07 10:42:11.988531 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 07 10:42:11.988531 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 07 10:42:11.988531 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 07 10:42:11.988531 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 07 10:42:11.988531 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 07 10:42:11.988531 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 07 10:42:11.988531 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 07 10:42:11.988531 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 07 10:42:11.989488 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 07 10:42:11.989488 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 07 10:42:11.989488 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 07 10:42:11.989488 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 07 10:42:11.989488 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 07 10:42:11.989488 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self.get_or_create( Aug 07 10:42:11.989488 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 07 10:42:11.989488 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application with Lock( Aug 07 10:42:11.989488 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 07 10:42:11.989488 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self._enter() Aug 07 10:42:11.989488 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 07 10:42:11.989488 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 07 10:42:11.989488 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 07 10:42:11.989488 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self.creator() Aug 07 10:42:11.989488 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 07 10:42:11.989488 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application created_value = creator( Aug 07 10:42:11.989488 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 07 10:42:11.990345 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 07 10:42:11.990345 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 07 10:42:11.990345 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application Aug 07 10:42:11.990345 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 338/661] 10.222.0.25 () {56 vars in 1108 bytes} [Sun Aug 7 10:42:11 2022] GET /identity/v3/domains/Default => generated 87 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 07 10:42:11.991075 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2de3540-a7fb-41c1-91b0-89a68a5a71f7 None None] REQUEST_METHOD: `POST` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:42:11.991162 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2de3540-a7fb-41c1-91b0-89a68a5a71f7 None None] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:42:11.991391 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2de3540-a7fb-41c1-91b0-89a68a5a71f7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:42:12.004892 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0274176f-68f7-4005-94ef-3b810b53fea1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 10:42:12.005528 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0274176f-68f7-4005-94ef-3b810b53fea1 None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:42:12.005806 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0274176f-68f7-4005-94ef-3b810b53fea1 None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:42:12.006082 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0274176f-68f7-4005-94ef-3b810b53fea1 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:42:12.012446 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.auth.core [None req-b2de3540-a7fb-41c1-91b0-89a68a5a71f7 None None] MFA Rules not processed for user `bac4e6b76fe64217ac8fa44e71cffe6a`. Rule list: `[]` (Enabled: `True`). {{(pid=80144) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 10:42:12.014302 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.fernet_utils [None req-b2de3540-a7fb-41c1-91b0-89a68a5a71f7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80144) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 10:42:12.017561 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0274176f-68f7-4005-94ef-3b810b53fea1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 10:42:12.019019 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0274176f-68f7-4005-94ef-3b810b53fea1 None admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 10:42:12.033253 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application [None req-0274176f-68f7-4005-94ef-3b810b53fea1 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 07 10:42:12.033253 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 07 10:42:12.033253 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 07 10:42:12.033253 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 07 10:42:12.033253 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 07 10:42:12.033253 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 07 10:42:12.033253 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 07 10:42:12.033253 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 07 10:42:12.033253 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 07 10:42:12.033253 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 07 10:42:12.033253 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 07 10:42:12.033253 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 07 10:42:12.033253 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 07 10:42:12.033253 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 07 10:42:12.033253 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 07 10:42:12.033253 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 07 10:42:12.034194 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 07 10:42:12.034194 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 07 10:42:12.034194 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 07 10:42:12.034194 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 07 10:42:12.034194 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 07 10:42:12.034194 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.get_or_create( Aug 07 10:42:12.034194 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 07 10:42:12.034194 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application with Lock( Aug 07 10:42:12.034194 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 07 10:42:12.034194 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self._enter() Aug 07 10:42:12.034194 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 07 10:42:12.034194 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 07 10:42:12.034194 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 07 10:42:12.034194 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.creator() Aug 07 10:42:12.034194 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 07 10:42:12.034194 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application created_value = creator( Aug 07 10:42:12.034194 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 07 10:42:12.035176 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 07 10:42:12.035176 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 07 10:42:12.035176 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application Aug 07 10:42:12.035176 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 324/662] 10.222.0.25 () {56 vars in 1108 bytes} [Sun Aug 7 10:42:11 2022] GET /identity/v3/domains/Default => generated 87 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 07 10:42:12.035176 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0274176f-68f7-4005-94ef-3b810b53fea1 None admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 10:42:12.063878 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=100 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 10:42:12.100001 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 339/663] 10.222.0.25 () {58 vars in 1024 bytes} [Sun Aug 7 10:42:11 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 07 10:42:12.100837 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2de3540-a7fb-41c1-91b0-89a68a5a71f7 None None] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 10:42:12.151676 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0274176f-68f7-4005-94ef-3b810b53fea1 None admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 10:42:12.186892 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8ee1b98-39c4-40b2-833c-1caaa59ba010 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 10:42:12.187522 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8ee1b98-39c4-40b2-833c-1caaa59ba010 None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:42:12.187801 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8ee1b98-39c4-40b2-833c-1caaa59ba010 None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:42:12.188074 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8ee1b98-39c4-40b2-833c-1caaa59ba010 None admin] PATH_INFO: `/v3/users` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:42:12.188799 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8ee1b98-39c4-40b2-833c-1caaa59ba010 None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=80143) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 10:42:12.189103 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8ee1b98-39c4-40b2-833c-1caaa59ba010 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 10:42:12.190529 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8ee1b98-39c4-40b2-833c-1caaa59ba010 None admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 10:42:12.213609 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 325/664] 10.222.0.25 () {56 vars in 1139 bytes} [Sun Aug 7 10:42:12 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 10:42:12.213609 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8ee1b98-39c4-40b2-833c-1caaa59ba010 None admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 10:42:12.222438 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2de3540-a7fb-41c1-91b0-89a68a5a71f7 None None] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 10:42:12.226080 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=100 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 10:42:12.260248 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2612014c-5372-4a15-b5d4-5561b1ae961d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 10:42:12.261101 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2612014c-5372-4a15-b5d4-5561b1ae961d None admin] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:42:12.261385 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2612014c-5372-4a15-b5d4-5561b1ae961d None admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:42:12.261658 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2612014c-5372-4a15-b5d4-5561b1ae961d None admin] PATH_INFO: `/v3/domains` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:42:12.262303 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2612014c-5372-4a15-b5d4-5561b1ae961d None admin] RBAC: Adding query filter params (name=Default) {{(pid=80144) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 10:42:12.262613 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2612014c-5372-4a15-b5d4-5561b1ae961d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 10:42:12.264086 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2612014c-5372-4a15-b5d4-5561b1ae961d None admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 10:42:12.276553 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 340/665] 10.222.0.25 () {56 vars in 1109 bytes} [Sun Aug 7 10:42:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 10:42:12.277321 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2612014c-5372-4a15-b5d4-5561b1ae961d None admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 10:42:12.312607 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=100 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 10:42:12.323247 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8ee1b98-39c4-40b2-833c-1caaa59ba010 None admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 10:42:12.362062 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37785cd6-b944-4060-b407-e190ce498534 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 10:42:12.362943 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37785cd6-b944-4060-b407-e190ce498534 None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:42:12.363247 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37785cd6-b944-4060-b407-e190ce498534 None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:42:12.363521 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37785cd6-b944-4060-b407-e190ce498534 None admin] PATH_INFO: `/v3/domains` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:42:12.364151 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37785cd6-b944-4060-b407-e190ce498534 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80143) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 10:42:12.364463 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37785cd6-b944-4060-b407-e190ce498534 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 10:42:12.365851 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37785cd6-b944-4060-b407-e190ce498534 None admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 10:42:12.378140 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 326/666] 10.222.0.25 () {56 vars in 1109 bytes} [Sun Aug 7 10:42:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 10:42:12.380496 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37785cd6-b944-4060-b407-e190ce498534 None admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 10:42:12.393712 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2612014c-5372-4a15-b5d4-5561b1ae961d None admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 10:42:12.433521 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a33f25d-f326-4b15-b52c-e75a17fb3ad2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 10:42:12.434240 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a33f25d-f326-4b15-b52c-e75a17fb3ad2 None admin] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:42:12.434592 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a33f25d-f326-4b15-b52c-e75a17fb3ad2 None admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:42:12.434977 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a33f25d-f326-4b15-b52c-e75a17fb3ad2 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:42:12.445936 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a33f25d-f326-4b15-b52c-e75a17fb3ad2 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 07 10:42:12.451875 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application [None req-2a33f25d-f326-4b15-b52c-e75a17fb3ad2 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 07 10:42:12.451875 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 07 10:42:12.451875 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 07 10:42:12.451875 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 07 10:42:12.451875 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 07 10:42:12.451875 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 07 10:42:12.451875 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 07 10:42:12.451875 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 07 10:42:12.451875 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 07 10:42:12.451875 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 07 10:42:12.451875 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 07 10:42:12.451875 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 07 10:42:12.451875 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 07 10:42:12.451875 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 07 10:42:12.451875 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 07 10:42:12.451875 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 07 10:42:12.452812 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 07 10:42:12.452812 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 07 10:42:12.452812 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 07 10:42:12.452812 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 07 10:42:12.452812 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 07 10:42:12.452812 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 07 10:42:12.452812 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 07 10:42:12.452812 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 07 10:42:12.452812 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 07 10:42:12.452812 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self.get_or_create( Aug 07 10:42:12.452812 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 07 10:42:12.452812 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application with Lock( Aug 07 10:42:12.452812 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 07 10:42:12.452812 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self._enter() Aug 07 10:42:12.452812 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 07 10:42:12.452812 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 07 10:42:12.453631 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 07 10:42:12.453631 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self.creator() Aug 07 10:42:12.453631 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 07 10:42:12.453631 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application created_value = creator( Aug 07 10:42:12.453631 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 07 10:42:12.453631 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 07 10:42:12.453631 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 07 10:42:12.453631 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 07 10:42:12.453631 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 07 10:42:12.453631 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 07 10:42:12.453631 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 07 10:42:12.453631 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application Aug 07 10:42:12.453631 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 341/667] 10.222.0.25 () {56 vars in 1102 bytes} [Sun Aug 7 10:42:12 2022] GET /identity/v3/roles/service => generated 85 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 07 10:42:12.454321 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a33f25d-f326-4b15-b52c-e75a17fb3ad2 None admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 10:42:12.472588 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=100 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 10:42:12.507154 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37785cd6-b944-4060-b407-e190ce498534 None admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 10:42:12.543563 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b65b52fa-db15-433a-bb36-6be4c88f56f1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 10:42:12.544179 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b65b52fa-db15-433a-bb36-6be4c88f56f1 None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:42:12.544463 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b65b52fa-db15-433a-bb36-6be4c88f56f1 None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:42:12.544751 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b65b52fa-db15-433a-bb36-6be4c88f56f1 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:42:12.552317 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=100 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 10:42:12.563107 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a33f25d-f326-4b15-b52c-e75a17fb3ad2 None admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 10:42:12.599729 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99c276b9-2509-4633-b3b1-d100868e75af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 10:42:12.600360 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99c276b9-2509-4633-b3b1-d100868e75af None admin] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:42:12.600644 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99c276b9-2509-4633-b3b1-d100868e75af None admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:42:12.600917 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99c276b9-2509-4633-b3b1-d100868e75af None admin] PATH_INFO: `/v3/projects/service` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:42:12.608428 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99c276b9-2509-4633-b3b1-d100868e75af None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 10:42:12.609866 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99c276b9-2509-4633-b3b1-d100868e75af None admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 10:42:12.621837 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b65b52fa-db15-433a-bb36-6be4c88f56f1 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 10:42:12.623370 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b65b52fa-db15-433a-bb36-6be4c88f56f1 None admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 10:42:12.701295 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=100 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 10:42:12.736491 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application [None req-99c276b9-2509-4633-b3b1-d100868e75af None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 07 10:42:12.736491 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 07 10:42:12.736491 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 07 10:42:12.736491 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 07 10:42:12.736491 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 07 10:42:12.736491 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 07 10:42:12.736491 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 07 10:42:12.736491 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 07 10:42:12.736491 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 07 10:42:12.736491 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 07 10:42:12.736491 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 07 10:42:12.736491 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 07 10:42:12.736491 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 07 10:42:12.736491 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 07 10:42:12.736491 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 07 10:42:12.737724 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 07 10:42:12.737724 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 07 10:42:12.737724 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 07 10:42:12.737724 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 07 10:42:12.737724 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 07 10:42:12.737724 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 07 10:42:12.737724 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self.get_or_create( Aug 07 10:42:12.737724 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 07 10:42:12.737724 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application with Lock( Aug 07 10:42:12.737724 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 07 10:42:12.737724 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self._enter() Aug 07 10:42:12.737724 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 07 10:42:12.737724 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 07 10:42:12.737724 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 07 10:42:12.737724 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self.creator() Aug 07 10:42:12.737724 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 07 10:42:12.737724 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application created_value = creator( Aug 07 10:42:12.743596 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 07 10:42:12.743596 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 07 10:42:12.743596 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 07 10:42:12.743596 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 07 10:42:12.743596 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 07 10:42:12.743596 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 07 10:42:12.743596 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 07 10:42:12.743596 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application Aug 07 10:42:12.743596 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 342/668] 10.222.0.25 () {56 vars in 1111 bytes} [Sun Aug 7 10:42:12 2022] GET /identity/v3/projects/service => generated 88 bytes in 284 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 07 10:42:12.743596 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99c276b9-2509-4633-b3b1-d100868e75af None admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 10:42:12.744214 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 327/669] 10.222.0.25 () {58 vars in 1282 bytes} [Sun Aug 7 10:42:12 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 364 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 07 10:42:12.744214 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b65b52fa-db15-433a-bb36-6be4c88f56f1 None admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 10:42:12.845779 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99c276b9-2509-4633-b3b1-d100868e75af None admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 10:42:12.855263 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b65b52fa-db15-433a-bb36-6be4c88f56f1 None admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 10:42:12.883727 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cfe195b-b041-4452-be1b-6db3a65ed9ba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 10:42:12.884357 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cfe195b-b041-4452-be1b-6db3a65ed9ba None admin] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:42:12.885163 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cfe195b-b041-4452-be1b-6db3a65ed9ba None admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:42:12.885449 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cfe195b-b041-4452-be1b-6db3a65ed9ba None admin] PATH_INFO: `/v3/projects/service` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:42:12.891446 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce5ea3e6-1d08-4666-b7f8-83fd16cdf08b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 10:42:12.892055 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce5ea3e6-1d08-4666-b7f8-83fd16cdf08b None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:42:12.892260 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cfe195b-b041-4452-be1b-6db3a65ed9ba None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 10:42:12.892374 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce5ea3e6-1d08-4666-b7f8-83fd16cdf08b None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:42:12.892597 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce5ea3e6-1d08-4666-b7f8-83fd16cdf08b None admin] PATH_INFO: `/v3/roles` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:42:12.893257 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce5ea3e6-1d08-4666-b7f8-83fd16cdf08b None admin] RBAC: Adding query filter params (name=service) {{(pid=80143) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 10:42:12.893562 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce5ea3e6-1d08-4666-b7f8-83fd16cdf08b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 10:42:12.893688 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cfe195b-b041-4452-be1b-6db3a65ed9ba None admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 10:42:12.894904 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce5ea3e6-1d08-4666-b7f8-83fd16cdf08b None admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 10:42:12.904697 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application [None req-9cfe195b-b041-4452-be1b-6db3a65ed9ba None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 07 10:42:12.904697 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 07 10:42:12.904697 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 07 10:42:12.904697 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 07 10:42:12.904697 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 07 10:42:12.904697 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 07 10:42:12.904697 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 07 10:42:12.904697 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 07 10:42:12.904697 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 07 10:42:12.904697 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 07 10:42:12.904697 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 07 10:42:12.904697 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 07 10:42:12.904697 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 07 10:42:12.904697 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 07 10:42:12.904697 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 07 10:42:12.905774 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 07 10:42:12.905774 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 07 10:42:12.905774 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 07 10:42:12.905774 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 07 10:42:12.905774 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 07 10:42:12.905774 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 07 10:42:12.905774 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self.get_or_create( Aug 07 10:42:12.905774 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 07 10:42:12.905774 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application with Lock( Aug 07 10:42:12.905774 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 07 10:42:12.905774 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self._enter() Aug 07 10:42:12.905774 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 07 10:42:12.905774 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 07 10:42:12.905774 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 07 10:42:12.905774 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self.creator() Aug 07 10:42:12.905774 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 07 10:42:12.905774 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application created_value = creator( Aug 07 10:42:12.906656 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 07 10:42:12.906656 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 07 10:42:12.906656 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 07 10:42:12.906656 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 07 10:42:12.906656 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 07 10:42:12.906656 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 07 10:42:12.906656 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 07 10:42:12.906656 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application Aug 07 10:42:12.907121 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 328/670] 10.222.0.25 () {56 vars in 1103 bytes} [Sun Aug 7 10:42:12 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 10:42:12.907193 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 343/671] 10.222.0.25 () {56 vars in 1111 bytes} [Sun Aug 7 10:42:12 2022] GET /identity/v3/projects/service => generated 88 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 07 10:42:12.907260 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce5ea3e6-1d08-4666-b7f8-83fd16cdf08b None admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 10:42:12.907342 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cfe195b-b041-4452-be1b-6db3a65ed9ba None admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 10:42:12.922319 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=100 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 10:42:12.945595 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=100 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 10:42:13.007497 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cfe195b-b041-4452-be1b-6db3a65ed9ba None admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 10:42:13.012494 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce5ea3e6-1d08-4666-b7f8-83fd16cdf08b None admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 10:42:13.040431 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b201b19b-0a20-441f-a256-8fc379d32bda None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 10:42:13.041567 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b201b19b-0a20-441f-a256-8fc379d32bda None admin] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:42:13.041845 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b201b19b-0a20-441f-a256-8fc379d32bda None admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:42:13.042115 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b201b19b-0a20-441f-a256-8fc379d32bda None admin] PATH_INFO: `/v3/projects` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:42:13.042808 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b201b19b-0a20-441f-a256-8fc379d32bda None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80144) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 10:42:13.043148 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b201b19b-0a20-441f-a256-8fc379d32bda None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 10:42:13.044550 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b201b19b-0a20-441f-a256-8fc379d32bda None admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 10:42:13.045336 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00689480-0043-4029-a98d-124d44701327 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 10:42:13.045964 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00689480-0043-4029-a98d-124d44701327 None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:42:13.046243 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00689480-0043-4029-a98d-124d44701327 None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:42:13.046515 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00689480-0043-4029-a98d-124d44701327 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:42:13.055754 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 344/672] 10.222.0.25 () {56 vars in 1148 bytes} [Sun Aug 7 10:42:12 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 10:42:13.057476 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4edd128-75db-4398-a236-e9d145de3f66 None None] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:42:13.057765 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4edd128-75db-4398-a236-e9d145de3f66 None None] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:42:13.058038 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4edd128-75db-4398-a236-e9d145de3f66 None None] PATH_INFO: `/` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:42:13.058154 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00689480-0043-4029-a98d-124d44701327 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 10:42:13.059419 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 345/673] 10.222.0.25 () {54 vars in 927 bytes} [Sun Aug 7 10:42:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 07 10:42:13.059678 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00689480-0043-4029-a98d-124d44701327 None admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 10:42:13.063915 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4edd128-75db-4398-a236-e9d145de3f66 None None] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 10:42:13.074241 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application [None req-00689480-0043-4029-a98d-124d44701327 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 07 10:42:13.074241 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 07 10:42:13.074241 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 07 10:42:13.074241 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 07 10:42:13.074241 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 07 10:42:13.074241 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 07 10:42:13.074241 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 07 10:42:13.074241 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 07 10:42:13.074241 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 07 10:42:13.074241 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 07 10:42:13.074241 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 07 10:42:13.074241 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 07 10:42:13.074241 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 07 10:42:13.074241 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 07 10:42:13.074241 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 07 10:42:13.074241 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 07 10:42:13.075342 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 07 10:42:13.075342 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 07 10:42:13.075342 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 07 10:42:13.075342 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 07 10:42:13.075342 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 07 10:42:13.075342 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.get_or_create( Aug 07 10:42:13.075342 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 07 10:42:13.075342 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application with Lock( Aug 07 10:42:13.075342 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 07 10:42:13.075342 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self._enter() Aug 07 10:42:13.075342 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 07 10:42:13.075342 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 07 10:42:13.075342 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 07 10:42:13.075342 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.creator() Aug 07 10:42:13.075342 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 07 10:42:13.075342 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application created_value = creator( Aug 07 10:42:13.075342 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 07 10:42:13.076220 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 07 10:42:13.076220 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 07 10:42:13.076220 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application Aug 07 10:42:13.076220 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 329/674] 10.222.0.25 () {56 vars in 1108 bytes} [Sun Aug 7 10:42:12 2022] GET /identity/v3/domains/Default => generated 87 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 07 10:42:13.076220 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00689480-0043-4029-a98d-124d44701327 None admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 10:42:13.142716 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=100 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 10:42:13.166766 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=100 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 10:42:13.174574 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4edd128-75db-4398-a236-e9d145de3f66 None None] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 10:42:13.181525 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00689480-0043-4029-a98d-124d44701327 None admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 10:42:13.207865 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48b7026b-f692-40db-bfd4-fc299007e75c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 10:42:13.208470 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48b7026b-f692-40db-bfd4-fc299007e75c None admin] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:42:13.208750 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48b7026b-f692-40db-bfd4-fc299007e75c None admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:42:13.209022 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48b7026b-f692-40db-bfd4-fc299007e75c None admin] PATH_INFO: `/v3/projects` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:42:13.209703 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48b7026b-f692-40db-bfd4-fc299007e75c None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80144) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 10:42:13.210007 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48b7026b-f692-40db-bfd4-fc299007e75c None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 10:42:13.211452 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48b7026b-f692-40db-bfd4-fc299007e75c None admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 10:42:13.216134 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52b5876b-9baf-4e74-afc4-7545940ad79c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 10:42:13.216763 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52b5876b-9baf-4e74-afc4-7545940ad79c None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:42:13.217049 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52b5876b-9baf-4e74-afc4-7545940ad79c None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:42:13.217321 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52b5876b-9baf-4e74-afc4-7545940ad79c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:42:13.223382 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 346/675] 10.222.0.25 () {56 vars in 1148 bytes} [Sun Aug 7 10:42:13 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 10:42:13.224180 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48b7026b-f692-40db-bfd4-fc299007e75c None admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 10:42:13.227725 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52b5876b-9baf-4e74-afc4-7545940ad79c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 10:42:13.230051 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52b5876b-9baf-4e74-afc4-7545940ad79c None admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 10:42:13.244852 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application [None req-52b5876b-9baf-4e74-afc4-7545940ad79c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 07 10:42:13.244852 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 07 10:42:13.244852 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 07 10:42:13.244852 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 07 10:42:13.244852 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 07 10:42:13.244852 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 07 10:42:13.244852 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 07 10:42:13.244852 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 07 10:42:13.244852 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 07 10:42:13.244852 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 07 10:42:13.244852 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 07 10:42:13.244852 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 07 10:42:13.244852 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 07 10:42:13.244852 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 07 10:42:13.244852 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 07 10:42:13.244852 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 07 10:42:13.245923 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 07 10:42:13.245923 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 07 10:42:13.245923 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 07 10:42:13.245923 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 07 10:42:13.245923 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 07 10:42:13.245923 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.get_or_create( Aug 07 10:42:13.245923 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 07 10:42:13.245923 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application with Lock( Aug 07 10:42:13.245923 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 07 10:42:13.245923 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self._enter() Aug 07 10:42:13.245923 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 07 10:42:13.245923 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 07 10:42:13.245923 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 07 10:42:13.245923 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.creator() Aug 07 10:42:13.245923 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 07 10:42:13.245923 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application created_value = creator( Aug 07 10:42:13.245923 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 07 10:42:13.247718 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 07 10:42:13.247718 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 07 10:42:13.247718 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application Aug 07 10:42:13.247718 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 330/676] 10.222.0.25 () {56 vars in 1108 bytes} [Sun Aug 7 10:42:13 2022] GET /identity/v3/domains/Default => generated 87 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 07 10:42:13.247718 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec1f367b-8bfc-4db1-a403-8dc07e5c8e53 None None] REQUEST_METHOD: `POST` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:42:13.247718 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec1f367b-8bfc-4db1-a403-8dc07e5c8e53 None None] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:42:13.247718 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec1f367b-8bfc-4db1-a403-8dc07e5c8e53 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:42:13.267530 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.auth.core [None req-ec1f367b-8bfc-4db1-a403-8dc07e5c8e53 None None] MFA Rules not processed for user `bac4e6b76fe64217ac8fa44e71cffe6a`. Rule list: `[]` (Enabled: `True`). {{(pid=80143) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 10:42:13.268726 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-ec1f367b-8bfc-4db1-a403-8dc07e5c8e53 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 10:42:13.332555 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48b7026b-f692-40db-bfd4-fc299007e75c None admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 10:42:13.335933 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 331/677] 10.222.0.25 () {58 vars in 1024 bytes} [Sun Aug 7 10:42:13 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 07 10:42:13.336753 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec1f367b-8bfc-4db1-a403-8dc07e5c8e53 None None] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 10:42:13.374409 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=100 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 10:42:13.380077 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b06a6ac2-8283-4654-9b47-c7ee9d0c7bd3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 10:42:13.380687 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b06a6ac2-8283-4654-9b47-c7ee9d0c7bd3 None admin] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:42:13.380964 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b06a6ac2-8283-4654-9b47-c7ee9d0c7bd3 None admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:42:13.381231 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b06a6ac2-8283-4654-9b47-c7ee9d0c7bd3 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:42:13.381944 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b06a6ac2-8283-4654-9b47-c7ee9d0c7bd3 None admin] RBAC: Adding query filter params (role.id=fbf375e9760340c9aa9529333eeb25f9, scope.project.id=6de1f3cac02c4bcdb05dd2337176033d, user.id=88dbd6dfd1164994a3e16da553561910) {{(pid=80144) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 10:42:13.382248 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b06a6ac2-8283-4654-9b47-c7ee9d0c7bd3 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 10:42:13.383687 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b06a6ac2-8283-4654-9b47-c7ee9d0c7bd3 None admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 10:42:13.389971 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 347/678] 10.222.0.25 () {56 vars in 1374 bytes} [Sun Aug 7 10:42:13 2022] GET /identity/v3/role_assignments?user.id=88dbd6dfd1164994a3e16da553561910&scope.project.id=6de1f3cac02c4bcdb05dd2337176033d&role.id=fbf375e9760340c9aa9529333eeb25f9 => generated 608 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 10:42:13.390747 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b06a6ac2-8283-4654-9b47-c7ee9d0c7bd3 None admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 10:42:13.403653 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=100 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 10:42:13.443215 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec1f367b-8bfc-4db1-a403-8dc07e5c8e53 None None] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 10:42:13.481127 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a76136b2-e156-4c86-9bbd-e4fc7e08784c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 10:42:13.481875 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a76136b2-e156-4c86-9bbd-e4fc7e08784c None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:42:13.482161 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a76136b2-e156-4c86-9bbd-e4fc7e08784c None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:42:13.482440 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a76136b2-e156-4c86-9bbd-e4fc7e08784c None admin] PATH_INFO: `/v3/domains` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:42:13.483135 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a76136b2-e156-4c86-9bbd-e4fc7e08784c None admin] RBAC: Adding query filter params (name=Default) {{(pid=80143) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 10:42:13.483443 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a76136b2-e156-4c86-9bbd-e4fc7e08784c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 10:42:13.484849 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a76136b2-e156-4c86-9bbd-e4fc7e08784c None admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 10:42:13.498428 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 332/679] 10.222.0.25 () {56 vars in 1109 bytes} [Sun Aug 7 10:42:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 10:42:13.499136 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a76136b2-e156-4c86-9bbd-e4fc7e08784c None admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 10:42:13.527338 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b06a6ac2-8283-4654-9b47-c7ee9d0c7bd3 None admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 10:42:13.565726 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51fc87fa-5fda-41fd-bd02-16d9b20ac8bd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 10:42:13.566665 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51fc87fa-5fda-41fd-bd02-16d9b20ac8bd None admin] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:42:13.566972 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51fc87fa-5fda-41fd-bd02-16d9b20ac8bd None admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:42:13.567251 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51fc87fa-5fda-41fd-bd02-16d9b20ac8bd None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:42:13.567975 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51fc87fa-5fda-41fd-bd02-16d9b20ac8bd None admin] RBAC: Adding query filter params (role.id=fbf375e9760340c9aa9529333eeb25f9, scope.project.id=6de1f3cac02c4bcdb05dd2337176033d, user.id=a11186d9d0d6488b953ede2f1628148d) {{(pid=80144) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 10:42:13.568285 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51fc87fa-5fda-41fd-bd02-16d9b20ac8bd None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 10:42:13.569696 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51fc87fa-5fda-41fd-bd02-16d9b20ac8bd None admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 10:42:13.576425 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 348/680] 10.222.0.25 () {56 vars in 1374 bytes} [Sun Aug 7 10:42:13 2022] GET /identity/v3/role_assignments?user.id=a11186d9d0d6488b953ede2f1628148d&scope.project.id=6de1f3cac02c4bcdb05dd2337176033d&role.id=fbf375e9760340c9aa9529333eeb25f9 => generated 608 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 10:42:13.577212 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51fc87fa-5fda-41fd-bd02-16d9b20ac8bd None admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 10:42:13.622863 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a76136b2-e156-4c86-9bbd-e4fc7e08784c None admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 10:42:13.651419 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=100 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 10:42:13.659893 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=100 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 10:42:13.667458 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c7eed5c-f9f8-4dd2-829d-3425a1cfb846 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 10:42:13.668103 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c7eed5c-f9f8-4dd2-829d-3425a1cfb846 None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:42:13.668384 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c7eed5c-f9f8-4dd2-829d-3425a1cfb846 None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:42:13.668659 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c7eed5c-f9f8-4dd2-829d-3425a1cfb846 None admin] PATH_INFO: `/v3/domains` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:42:13.669330 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c7eed5c-f9f8-4dd2-829d-3425a1cfb846 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80143) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 10:42:13.669637 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c7eed5c-f9f8-4dd2-829d-3425a1cfb846 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 10:42:13.671108 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c7eed5c-f9f8-4dd2-829d-3425a1cfb846 None admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 10:42:13.691085 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 333/681] 10.222.0.25 () {56 vars in 1109 bytes} [Sun Aug 7 10:42:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 10:42:13.692287 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c7eed5c-f9f8-4dd2-829d-3425a1cfb846 None admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 10:42:13.712452 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51fc87fa-5fda-41fd-bd02-16d9b20ac8bd None admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 10:42:13.751162 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cba5509-ca04-489b-8c52-cc8928b13c3f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 10:42:13.751855 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cba5509-ca04-489b-8c52-cc8928b13c3f None admin] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:42:13.752142 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cba5509-ca04-489b-8c52-cc8928b13c3f None admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:42:13.752416 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cba5509-ca04-489b-8c52-cc8928b13c3f None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:42:13.767147 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cba5509-ca04-489b-8c52-cc8928b13c3f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 07 10:42:13.772306 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application [None req-3cba5509-ca04-489b-8c52-cc8928b13c3f None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 07 10:42:13.772306 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 07 10:42:13.772306 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 07 10:42:13.772306 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 07 10:42:13.772306 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 07 10:42:13.772306 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 07 10:42:13.772306 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 07 10:42:13.772306 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 07 10:42:13.772306 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 07 10:42:13.772306 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 07 10:42:13.772306 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 07 10:42:13.772306 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 07 10:42:13.772306 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 07 10:42:13.772306 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 07 10:42:13.772306 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 07 10:42:13.772306 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 07 10:42:13.773311 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 07 10:42:13.773311 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 07 10:42:13.773311 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 07 10:42:13.773311 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 07 10:42:13.773311 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 07 10:42:13.773311 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 07 10:42:13.773311 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 07 10:42:13.773311 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 07 10:42:13.773311 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 07 10:42:13.773311 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self.get_or_create( Aug 07 10:42:13.773311 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 07 10:42:13.773311 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application with Lock( Aug 07 10:42:13.773311 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 07 10:42:13.773311 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self._enter() Aug 07 10:42:13.773311 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 07 10:42:13.773311 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 07 10:42:13.774141 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 07 10:42:13.774141 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self.creator() Aug 07 10:42:13.774141 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 07 10:42:13.774141 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application created_value = creator( Aug 07 10:42:13.774141 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 07 10:42:13.774141 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 07 10:42:13.774141 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 07 10:42:13.774141 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 07 10:42:13.774141 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 07 10:42:13.774141 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 07 10:42:13.774141 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 07 10:42:13.774141 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application Aug 07 10:42:13.778399 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 349/682] 10.222.0.25 () {56 vars in 1096 bytes} [Sun Aug 7 10:42:13 2022] GET /identity/v3/roles/admin => generated 83 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 07 10:42:13.779822 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cba5509-ca04-489b-8c52-cc8928b13c3f None admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 10:42:13.828879 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c7eed5c-f9f8-4dd2-829d-3425a1cfb846 None admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 10:42:13.874993 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ac51105-9b51-4ec2-b811-45dff429438e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 10:42:13.875632 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ac51105-9b51-4ec2-b811-45dff429438e None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:42:13.875907 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ac51105-9b51-4ec2-b811-45dff429438e None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:42:13.876176 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ac51105-9b51-4ec2-b811-45dff429438e None admin] PATH_INFO: `/v3/projects/service` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:42:13.883268 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ac51105-9b51-4ec2-b811-45dff429438e None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 10:42:13.885375 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ac51105-9b51-4ec2-b811-45dff429438e None admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 10:42:13.897590 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application [None req-4ac51105-9b51-4ec2-b811-45dff429438e None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 07 10:42:13.897590 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 07 10:42:13.897590 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 07 10:42:13.897590 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 07 10:42:13.897590 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 07 10:42:13.897590 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 07 10:42:13.897590 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 07 10:42:13.897590 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 07 10:42:13.897590 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 07 10:42:13.897590 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 07 10:42:13.897590 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 07 10:42:13.897590 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 07 10:42:13.897590 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 07 10:42:13.897590 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 07 10:42:13.897590 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 07 10:42:13.898612 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 07 10:42:13.898612 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 07 10:42:13.898612 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 07 10:42:13.898612 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 07 10:42:13.898612 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 07 10:42:13.898612 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 07 10:42:13.898612 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.get_or_create( Aug 07 10:42:13.898612 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 07 10:42:13.898612 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application with Lock( Aug 07 10:42:13.898612 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 07 10:42:13.898612 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self._enter() Aug 07 10:42:13.898612 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 07 10:42:13.898612 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 07 10:42:13.898612 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 07 10:42:13.898612 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.creator() Aug 07 10:42:13.898612 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 07 10:42:13.898612 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application created_value = creator( Aug 07 10:42:13.899688 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 07 10:42:13.899688 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 07 10:42:13.899688 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 07 10:42:13.899688 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 07 10:42:13.899688 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 07 10:42:13.899688 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 07 10:42:13.899688 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 07 10:42:13.899688 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application Aug 07 10:42:13.899688 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 334/683] 10.222.0.25 () {56 vars in 1111 bytes} [Sun Aug 7 10:42:13 2022] GET /identity/v3/projects/service => generated 88 bytes in 207 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 07 10:42:13.900881 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ac51105-9b51-4ec2-b811-45dff429438e None admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 10:42:13.926634 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cba5509-ca04-489b-8c52-cc8928b13c3f None admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 10:42:13.953518 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=100 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 10:42:13.956731 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=100 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 10:42:13.970960 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38fc27c1-1757-4869-981b-edb1ac5ea1a3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 10:42:13.971719 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38fc27c1-1757-4869-981b-edb1ac5ea1a3 None admin] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:42:13.972013 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38fc27c1-1757-4869-981b-edb1ac5ea1a3 None admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:42:13.972290 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38fc27c1-1757-4869-981b-edb1ac5ea1a3 None admin] PATH_INFO: `/v3/users/swift` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:42:13.981238 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38fc27c1-1757-4869-981b-edb1ac5ea1a3 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 10:42:13.982687 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38fc27c1-1757-4869-981b-edb1ac5ea1a3 None admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 10:42:13.995205 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application [None req-38fc27c1-1757-4869-981b-edb1ac5ea1a3 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Aug 07 10:42:13.995205 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 07 10:42:13.995205 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 07 10:42:13.995205 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 07 10:42:13.995205 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 07 10:42:13.995205 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 07 10:42:13.995205 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 07 10:42:13.995205 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 07 10:42:13.995205 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 07 10:42:13.995205 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 07 10:42:13.995205 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 07 10:42:13.995205 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 07 10:42:13.995205 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 07 10:42:13.995205 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 07 10:42:13.995205 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 07 10:42:13.995205 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 07 10:42:13.997200 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 07 10:42:13.997200 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 07 10:42:13.997200 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 07 10:42:13.997200 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 07 10:42:13.997200 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 07 10:42:13.997200 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 07 10:42:13.997200 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 07 10:42:13.997200 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 07 10:42:13.997200 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 07 10:42:13.997200 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self.get_or_create( Aug 07 10:42:13.997200 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 07 10:42:13.997200 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application with Lock( Aug 07 10:42:13.997200 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 07 10:42:13.997200 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self._enter() Aug 07 10:42:13.997200 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 07 10:42:13.997200 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 07 10:42:13.997200 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 07 10:42:13.997200 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self.creator() Aug 07 10:42:13.999348 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 07 10:42:13.999348 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application created_value = creator( Aug 07 10:42:13.999348 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 07 10:42:13.999348 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 07 10:42:13.999348 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 07 10:42:13.999348 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 07 10:42:13.999348 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 07 10:42:13.999348 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 07 10:42:13.999348 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Aug 07 10:42:13.999348 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application Aug 07 10:42:13.999348 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 350/684] 10.222.0.25 () {56 vars in 1096 bytes} [Sun Aug 7 10:42:13 2022] GET /identity/v3/users/swift => generated 83 bytes in 218 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 07 10:42:13.999348 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38fc27c1-1757-4869-981b-edb1ac5ea1a3 None admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 10:42:14.018033 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ac51105-9b51-4ec2-b811-45dff429438e None admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 10:42:14.053004 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdfa4637-7006-4f0b-923f-1b0288b933b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 10:42:14.053646 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdfa4637-7006-4f0b-923f-1b0288b933b5 None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:42:14.053927 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdfa4637-7006-4f0b-923f-1b0288b933b5 None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:42:14.054202 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdfa4637-7006-4f0b-923f-1b0288b933b5 None admin] PATH_INFO: `/v3/roles` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:42:14.054874 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdfa4637-7006-4f0b-923f-1b0288b933b5 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80143) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 10:42:14.055206 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdfa4637-7006-4f0b-923f-1b0288b933b5 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 10:42:14.057227 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdfa4637-7006-4f0b-923f-1b0288b933b5 None admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 10:42:14.070487 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 335/685] 10.222.0.25 () {56 vars in 1099 bytes} [Sun Aug 7 10:42:13 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 10:42:14.071386 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdfa4637-7006-4f0b-923f-1b0288b933b5 None admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 10:42:14.111382 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38fc27c1-1757-4869-981b-edb1ac5ea1a3 None admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 10:42:14.156294 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2fa8521-b7ca-4ff0-8503-da615f98253e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 10:42:14.157380 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2fa8521-b7ca-4ff0-8503-da615f98253e None admin] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:42:14.157669 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2fa8521-b7ca-4ff0-8503-da615f98253e None admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:42:14.158098 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2fa8521-b7ca-4ff0-8503-da615f98253e None admin] PATH_INFO: `/v3/projects` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:42:14.158943 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2fa8521-b7ca-4ff0-8503-da615f98253e None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80144) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 10:42:14.159260 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2fa8521-b7ca-4ff0-8503-da615f98253e None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 10:42:14.161342 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2fa8521-b7ca-4ff0-8503-da615f98253e None admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 10:42:14.174808 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 351/686] 10.222.0.25 () {56 vars in 1148 bytes} [Sun Aug 7 10:42:13 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 10:42:14.175783 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2fa8521-b7ca-4ff0-8503-da615f98253e None admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 10:42:14.176439 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdfa4637-7006-4f0b-923f-1b0288b933b5 None admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 10:42:14.191482 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=100 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 10:42:14.218357 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10f56e2d-ad86-4e07-8dec-f4013469d157 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 10:42:14.219433 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=100 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 10:42:14.220140 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10f56e2d-ad86-4e07-8dec-f4013469d157 None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:42:14.220593 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10f56e2d-ad86-4e07-8dec-f4013469d157 None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:42:14.221034 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10f56e2d-ad86-4e07-8dec-f4013469d157 None admin] PATH_INFO: `/v3/users` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:42:14.221916 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10f56e2d-ad86-4e07-8dec-f4013469d157 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80143) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 10:42:14.222415 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10f56e2d-ad86-4e07-8dec-f4013469d157 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 10:42:14.224000 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10f56e2d-ad86-4e07-8dec-f4013469d157 None admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 10:42:14.245451 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 336/687] 10.222.0.25 () {56 vars in 1135 bytes} [Sun Aug 7 10:42:14 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 10:42:14.246753 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10f56e2d-ad86-4e07-8dec-f4013469d157 None admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 10:42:14.280071 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2fa8521-b7ca-4ff0-8503-da615f98253e None admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 10:42:14.315531 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5731529-2fa3-44a4-b4e7-e454262e8585 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 10:42:14.316159 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5731529-2fa3-44a4-b4e7-e454262e8585 None admin] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:42:14.316429 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5731529-2fa3-44a4-b4e7-e454262e8585 None admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:42:14.316775 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5731529-2fa3-44a4-b4e7-e454262e8585 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:42:14.327028 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5731529-2fa3-44a4-b4e7-e454262e8585 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 10:42:14.328523 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5731529-2fa3-44a4-b4e7-e454262e8585 None admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 10:42:14.343673 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application [None req-b5731529-2fa3-44a4-b4e7-e454262e8585 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 07 10:42:14.343673 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 07 10:42:14.343673 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 07 10:42:14.343673 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 07 10:42:14.343673 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 07 10:42:14.343673 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 07 10:42:14.343673 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 07 10:42:14.343673 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 07 10:42:14.343673 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 07 10:42:14.343673 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 07 10:42:14.343673 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 07 10:42:14.343673 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 07 10:42:14.343673 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 07 10:42:14.343673 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 07 10:42:14.343673 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 07 10:42:14.343673 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 07 10:42:14.344647 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 07 10:42:14.344647 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 07 10:42:14.344647 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 07 10:42:14.344647 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 07 10:42:14.344647 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 07 10:42:14.344647 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self.get_or_create( Aug 07 10:42:14.344647 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 07 10:42:14.344647 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application with Lock( Aug 07 10:42:14.344647 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 07 10:42:14.344647 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self._enter() Aug 07 10:42:14.344647 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 07 10:42:14.344647 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 07 10:42:14.344647 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 07 10:42:14.344647 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self.creator() Aug 07 10:42:14.344647 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 07 10:42:14.344647 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application created_value = creator( Aug 07 10:42:14.344647 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 07 10:42:14.345849 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 07 10:42:14.345849 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 07 10:42:14.345849 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application Aug 07 10:42:14.345849 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 352/688] 10.222.0.25 () {56 vars in 1108 bytes} [Sun Aug 7 10:42:14 2022] GET /identity/v3/domains/Default => generated 87 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 07 10:42:14.345849 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5731529-2fa3-44a4-b4e7-e454262e8585 None admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 10:42:14.360687 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10f56e2d-ad86-4e07-8dec-f4013469d157 None admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 10:42:14.398912 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52c8913e-d5dd-4370-9fac-1c6202f44ce1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 10:42:14.399804 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52c8913e-d5dd-4370-9fac-1c6202f44ce1 None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:42:14.400483 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52c8913e-d5dd-4370-9fac-1c6202f44ce1 None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:42:14.400960 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52c8913e-d5dd-4370-9fac-1c6202f44ce1 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:42:14.401829 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52c8913e-d5dd-4370-9fac-1c6202f44ce1 None admin] RBAC: Adding query filter params (role.id=fbf375e9760340c9aa9529333eeb25f9, scope.project.id=6de1f3cac02c4bcdb05dd2337176033d, user.id=be1283c9bf47424c9aa3ad27cab03027) {{(pid=80143) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 10:42:14.402291 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52c8913e-d5dd-4370-9fac-1c6202f44ce1 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 10:42:14.404143 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52c8913e-d5dd-4370-9fac-1c6202f44ce1 None admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 10:42:14.419045 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 337/689] 10.222.0.25 () {56 vars in 1374 bytes} [Sun Aug 7 10:42:14 2022] GET /identity/v3/role_assignments?user.id=be1283c9bf47424c9aa3ad27cab03027&scope.project.id=6de1f3cac02c4bcdb05dd2337176033d&role.id=fbf375e9760340c9aa9529333eeb25f9 => generated 608 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 10:42:14.420310 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52c8913e-d5dd-4370-9fac-1c6202f44ce1 None admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 10:42:14.456727 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=100 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 10:42:14.480293 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=100 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 10:42:14.494783 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5731529-2fa3-44a4-b4e7-e454262e8585 None admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 10:42:14.533922 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52c8913e-d5dd-4370-9fac-1c6202f44ce1 None admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 10:42:14.563708 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caa20840-7cd5-4e2c-8b53-933084f2194d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 10:42:14.564340 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caa20840-7cd5-4e2c-8b53-933084f2194d None admin] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:42:14.564624 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caa20840-7cd5-4e2c-8b53-933084f2194d None admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:42:14.564899 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caa20840-7cd5-4e2c-8b53-933084f2194d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:42:14.573001 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e38a6d5-f98e-4b71-ac0d-e91054b17d07 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 10:42:14.573791 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e38a6d5-f98e-4b71-ac0d-e91054b17d07 None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:42:14.574210 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e38a6d5-f98e-4b71-ac0d-e91054b17d07 None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:42:14.574638 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e38a6d5-f98e-4b71-ac0d-e91054b17d07 None admin] PATH_INFO: `/v3/domains` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:42:14.575458 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e38a6d5-f98e-4b71-ac0d-e91054b17d07 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80143) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 10:42:14.575900 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e38a6d5-f98e-4b71-ac0d-e91054b17d07 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 10:42:14.577431 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e38a6d5-f98e-4b71-ac0d-e91054b17d07 None admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 10:42:14.591887 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 338/690] 10.222.0.25 () {56 vars in 1109 bytes} [Sun Aug 7 10:42:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 10:42:14.597405 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e38a6d5-f98e-4b71-ac0d-e91054b17d07 None admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 10:42:14.708276 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caa20840-7cd5-4e2c-8b53-933084f2194d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 10:42:14.710313 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caa20840-7cd5-4e2c-8b53-933084f2194d None admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 10:42:14.764430 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=100 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 10:42:14.774876 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e38a6d5-f98e-4b71-ac0d-e91054b17d07 None admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 10:42:14.811416 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-237a15b7-3fb3-46e0-a325-54e765534318 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 10:42:14.812118 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-237a15b7-3fb3-46e0-a325-54e765534318 None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:42:14.812405 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-237a15b7-3fb3-46e0-a325-54e765534318 None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:42:14.812678 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-237a15b7-3fb3-46e0-a325-54e765534318 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:42:14.820234 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-237a15b7-3fb3-46e0-a325-54e765534318 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 10:42:14.821696 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-237a15b7-3fb3-46e0-a325-54e765534318 None admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 10:42:14.825198 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=100 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 10:42:14.833903 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application [None req-237a15b7-3fb3-46e0-a325-54e765534318 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Aug 07 10:42:14.833903 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 07 10:42:14.833903 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 07 10:42:14.833903 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 07 10:42:14.833903 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 07 10:42:14.833903 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 07 10:42:14.833903 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 07 10:42:14.833903 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 07 10:42:14.833903 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 07 10:42:14.833903 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 07 10:42:14.833903 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 07 10:42:14.833903 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 07 10:42:14.833903 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 07 10:42:14.833903 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 07 10:42:14.833903 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 07 10:42:14.833903 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 07 10:42:14.835081 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 07 10:42:14.835081 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 07 10:42:14.835081 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 07 10:42:14.835081 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 07 10:42:14.835081 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 07 10:42:14.835081 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 07 10:42:14.835081 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 07 10:42:14.835081 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 07 10:42:14.835081 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 07 10:42:14.835081 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.get_or_create( Aug 07 10:42:14.835081 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 07 10:42:14.835081 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application with Lock( Aug 07 10:42:14.835081 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 07 10:42:14.835081 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self._enter() Aug 07 10:42:14.835081 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 07 10:42:14.835081 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 07 10:42:14.835081 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 07 10:42:14.835081 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.creator() Aug 07 10:42:14.836392 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 07 10:42:14.836392 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application created_value = creator( Aug 07 10:42:14.836392 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 07 10:42:14.836392 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 07 10:42:14.836392 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 07 10:42:14.836392 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 07 10:42:14.836392 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 07 10:42:14.836392 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 07 10:42:14.836392 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Aug 07 10:42:14.836392 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application Aug 07 10:42:14.836392 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 339/691] 10.222.0.25 () {56 vars in 1093 bytes} [Sun Aug 7 10:42:14 2022] GET /identity/v3/users/nova => generated 82 bytes in 238 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 07 10:42:14.839948 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 353/692] 10.222.0.25 () {58 vars in 1282 bytes} [Sun Aug 7 10:42:14 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 494 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 07 10:42:14.845185 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-237a15b7-3fb3-46e0-a325-54e765534318 None admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 10:42:14.846881 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caa20840-7cd5-4e2c-8b53-933084f2194d None admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 10:42:14.970487 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caa20840-7cd5-4e2c-8b53-933084f2194d None admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 10:42:14.974615 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-237a15b7-3fb3-46e0-a325-54e765534318 None admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 10:42:15.014286 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de9898f2-d4b2-425d-8617-ad0a60e7fa6a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 10:42:15.014286 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de9898f2-d4b2-425d-8617-ad0a60e7fa6a None admin] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:42:15.014286 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de9898f2-d4b2-425d-8617-ad0a60e7fa6a None admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:42:15.014286 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de9898f2-d4b2-425d-8617-ad0a60e7fa6a None admin] PATH_INFO: `/v3/users` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:42:15.014760 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de9898f2-d4b2-425d-8617-ad0a60e7fa6a None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80144) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 10:42:15.014760 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de9898f2-d4b2-425d-8617-ad0a60e7fa6a None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 10:42:15.014760 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de9898f2-d4b2-425d-8617-ad0a60e7fa6a None admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 10:42:15.019397 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1f9dc73-9161-4c68-927a-8f0deaef64e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 10:42:15.020224 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1f9dc73-9161-4c68-927a-8f0deaef64e6 None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:42:15.020659 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1f9dc73-9161-4c68-927a-8f0deaef64e6 None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:42:15.021067 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1f9dc73-9161-4c68-927a-8f0deaef64e6 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:42:15.033593 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 354/693] 10.222.0.25 () {56 vars in 1133 bytes} [Sun Aug 7 10:42:14 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 07 10:42:15.042626 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1f9dc73-9161-4c68-927a-8f0deaef64e6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 10:42:15.044115 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de9898f2-d4b2-425d-8617-ad0a60e7fa6a None admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 10:42:15.044626 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1f9dc73-9161-4c68-927a-8f0deaef64e6 None admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 10:42:15.060795 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application [None req-f1f9dc73-9161-4c68-927a-8f0deaef64e6 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 07 10:42:15.060795 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 07 10:42:15.060795 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 07 10:42:15.060795 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 07 10:42:15.060795 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 07 10:42:15.060795 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 07 10:42:15.060795 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 07 10:42:15.060795 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 07 10:42:15.060795 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 07 10:42:15.060795 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 07 10:42:15.060795 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 07 10:42:15.060795 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 07 10:42:15.060795 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 07 10:42:15.060795 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 07 10:42:15.060795 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 07 10:42:15.060795 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 07 10:42:15.061864 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 07 10:42:15.061864 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 07 10:42:15.061864 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 07 10:42:15.061864 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 07 10:42:15.061864 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 07 10:42:15.061864 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.get_or_create( Aug 07 10:42:15.061864 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 07 10:42:15.061864 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application with Lock( Aug 07 10:42:15.061864 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 07 10:42:15.061864 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self._enter() Aug 07 10:42:15.061864 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 07 10:42:15.061864 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 07 10:42:15.061864 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 07 10:42:15.061864 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.creator() Aug 07 10:42:15.061864 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 07 10:42:15.061864 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application created_value = creator( Aug 07 10:42:15.061864 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 07 10:42:15.062973 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 07 10:42:15.062973 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 07 10:42:15.062973 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application Aug 07 10:42:15.064652 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 340/694] 10.222.0.25 () {56 vars in 1108 bytes} [Sun Aug 7 10:42:14 2022] GET /identity/v3/domains/Default => generated 87 bytes in 220 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 07 10:42:15.070192 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1f9dc73-9161-4c68-927a-8f0deaef64e6 None admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 10:42:15.075438 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=100 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 10:42:15.119908 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=100 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 10:42:15.174469 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de9898f2-d4b2-425d-8617-ad0a60e7fa6a None admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 10:42:15.228878 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89470dd1-f9cb-4f6d-b447-5d8f49487079 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 10:42:15.229911 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89470dd1-f9cb-4f6d-b447-5d8f49487079 None admin] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:42:15.230291 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89470dd1-f9cb-4f6d-b447-5d8f49487079 None admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:42:15.230776 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89470dd1-f9cb-4f6d-b447-5d8f49487079 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:42:15.245079 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1f9dc73-9161-4c68-927a-8f0deaef64e6 None admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 10:42:15.303638 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89470dd1-f9cb-4f6d-b447-5d8f49487079 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 10:42:15.305262 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89470dd1-f9cb-4f6d-b447-5d8f49487079 None admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 10:42:15.343637 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d324ab8-d058-4faf-a26e-a08fc452e6cd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 10:42:15.344259 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d324ab8-d058-4faf-a26e-a08fc452e6cd None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:42:15.344536 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d324ab8-d058-4faf-a26e-a08fc452e6cd None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:42:15.344807 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d324ab8-d058-4faf-a26e-a08fc452e6cd None admin] PATH_INFO: `/v3/domains` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:42:15.345454 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d324ab8-d058-4faf-a26e-a08fc452e6cd None admin] RBAC: Adding query filter params (name=Default) {{(pid=80143) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 10:42:15.345757 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d324ab8-d058-4faf-a26e-a08fc452e6cd None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 10:42:15.347167 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d324ab8-d058-4faf-a26e-a08fc452e6cd None admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 10:42:15.361343 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=100 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 10:42:15.381186 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 341/695] 10.222.0.25 () {56 vars in 1109 bytes} [Sun Aug 7 10:42:15 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 311 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 07 10:42:15.387528 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d324ab8-d058-4faf-a26e-a08fc452e6cd None admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 10:42:15.424741 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 355/696] 10.222.0.25 () {58 vars in 1282 bytes} [Sun Aug 7 10:42:15 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 381 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Aug 07 10:42:15.430176 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89470dd1-f9cb-4f6d-b447-5d8f49487079 None admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 10:42:15.536533 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89470dd1-f9cb-4f6d-b447-5d8f49487079 None admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 10:42:15.547019 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=100 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 10:42:15.576468 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35939c93-dd2c-4eb5-b19d-93d00eb4ae67 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 10:42:15.577349 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35939c93-dd2c-4eb5-b19d-93d00eb4ae67 None admin] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:42:15.577812 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35939c93-dd2c-4eb5-b19d-93d00eb4ae67 None admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:42:15.578184 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35939c93-dd2c-4eb5-b19d-93d00eb4ae67 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:42:15.590252 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35939c93-dd2c-4eb5-b19d-93d00eb4ae67 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 10:42:15.591949 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35939c93-dd2c-4eb5-b19d-93d00eb4ae67 None admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 10:42:15.607922 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application [None req-35939c93-dd2c-4eb5-b19d-93d00eb4ae67 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 07 10:42:15.607922 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 07 10:42:15.607922 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 07 10:42:15.607922 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 07 10:42:15.607922 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 07 10:42:15.607922 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 07 10:42:15.607922 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 07 10:42:15.607922 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 07 10:42:15.607922 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 07 10:42:15.607922 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 07 10:42:15.607922 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 07 10:42:15.607922 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 07 10:42:15.607922 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 07 10:42:15.607922 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 07 10:42:15.607922 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 07 10:42:15.607922 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 07 10:42:15.609069 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 07 10:42:15.609069 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 07 10:42:15.609069 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 07 10:42:15.609069 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 07 10:42:15.609069 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 07 10:42:15.609069 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self.get_or_create( Aug 07 10:42:15.609069 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 07 10:42:15.609069 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application with Lock( Aug 07 10:42:15.609069 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 07 10:42:15.609069 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self._enter() Aug 07 10:42:15.609069 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 07 10:42:15.609069 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 07 10:42:15.609069 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 07 10:42:15.609069 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self.creator() Aug 07 10:42:15.609069 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 07 10:42:15.609069 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application created_value = creator( Aug 07 10:42:15.609069 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 07 10:42:15.610204 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 07 10:42:15.610204 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 07 10:42:15.610204 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application Aug 07 10:42:15.611683 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 356/697] 10.222.0.25 () {56 vars in 1108 bytes} [Sun Aug 7 10:42:15 2022] GET /identity/v3/domains/Default => generated 87 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 07 10:42:15.617183 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35939c93-dd2c-4eb5-b19d-93d00eb4ae67 None admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 10:42:15.622285 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d324ab8-d058-4faf-a26e-a08fc452e6cd None admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 10:42:15.622728 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=100 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 10:42:15.695644 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75e9ffcf-bf3e-4ecc-bfa2-b0cce189114d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 10:42:15.696289 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75e9ffcf-bf3e-4ecc-bfa2-b0cce189114d None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:42:15.696569 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75e9ffcf-bf3e-4ecc-bfa2-b0cce189114d None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:42:15.696841 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75e9ffcf-bf3e-4ecc-bfa2-b0cce189114d None admin] PATH_INFO: `/v3/projects/service` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:42:15.704632 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75e9ffcf-bf3e-4ecc-bfa2-b0cce189114d None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 10:42:15.706073 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75e9ffcf-bf3e-4ecc-bfa2-b0cce189114d None admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 10:42:15.726240 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35939c93-dd2c-4eb5-b19d-93d00eb4ae67 None admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 10:42:15.734092 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application [None req-75e9ffcf-bf3e-4ecc-bfa2-b0cce189114d None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 07 10:42:15.734092 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 07 10:42:15.734092 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 07 10:42:15.734092 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 07 10:42:15.734092 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 07 10:42:15.734092 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 07 10:42:15.734092 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 07 10:42:15.734092 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 07 10:42:15.734092 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 07 10:42:15.734092 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 07 10:42:15.734092 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 07 10:42:15.734092 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 07 10:42:15.734092 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 07 10:42:15.734092 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 07 10:42:15.734092 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 07 10:42:15.735253 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 07 10:42:15.735253 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 07 10:42:15.735253 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 07 10:42:15.735253 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 07 10:42:15.735253 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 07 10:42:15.735253 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 07 10:42:15.735253 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.get_or_create( Aug 07 10:42:15.735253 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 07 10:42:15.735253 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application with Lock( Aug 07 10:42:15.735253 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 07 10:42:15.735253 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self._enter() Aug 07 10:42:15.735253 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 07 10:42:15.735253 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 07 10:42:15.735253 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 07 10:42:15.735253 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.creator() Aug 07 10:42:15.735253 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 07 10:42:15.735253 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application created_value = creator( Aug 07 10:42:15.736419 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 07 10:42:15.736419 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 07 10:42:15.736419 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 07 10:42:15.736419 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 07 10:42:15.736419 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 07 10:42:15.736419 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 07 10:42:15.736419 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 07 10:42:15.736419 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application Aug 07 10:42:15.736419 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 342/698] 10.222.0.25 () {56 vars in 1111 bytes} [Sun Aug 7 10:42:15 2022] GET /identity/v3/projects/service => generated 88 bytes in 348 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 07 10:42:15.742888 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75e9ffcf-bf3e-4ecc-bfa2-b0cce189114d None admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 10:42:15.768424 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36deca16-b604-409d-84d8-d3b360ecab2d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 10:42:15.769233 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36deca16-b604-409d-84d8-d3b360ecab2d None admin] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:42:15.769516 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36deca16-b604-409d-84d8-d3b360ecab2d None admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:42:15.769791 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36deca16-b604-409d-84d8-d3b360ecab2d None admin] PATH_INFO: `/v3/domains` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:42:15.770528 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36deca16-b604-409d-84d8-d3b360ecab2d None admin] RBAC: Adding query filter params (name=Default) {{(pid=80144) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 10:42:15.770832 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36deca16-b604-409d-84d8-d3b360ecab2d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 10:42:15.772272 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36deca16-b604-409d-84d8-d3b360ecab2d None admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 10:42:15.785616 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 357/699] 10.222.0.25 () {56 vars in 1109 bytes} [Sun Aug 7 10:42:15 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 10:42:15.787011 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-944d2402-e436-4307-8a08-b438e7eba08d None None] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:42:15.787253 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-944d2402-e436-4307-8a08-b438e7eba08d None None] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:42:15.787529 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-944d2402-e436-4307-8a08-b438e7eba08d None None] PATH_INFO: `/` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:42:15.789552 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 358/700] 10.222.0.25 () {54 vars in 927 bytes} [Sun Aug 7 10:42:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 07 10:42:15.790427 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-944d2402-e436-4307-8a08-b438e7eba08d None None] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 10:42:15.874077 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=100 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 10:42:15.883124 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75e9ffcf-bf3e-4ecc-bfa2-b0cce189114d None admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 10:42:15.918077 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-944d2402-e436-4307-8a08-b438e7eba08d None None] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 10:42:15.928172 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=100 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 10:42:15.935941 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1726828b-729f-432a-80a7-e5caf8711ad1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 10:42:15.936945 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1726828b-729f-432a-80a7-e5caf8711ad1 None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:42:15.937228 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1726828b-729f-432a-80a7-e5caf8711ad1 None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:42:15.937505 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1726828b-729f-432a-80a7-e5caf8711ad1 None admin] PATH_INFO: `/v3/projects` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:42:15.938211 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1726828b-729f-432a-80a7-e5caf8711ad1 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80143) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 10:42:15.938518 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1726828b-729f-432a-80a7-e5caf8711ad1 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 10:42:15.940605 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1726828b-729f-432a-80a7-e5caf8711ad1 None admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 10:42:15.956209 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3744dde-cc7c-4e76-b5c8-521e5311e96f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 10:42:15.956908 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3744dde-cc7c-4e76-b5c8-521e5311e96f None admin] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:42:15.957185 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3744dde-cc7c-4e76-b5c8-521e5311e96f None admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:42:15.957441 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3744dde-cc7c-4e76-b5c8-521e5311e96f None admin] PATH_INFO: `/v3/projects/service` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:42:15.958149 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 343/701] 10.222.0.25 () {56 vars in 1148 bytes} [Sun Aug 7 10:42:15 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 10:42:15.959451 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b15cf63-c2bb-401b-9f3b-388649c8a5e8 None None] REQUEST_METHOD: `POST` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:42:15.960046 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b15cf63-c2bb-401b-9f3b-388649c8a5e8 None None] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:42:15.960329 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b15cf63-c2bb-401b-9f3b-388649c8a5e8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:42:15.965902 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3744dde-cc7c-4e76-b5c8-521e5311e96f None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 10:42:15.967482 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3744dde-cc7c-4e76-b5c8-521e5311e96f None admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 10:42:15.980382 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application [None req-c3744dde-cc7c-4e76-b5c8-521e5311e96f None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 07 10:42:15.980382 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 07 10:42:15.980382 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 07 10:42:15.980382 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 07 10:42:15.980382 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 07 10:42:15.980382 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 07 10:42:15.980382 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 07 10:42:15.980382 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 07 10:42:15.980382 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 07 10:42:15.980382 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 07 10:42:15.980382 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 07 10:42:15.980382 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 07 10:42:15.980382 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 07 10:42:15.980382 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 07 10:42:15.980382 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 07 10:42:15.981297 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 07 10:42:15.981297 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 07 10:42:15.981297 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 07 10:42:15.981297 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 07 10:42:15.981297 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 07 10:42:15.981297 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 07 10:42:15.981297 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self.get_or_create( Aug 07 10:42:15.981297 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 07 10:42:15.981297 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application with Lock( Aug 07 10:42:15.981297 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 07 10:42:15.981297 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self._enter() Aug 07 10:42:15.981297 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 07 10:42:15.981297 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 07 10:42:15.981297 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 07 10:42:15.981297 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self.creator() Aug 07 10:42:15.981297 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 07 10:42:15.981297 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application created_value = creator( Aug 07 10:42:15.982196 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 07 10:42:15.982196 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 07 10:42:15.982196 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 07 10:42:15.982196 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 07 10:42:15.982196 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 07 10:42:15.982196 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 07 10:42:15.982196 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 07 10:42:15.982196 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application Aug 07 10:42:15.983351 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.auth.core [None req-2b15cf63-c2bb-401b-9f3b-388649c8a5e8 None None] MFA Rules not processed for user `bac4e6b76fe64217ac8fa44e71cffe6a`. Rule list: `[]` (Enabled: `True`). {{(pid=80143) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 10:42:15.983497 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 359/702] 10.222.0.25 () {56 vars in 1111 bytes} [Sun Aug 7 10:42:15 2022] GET /identity/v3/projects/service => generated 88 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 07 10:42:15.983646 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-2b15cf63-c2bb-401b-9f3b-388649c8a5e8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 10:42:15.983789 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3744dde-cc7c-4e76-b5c8-521e5311e96f None admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 10:42:16.064643 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 344/703] 10.222.0.25 () {58 vars in 1024 bytes} [Sun Aug 7 10:42:15 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 07 10:42:16.065551 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b15cf63-c2bb-401b-9f3b-388649c8a5e8 None None] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 10:42:16.094298 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3744dde-cc7c-4e76-b5c8-521e5311e96f None admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 10:42:16.118137 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=100 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 10:42:16.131153 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcebea91-530c-4bc0-800b-e10a541a799f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 10:42:16.131810 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcebea91-530c-4bc0-800b-e10a541a799f None admin] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:42:16.132099 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcebea91-530c-4bc0-800b-e10a541a799f None admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:42:16.132375 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcebea91-530c-4bc0-800b-e10a541a799f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:42:16.133096 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcebea91-530c-4bc0-800b-e10a541a799f None admin] RBAC: Adding query filter params (role.id=fbf375e9760340c9aa9529333eeb25f9, scope.project.id=6de1f3cac02c4bcdb05dd2337176033d, user.id=69490ea2a9a340bf91aca38bf5655b45) {{(pid=80144) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 10:42:16.133408 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcebea91-530c-4bc0-800b-e10a541a799f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 10:42:16.135059 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcebea91-530c-4bc0-800b-e10a541a799f None admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 10:42:16.140767 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 360/704] 10.222.0.25 () {56 vars in 1374 bytes} [Sun Aug 7 10:42:15 2022] GET /identity/v3/role_assignments?user.id=69490ea2a9a340bf91aca38bf5655b45&scope.project.id=6de1f3cac02c4bcdb05dd2337176033d&role.id=fbf375e9760340c9aa9529333eeb25f9 => generated 608 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 10:42:16.141934 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bae4ee83-2062-42d1-b984-67505840531d None None] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:42:16.142217 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bae4ee83-2062-42d1-b984-67505840531d None None] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:42:16.142492 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bae4ee83-2062-42d1-b984-67505840531d None None] PATH_INFO: `/` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:42:16.148065 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 361/705] 10.222.0.25 () {54 vars in 927 bytes} [Sun Aug 7 10:42:16 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 07 10:42:16.148902 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bae4ee83-2062-42d1-b984-67505840531d None None] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 10:42:16.210817 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b15cf63-c2bb-401b-9f3b-388649c8a5e8 None None] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 10:42:16.227229 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=100 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 10:42:16.252066 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d547fa8d-ac36-4154-95de-f7c713f676c7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 10:42:16.252765 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d547fa8d-ac36-4154-95de-f7c713f676c7 None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:42:16.253105 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d547fa8d-ac36-4154-95de-f7c713f676c7 None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:42:16.253364 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d547fa8d-ac36-4154-95de-f7c713f676c7 None admin] PATH_INFO: `/v3/projects` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:42:16.254079 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d547fa8d-ac36-4154-95de-f7c713f676c7 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80143) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 10:42:16.254388 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d547fa8d-ac36-4154-95de-f7c713f676c7 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 10:42:16.256254 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d547fa8d-ac36-4154-95de-f7c713f676c7 None admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 10:42:16.271130 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 345/706] 10.222.0.25 () {56 vars in 1148 bytes} [Sun Aug 7 10:42:16 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 10:42:16.272818 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9492124e-4ad3-4d3e-aa74-2a5d2491082a None None] REQUEST_METHOD: `POST` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:42:16.273278 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9492124e-4ad3-4d3e-aa74-2a5d2491082a None None] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:42:16.273735 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9492124e-4ad3-4d3e-aa74-2a5d2491082a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:42:16.277453 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bae4ee83-2062-42d1-b984-67505840531d None None] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 10:42:16.293901 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.auth.core [None req-9492124e-4ad3-4d3e-aa74-2a5d2491082a None None] MFA Rules not processed for user `bac4e6b76fe64217ac8fa44e71cffe6a`. Rule list: `[]` (Enabled: `True`). {{(pid=80143) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 10:42:16.295464 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-9492124e-4ad3-4d3e-aa74-2a5d2491082a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 10:42:16.319254 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3d652e9-b32b-4edb-bd84-419cf5f6df7c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 10:42:16.319756 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3d652e9-b32b-4edb-bd84-419cf5f6df7c None admin] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:42:16.320032 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3d652e9-b32b-4edb-bd84-419cf5f6df7c None admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:42:16.320379 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3d652e9-b32b-4edb-bd84-419cf5f6df7c None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:42:16.321032 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3d652e9-b32b-4edb-bd84-419cf5f6df7c None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 10:42:16.322580 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3d652e9-b32b-4edb-bd84-419cf5f6df7c None admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 10:42:16.334164 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application [None req-d3d652e9-b32b-4edb-bd84-419cf5f6df7c None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Aug 07 10:42:16.334164 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 07 10:42:16.334164 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 07 10:42:16.334164 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 07 10:42:16.334164 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 07 10:42:16.334164 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 07 10:42:16.334164 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 07 10:42:16.334164 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 07 10:42:16.334164 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Aug 07 10:42:16.334164 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application Aug 07 10:42:16.334164 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 07 10:42:16.334164 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application Aug 07 10:42:16.334164 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 07 10:42:16.334164 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 07 10:42:16.334164 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 07 10:42:16.334164 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 07 10:42:16.334164 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 07 10:42:16.335259 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 07 10:42:16.335259 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 07 10:42:16.335259 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 07 10:42:16.335259 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 07 10:42:16.335259 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 07 10:42:16.335259 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 07 10:42:16.335259 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 07 10:42:16.335259 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 07 10:42:16.335259 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 07 10:42:16.335259 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 07 10:42:16.335259 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 07 10:42:16.335259 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 07 10:42:16.335259 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 07 10:42:16.335259 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 07 10:42:16.335259 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 07 10:42:16.335259 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self.get_or_create( Aug 07 10:42:16.336104 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 07 10:42:16.336104 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application with Lock( Aug 07 10:42:16.336104 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 07 10:42:16.336104 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self._enter() Aug 07 10:42:16.336104 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 07 10:42:16.336104 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 07 10:42:16.336104 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 07 10:42:16.336104 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self.creator() Aug 07 10:42:16.336104 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 07 10:42:16.336104 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application created_value = creator( Aug 07 10:42:16.336104 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 07 10:42:16.336104 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 07 10:42:16.336104 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Aug 07 10:42:16.336104 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application Aug 07 10:42:16.336841 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 362/707] 10.222.0.25 () {56 vars in 1129 bytes} [Sun Aug 7 10:42:16 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 07 10:42:16.337341 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3d652e9-b32b-4edb-bd84-419cf5f6df7c None admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 10:42:16.387526 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 346/708] 10.222.0.25 () {58 vars in 1024 bytes} [Sun Aug 7 10:42:16 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 07 10:42:16.388486 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9492124e-4ad3-4d3e-aa74-2a5d2491082a None None] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 10:42:16.402635 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=100 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 10:42:16.474549 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3d652e9-b32b-4edb-bd84-419cf5f6df7c None admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 10:42:16.521183 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9492124e-4ad3-4d3e-aa74-2a5d2491082a None None] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 10:42:16.521517 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a05cca3c-749e-473b-be0d-cc78ef064865 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 10:42:16.522239 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a05cca3c-749e-473b-be0d-cc78ef064865 None admin] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:42:16.522525 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a05cca3c-749e-473b-be0d-cc78ef064865 None admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:42:16.522801 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a05cca3c-749e-473b-be0d-cc78ef064865 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:42:16.523602 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a05cca3c-749e-473b-be0d-cc78ef064865 None admin] RBAC: Adding query filter params (role.id=81b1f1bcee2a473088fbbd327c1c5b23, scope.project.id=6de1f3cac02c4bcdb05dd2337176033d, user.id=31d16d1b96104b4eb690398432b9829e) {{(pid=80144) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 10:42:16.524016 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a05cca3c-749e-473b-be0d-cc78ef064865 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 10:42:16.525838 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=100 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 10:42:16.526201 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a05cca3c-749e-473b-be0d-cc78ef064865 None admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 10:42:16.533210 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 363/709] 10.222.0.25 () {56 vars in 1374 bytes} [Sun Aug 7 10:42:16 2022] GET /identity/v3/role_assignments?user.id=31d16d1b96104b4eb690398432b9829e&scope.project.id=6de1f3cac02c4bcdb05dd2337176033d&role.id=81b1f1bcee2a473088fbbd327c1c5b23 => generated 259 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 10:42:16.534614 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a05cca3c-749e-473b-be0d-cc78ef064865 None admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 10:42:16.567085 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc96d5fd-3c2f-4c8d-adb7-ee95bd481058 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 10:42:16.568124 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc96d5fd-3c2f-4c8d-adb7-ee95bd481058 None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:42:16.569341 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc96d5fd-3c2f-4c8d-adb7-ee95bd481058 None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:42:16.569835 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc96d5fd-3c2f-4c8d-adb7-ee95bd481058 None admin] PATH_INFO: `/v3/services` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:42:16.570951 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc96d5fd-3c2f-4c8d-adb7-ee95bd481058 None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=80143) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 10:42:16.571457 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc96d5fd-3c2f-4c8d-adb7-ee95bd481058 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 10:42:16.573309 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc96d5fd-3c2f-4c8d-adb7-ee95bd481058 None admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 10:42:16.584375 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 347/710] 10.222.0.25 () {56 vars in 1124 bytes} [Sun Aug 7 10:42:16 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 10:42:16.586474 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc96d5fd-3c2f-4c8d-adb7-ee95bd481058 None admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 10:42:16.711590 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc96d5fd-3c2f-4c8d-adb7-ee95bd481058 None admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 10:42:16.748050 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a05cca3c-749e-473b-be0d-cc78ef064865 None admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 10:42:16.754711 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a9f53f7-d7e2-46ac-b34d-021ead6781a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 10:42:16.755606 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a9f53f7-d7e2-46ac-b34d-021ead6781a0 None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:42:16.757050 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a9f53f7-d7e2-46ac-b34d-021ead6781a0 None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:42:16.757487 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a9f53f7-d7e2-46ac-b34d-021ead6781a0 None admin] PATH_INFO: `/v3/services` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:42:16.758269 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a9f53f7-d7e2-46ac-b34d-021ead6781a0 None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=80143) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 10:42:16.758713 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a9f53f7-d7e2-46ac-b34d-021ead6781a0 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 10:42:16.760995 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a9f53f7-d7e2-46ac-b34d-021ead6781a0 None admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 10:42:16.775879 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 348/711] 10.222.0.25 () {56 vars in 1124 bytes} [Sun Aug 7 10:42:16 2022] GET /identity/v3/services?type=block-storage => generated 130 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 07 10:42:16.795622 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=100 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 10:42:16.834713 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f51760c-e675-477a-9bab-2eef2e2cd6f7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 10:42:16.835316 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f51760c-e675-477a-9bab-2eef2e2cd6f7 None admin] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:42:16.835602 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f51760c-e675-477a-9bab-2eef2e2cd6f7 None admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:42:16.835874 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f51760c-e675-477a-9bab-2eef2e2cd6f7 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:42:16.856118 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f51760c-e675-477a-9bab-2eef2e2cd6f7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 10:42:16.857646 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f51760c-e675-477a-9bab-2eef2e2cd6f7 None admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 10:42:16.878675 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9de3cfe1-bb55-48ad-9fb6-6a482db2e49c None None] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:42:16.879021 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application [None req-3f51760c-e675-477a-9bab-2eef2e2cd6f7 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 07 10:42:16.879021 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 07 10:42:16.879021 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 07 10:42:16.879021 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 07 10:42:16.879021 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 07 10:42:16.879021 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 07 10:42:16.879021 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 07 10:42:16.879021 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 07 10:42:16.879021 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 07 10:42:16.879021 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 07 10:42:16.879021 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 07 10:42:16.879021 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 07 10:42:16.879021 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 07 10:42:16.879021 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 07 10:42:16.879021 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 07 10:42:16.879021 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 07 10:42:16.881091 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 07 10:42:16.881091 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 07 10:42:16.881091 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 07 10:42:16.881091 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 07 10:42:16.881091 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 07 10:42:16.881091 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self.get_or_create( Aug 07 10:42:16.881091 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 07 10:42:16.881091 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application with Lock( Aug 07 10:42:16.881091 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 07 10:42:16.881091 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self._enter() Aug 07 10:42:16.881091 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 07 10:42:16.881091 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 07 10:42:16.881091 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 07 10:42:16.881091 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self.creator() Aug 07 10:42:16.881091 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 07 10:42:16.881091 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application created_value = creator( Aug 07 10:42:16.881091 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 07 10:42:16.883223 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 07 10:42:16.883223 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 07 10:42:16.883223 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application Aug 07 10:42:16.883691 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9de3cfe1-bb55-48ad-9fb6-6a482db2e49c None None] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:42:16.883691 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9de3cfe1-bb55-48ad-9fb6-6a482db2e49c None None] PATH_INFO: `/` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:42:16.883960 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 364/713] 10.222.0.25 () {56 vars in 1108 bytes} [Sun Aug 7 10:42:16 2022] GET /identity/v3/domains/Default => generated 87 bytes in 347 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 07 10:42:16.884107 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 349/713] 10.222.0.25 () {54 vars in 927 bytes} [Sun Aug 7 10:42:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 07 10:42:16.892439 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f51760c-e675-477a-9bab-2eef2e2cd6f7 None admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 10:42:16.892754 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d7c2c07-0cdf-4faa-9a85-e837ac0722f8 None None] REQUEST_METHOD: `POST` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:42:16.892754 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d7c2c07-0cdf-4faa-9a85-e837ac0722f8 None None] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:42:16.893062 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d7c2c07-0cdf-4faa-9a85-e837ac0722f8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:42:16.921807 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.auth.core [None req-1d7c2c07-0cdf-4faa-9a85-e837ac0722f8 None None] MFA Rules not processed for user `bac4e6b76fe64217ac8fa44e71cffe6a`. Rule list: `[]` (Enabled: `True`). {{(pid=80143) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 10:42:16.923909 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-1d7c2c07-0cdf-4faa-9a85-e837ac0722f8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 10:42:16.961702 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=100 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 10:42:17.039111 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f51760c-e675-477a-9bab-2eef2e2cd6f7 None admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 10:42:17.088714 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e112c2df-ca11-4be2-9b7e-ee7087028b69 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 10:42:17.089316 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e112c2df-ca11-4be2-9b7e-ee7087028b69 None admin] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:42:17.089601 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e112c2df-ca11-4be2-9b7e-ee7087028b69 None admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:42:17.089871 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e112c2df-ca11-4be2-9b7e-ee7087028b69 None admin] PATH_INFO: `/v3/domains` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:42:17.090536 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e112c2df-ca11-4be2-9b7e-ee7087028b69 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80144) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 10:42:17.090837 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e112c2df-ca11-4be2-9b7e-ee7087028b69 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 10:42:17.092243 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e112c2df-ca11-4be2-9b7e-ee7087028b69 None admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 10:42:17.112186 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 365/714] 10.222.0.25 () {56 vars in 1109 bytes} [Sun Aug 7 10:42:16 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 220 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 10:42:17.121130 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e112c2df-ca11-4be2-9b7e-ee7087028b69 None admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 10:42:17.144120 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 350/715] 10.222.0.25 () {58 vars in 1024 bytes} [Sun Aug 7 10:42:16 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 252 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 07 10:42:17.162326 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=100 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 10:42:17.178433 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d7c2c07-0cdf-4faa-9a85-e837ac0722f8 None None] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 10:42:17.288764 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e112c2df-ca11-4be2-9b7e-ee7087028b69 None admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 10:42:17.327100 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3693f6e-6876-4344-9303-c4212cce6280 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 10:42:17.327728 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3693f6e-6876-4344-9303-c4212cce6280 None admin] REQUEST_METHOD: `POST` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:42:17.328521 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3693f6e-6876-4344-9303-c4212cce6280 None admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:42:17.328802 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3693f6e-6876-4344-9303-c4212cce6280 None admin] PATH_INFO: `/v3/users` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:42:17.329596 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3693f6e-6876-4344-9303-c4212cce6280 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 10:42:17.331036 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3693f6e-6876-4344-9303-c4212cce6280 None admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 10:42:17.361464 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 366/716] 10.222.0.25 () {60 vars in 1132 bytes} [Sun Aug 7 10:42:17 2022] POST /identity/v3/users => generated 254 bytes in 236 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 07 10:42:17.399239 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d7c2c07-0cdf-4faa-9a85-e837ac0722f8 None None] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 10:42:17.461617 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfe5619f-ff7c-4261-ae09-3738de171749 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 10:42:17.462825 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfe5619f-ff7c-4261-ae09-3738de171749 None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:42:17.463343 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfe5619f-ff7c-4261-ae09-3738de171749 None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:42:17.463821 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfe5619f-ff7c-4261-ae09-3738de171749 None admin] PATH_INFO: `/v3/services/network` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:42:17.464709 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfe5619f-ff7c-4261-ae09-3738de171749 None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 10:42:17.466332 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfe5619f-ff7c-4261-ae09-3738de171749 None admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 10:42:17.482045 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application [None req-cfe5619f-ff7c-4261-ae09-3738de171749 None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Aug 07 10:42:17.482045 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 07 10:42:17.482045 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 07 10:42:17.482045 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 07 10:42:17.482045 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 07 10:42:17.482045 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 07 10:42:17.482045 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 07 10:42:17.482045 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 07 10:42:17.482045 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Aug 07 10:42:17.482045 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application Aug 07 10:42:17.482045 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 07 10:42:17.482045 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application Aug 07 10:42:17.482045 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 07 10:42:17.482045 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 07 10:42:17.482045 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 07 10:42:17.482045 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 07 10:42:17.482045 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 07 10:42:17.483150 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 07 10:42:17.483150 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 07 10:42:17.483150 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 07 10:42:17.483150 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 07 10:42:17.483150 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 07 10:42:17.483150 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 07 10:42:17.483150 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 07 10:42:17.483150 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 07 10:42:17.483150 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 07 10:42:17.483150 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 07 10:42:17.483150 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 07 10:42:17.483150 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 07 10:42:17.483150 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 07 10:42:17.483150 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 07 10:42:17.483150 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 07 10:42:17.483150 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.get_or_create( Aug 07 10:42:17.484014 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 07 10:42:17.484014 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application with Lock( Aug 07 10:42:17.484014 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 07 10:42:17.484014 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self._enter() Aug 07 10:42:17.484014 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 07 10:42:17.484014 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 07 10:42:17.484014 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 07 10:42:17.484014 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.creator() Aug 07 10:42:17.484014 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 07 10:42:17.484014 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application created_value = creator( Aug 07 10:42:17.484014 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 07 10:42:17.484014 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 07 10:42:17.484014 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Aug 07 10:42:17.484014 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application Aug 07 10:42:17.486974 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 351/717] 10.222.0.25 () {56 vars in 1111 bytes} [Sun Aug 7 10:42:17 2022] GET /identity/v3/services/network => generated 88 bytes in 309 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 07 10:42:17.495299 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3693f6e-6876-4344-9303-c4212cce6280 None admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 10:42:17.620340 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 10:42:17.644275 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3693f6e-6876-4344-9303-c4212cce6280 None admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 10:42:17.681138 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4781b84c-9efc-4afe-bd81-09df3de6568f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 10:42:17.681929 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4781b84c-9efc-4afe-bd81-09df3de6568f None admin] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:42:17.682428 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4781b84c-9efc-4afe-bd81-09df3de6568f None admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:42:17.682721 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4781b84c-9efc-4afe-bd81-09df3de6568f None admin] PATH_INFO: `/v3/services` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:42:17.683385 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4781b84c-9efc-4afe-bd81-09df3de6568f None admin] RBAC: Adding query filter params (name=network) {{(pid=80144) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 10:42:17.683692 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4781b84c-9efc-4afe-bd81-09df3de6568f None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 10:42:17.685046 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4781b84c-9efc-4afe-bd81-09df3de6568f None admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 10:42:17.693667 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 367/718] 10.222.0.25 () {56 vars in 1112 bytes} [Sun Aug 7 10:42:17 2022] GET /identity/v3/services?name=network => generated 124 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 10:42:17.700162 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfe5619f-ff7c-4261-ae09-3738de171749 None admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 10:42:17.737913 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=100 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 10:42:17.814270 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfe5619f-ff7c-4261-ae09-3738de171749 None admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 10:42:17.869446 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12aa84f9-0920-4311-8152-636ff8cc294e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 10:42:17.870125 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12aa84f9-0920-4311-8152-636ff8cc294e None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:42:17.870560 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12aa84f9-0920-4311-8152-636ff8cc294e None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:42:17.870781 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12aa84f9-0920-4311-8152-636ff8cc294e None admin] PATH_INFO: `/v3/services` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:42:17.871435 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12aa84f9-0920-4311-8152-636ff8cc294e None admin] RBAC: Adding query filter params (type=network) {{(pid=80143) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 10:42:17.871733 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12aa84f9-0920-4311-8152-636ff8cc294e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 10:42:17.873088 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12aa84f9-0920-4311-8152-636ff8cc294e None admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 10:42:17.883438 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 352/719] 10.222.0.25 () {56 vars in 1112 bytes} [Sun Aug 7 10:42:17 2022] GET /identity/v3/services?type=network => generated 124 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 10:42:18.588865 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-604ac4c5-920b-407a-a75b-3b153a144132 None None] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:42:18.589397 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-604ac4c5-920b-407a-a75b-3b153a144132 None None] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:42:18.589397 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-604ac4c5-920b-407a-a75b-3b153a144132 None None] PATH_INFO: `/` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:42:18.591075 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 368/720] 10.222.0.25 () {54 vars in 927 bytes} [Sun Aug 7 10:42:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 07 10:42:18.596531 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1e01cda-34b6-4c43-85e7-72f24bc9ec6d None None] REQUEST_METHOD: `POST` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:42:18.596936 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1e01cda-34b6-4c43-85e7-72f24bc9ec6d None None] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:42:18.597131 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1e01cda-34b6-4c43-85e7-72f24bc9ec6d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:42:18.619640 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.auth.core [None req-c1e01cda-34b6-4c43-85e7-72f24bc9ec6d None None] MFA Rules not processed for user `bac4e6b76fe64217ac8fa44e71cffe6a`. Rule list: `[]` (Enabled: `True`). {{(pid=80143) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 10:42:18.620827 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-c1e01cda-34b6-4c43-85e7-72f24bc9ec6d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 10:42:18.704553 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 353/721] 10.222.0.25 () {58 vars in 1024 bytes} [Sun Aug 7 10:42:18 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 07 10:42:18.741400 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-604ac4c5-920b-407a-a75b-3b153a144132 None None] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 10:42:18.875756 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-604ac4c5-920b-407a-a75b-3b153a144132 None None] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 10:42:18.925465 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d25d8409-386f-45ec-831f-92e4dae4fca1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 10:42:18.926101 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d25d8409-386f-45ec-831f-92e4dae4fca1 None admin] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:42:18.926382 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d25d8409-386f-45ec-831f-92e4dae4fca1 None admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:42:18.926652 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d25d8409-386f-45ec-831f-92e4dae4fca1 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:42:18.942680 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d25d8409-386f-45ec-831f-92e4dae4fca1 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 07 10:42:18.947952 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application [None req-d25d8409-386f-45ec-831f-92e4dae4fca1 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 07 10:42:18.947952 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 07 10:42:18.947952 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 07 10:42:18.947952 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 07 10:42:18.947952 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 07 10:42:18.947952 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 07 10:42:18.947952 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 07 10:42:18.947952 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 07 10:42:18.947952 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 07 10:42:18.947952 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 07 10:42:18.947952 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 07 10:42:18.947952 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 07 10:42:18.947952 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 07 10:42:18.947952 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 07 10:42:18.947952 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 07 10:42:18.947952 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 07 10:42:18.948890 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 07 10:42:18.948890 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 07 10:42:18.948890 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 07 10:42:18.948890 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 07 10:42:18.948890 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 07 10:42:18.948890 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 07 10:42:18.948890 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 07 10:42:18.948890 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 07 10:42:18.948890 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 07 10:42:18.948890 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self.get_or_create( Aug 07 10:42:18.948890 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 07 10:42:18.948890 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application with Lock( Aug 07 10:42:18.948890 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 07 10:42:18.948890 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self._enter() Aug 07 10:42:18.948890 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 07 10:42:18.948890 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 07 10:42:18.949757 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 07 10:42:18.949757 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self.creator() Aug 07 10:42:18.949757 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 07 10:42:18.949757 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application created_value = creator( Aug 07 10:42:18.949757 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 07 10:42:18.949757 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 07 10:42:18.949757 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 07 10:42:18.949757 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 07 10:42:18.949757 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 07 10:42:18.949757 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 07 10:42:18.949757 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 07 10:42:18.949757 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application Aug 07 10:42:18.949757 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 369/722] 10.222.0.25 () {56 vars in 1096 bytes} [Sun Aug 7 10:42:18 2022] GET /identity/v3/roles/admin => generated 83 bytes in 208 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 07 10:42:18.955274 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1e01cda-34b6-4c43-85e7-72f24bc9ec6d None None] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 10:42:18.980719 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=100 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 10:42:19.091563 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1e01cda-34b6-4c43-85e7-72f24bc9ec6d None None] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 10:42:19.095491 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4392bc05-8546-48d3-ba8c-2a7ead1b018f None None] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:42:19.095781 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4392bc05-8546-48d3-ba8c-2a7ead1b018f None None] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:42:19.096058 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4392bc05-8546-48d3-ba8c-2a7ead1b018f None None] PATH_INFO: `/` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:42:19.097642 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 370/723] 10.222.0.25 () {54 vars in 927 bytes} [Sun Aug 7 10:42:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 07 10:42:19.104815 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3567391a-3a45-4bbf-965f-e08ceaab5c08 None None] REQUEST_METHOD: `POST` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:42:19.105098 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3567391a-3a45-4bbf-965f-e08ceaab5c08 None None] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:42:19.105355 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3567391a-3a45-4bbf-965f-e08ceaab5c08 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:42:19.125143 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=100 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 10:42:19.139969 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.auth.core [None req-3567391a-3a45-4bbf-965f-e08ceaab5c08 None None] MFA Rules not processed for user `bac4e6b76fe64217ac8fa44e71cffe6a`. Rule list: `[]` (Enabled: `True`). {{(pid=80144) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 10:42:19.143438 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.fernet_utils [None req-3567391a-3a45-4bbf-965f-e08ceaab5c08 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80144) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 10:42:19.160696 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02bde0d3-f305-431e-9161-9f52ca44160f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 10:42:19.161315 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02bde0d3-f305-431e-9161-9f52ca44160f None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:42:19.161590 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02bde0d3-f305-431e-9161-9f52ca44160f None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:42:19.161864 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02bde0d3-f305-431e-9161-9f52ca44160f None admin] PATH_INFO: `/v3/roles` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:42:19.162529 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02bde0d3-f305-431e-9161-9f52ca44160f None admin] RBAC: Adding query filter params (name=admin) {{(pid=80143) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 10:42:19.162827 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02bde0d3-f305-431e-9161-9f52ca44160f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 10:42:19.164216 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02bde0d3-f305-431e-9161-9f52ca44160f None admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 10:42:19.187281 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 354/724] 10.222.0.25 () {56 vars in 1099 bytes} [Sun Aug 7 10:42:18 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 232 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 10:42:19.193616 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02bde0d3-f305-431e-9161-9f52ca44160f None admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 10:42:19.284493 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 371/725] 10.222.0.25 () {58 vars in 1024 bytes} [Sun Aug 7 10:42:19 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 181 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 07 10:42:19.325973 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3567391a-3a45-4bbf-965f-e08ceaab5c08 None None] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 10:42:19.345336 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=100 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 10:42:19.384095 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02bde0d3-f305-431e-9161-9f52ca44160f None admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 10:42:19.435743 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaad7367-182a-4101-b51f-3e19a5b30d9d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 10:42:19.436540 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaad7367-182a-4101-b51f-3e19a5b30d9d None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:42:19.436821 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaad7367-182a-4101-b51f-3e19a5b30d9d None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:42:19.437090 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaad7367-182a-4101-b51f-3e19a5b30d9d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:42:19.452796 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaad7367-182a-4101-b51f-3e19a5b30d9d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 10:42:19.454724 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaad7367-182a-4101-b51f-3e19a5b30d9d None admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 10:42:19.470376 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application [None req-eaad7367-182a-4101-b51f-3e19a5b30d9d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 07 10:42:19.470376 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 07 10:42:19.470376 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 07 10:42:19.470376 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 07 10:42:19.470376 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 07 10:42:19.470376 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 07 10:42:19.470376 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 07 10:42:19.470376 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 07 10:42:19.470376 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 07 10:42:19.470376 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 07 10:42:19.470376 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 07 10:42:19.470376 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 07 10:42:19.470376 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 07 10:42:19.470376 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 07 10:42:19.470376 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 07 10:42:19.470376 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 07 10:42:19.471460 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 07 10:42:19.471460 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 07 10:42:19.471460 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 07 10:42:19.471460 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 07 10:42:19.471460 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 07 10:42:19.471460 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.get_or_create( Aug 07 10:42:19.471460 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 07 10:42:19.471460 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application with Lock( Aug 07 10:42:19.471460 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 07 10:42:19.471460 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self._enter() Aug 07 10:42:19.471460 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 07 10:42:19.471460 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 07 10:42:19.471460 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 07 10:42:19.471460 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.creator() Aug 07 10:42:19.471460 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 07 10:42:19.471460 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application created_value = creator( Aug 07 10:42:19.471460 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 07 10:42:19.472361 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 07 10:42:19.472361 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 07 10:42:19.472361 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application Aug 07 10:42:19.474460 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 355/726] 10.222.0.25 () {56 vars in 1108 bytes} [Sun Aug 7 10:42:19 2022] GET /identity/v3/domains/Default => generated 87 bytes in 281 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 07 10:42:19.475895 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cfeaf42-5b93-4b6c-ab32-c17559cc7146 None None] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:42:19.476382 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cfeaf42-5b93-4b6c-ab32-c17559cc7146 None None] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:42:19.476847 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cfeaf42-5b93-4b6c-ab32-c17559cc7146 None None] PATH_INFO: `/` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:42:19.479274 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 356/727] 10.222.0.25 () {54 vars in 927 bytes} [Sun Aug 7 10:42:19 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 07 10:42:19.480249 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cfeaf42-5b93-4b6c-ab32-c17559cc7146 None None] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 10:42:19.490976 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3567391a-3a45-4bbf-965f-e08ceaab5c08 None None] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 10:42:19.537775 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7be77683-3776-4d71-80fc-7455422c1700 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 10:42:19.538443 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7be77683-3776-4d71-80fc-7455422c1700 None admin] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:42:19.538796 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7be77683-3776-4d71-80fc-7455422c1700 None admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:42:19.539180 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7be77683-3776-4d71-80fc-7455422c1700 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:42:19.543435 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=100 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 10:42:19.554641 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7be77683-3776-4d71-80fc-7455422c1700 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 07 10:42:19.560744 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application [None req-7be77683-3776-4d71-80fc-7455422c1700 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 07 10:42:19.560744 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 07 10:42:19.560744 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 07 10:42:19.560744 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 07 10:42:19.560744 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 07 10:42:19.560744 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 07 10:42:19.560744 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 07 10:42:19.560744 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 07 10:42:19.560744 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 07 10:42:19.560744 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 07 10:42:19.560744 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 07 10:42:19.560744 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 07 10:42:19.560744 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 07 10:42:19.560744 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 07 10:42:19.560744 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 07 10:42:19.560744 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 07 10:42:19.561743 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 07 10:42:19.561743 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 07 10:42:19.561743 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 07 10:42:19.561743 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 07 10:42:19.561743 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 07 10:42:19.561743 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 07 10:42:19.561743 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 07 10:42:19.561743 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 07 10:42:19.561743 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 07 10:42:19.561743 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self.get_or_create( Aug 07 10:42:19.561743 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 07 10:42:19.561743 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application with Lock( Aug 07 10:42:19.561743 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 07 10:42:19.561743 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self._enter() Aug 07 10:42:19.561743 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 07 10:42:19.561743 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 07 10:42:19.562629 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 07 10:42:19.562629 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self.creator() Aug 07 10:42:19.562629 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 07 10:42:19.562629 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application created_value = creator( Aug 07 10:42:19.562629 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 07 10:42:19.562629 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 07 10:42:19.562629 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 07 10:42:19.562629 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 07 10:42:19.562629 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 07 10:42:19.562629 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 07 10:42:19.562629 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 07 10:42:19.562629 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application Aug 07 10:42:19.562629 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 372/728] 10.222.0.25 () {56 vars in 1096 bytes} [Sun Aug 7 10:42:19 2022] GET /identity/v3/roles/admin => generated 83 bytes in 237 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 07 10:42:19.563383 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2de24f9-5329-4fe9-b631-09daee39d30e None None] REQUEST_METHOD: `POST` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:42:19.564193 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2de24f9-5329-4fe9-b631-09daee39d30e None None] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:42:19.564543 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2de24f9-5329-4fe9-b631-09daee39d30e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:42:19.584674 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.auth.core [None req-a2de24f9-5329-4fe9-b631-09daee39d30e None None] MFA Rules not processed for user `bac4e6b76fe64217ac8fa44e71cffe6a`. Rule list: `[]` (Enabled: `True`). {{(pid=80144) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 10:42:19.586189 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.fernet_utils [None req-a2de24f9-5329-4fe9-b631-09daee39d30e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80144) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 10:42:19.606819 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cfeaf42-5b93-4b6c-ab32-c17559cc7146 None None] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 10:42:19.652510 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ece1926-34c0-48b4-afde-0747b81f12c7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 10:42:19.653219 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ece1926-34c0-48b4-afde-0747b81f12c7 None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:42:19.653896 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ece1926-34c0-48b4-afde-0747b81f12c7 None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:42:19.654251 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ece1926-34c0-48b4-afde-0747b81f12c7 None admin] PATH_INFO: `/v3/domains` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:42:19.655268 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ece1926-34c0-48b4-afde-0747b81f12c7 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80143) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 10:42:19.655648 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ece1926-34c0-48b4-afde-0747b81f12c7 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 10:42:19.657089 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ece1926-34c0-48b4-afde-0747b81f12c7 None admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 10:42:19.658620 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=100 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 10:42:19.660017 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 373/729] 10.222.0.25 () {58 vars in 1024 bytes} [Sun Aug 7 10:42:19 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 07 10:42:19.662730 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2de24f9-5329-4fe9-b631-09daee39d30e None None] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 10:42:19.686375 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 357/730] 10.222.0.25 () {56 vars in 1109 bytes} [Sun Aug 7 10:42:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 10:42:19.692125 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ece1926-34c0-48b4-afde-0747b81f12c7 None admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 10:42:19.788016 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2de24f9-5329-4fe9-b631-09daee39d30e None None] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 10:42:19.814957 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ece1926-34c0-48b4-afde-0747b81f12c7 None admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 10:42:19.817394 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=100 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 10:42:19.828602 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9223f45-07b4-46cb-bc76-96baa168352d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 10:42:19.829526 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9223f45-07b4-46cb-bc76-96baa168352d None admin] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:42:19.830030 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9223f45-07b4-46cb-bc76-96baa168352d None admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:42:19.830625 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9223f45-07b4-46cb-bc76-96baa168352d None admin] PATH_INFO: `/v3/roles` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:42:19.831649 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9223f45-07b4-46cb-bc76-96baa168352d None admin] RBAC: Adding query filter params (name=admin) {{(pid=80144) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 10:42:19.832161 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9223f45-07b4-46cb-bc76-96baa168352d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 10:42:19.833729 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9223f45-07b4-46cb-bc76-96baa168352d None admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 10:42:19.843475 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 374/731] 10.222.0.25 () {56 vars in 1099 bytes} [Sun Aug 7 10:42:19 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 10:42:19.844687 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9223f45-07b4-46cb-bc76-96baa168352d None admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 10:42:19.854897 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-879d0698-ee81-4802-bc29-60eb9095abfe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 10:42:19.855202 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-879d0698-ee81-4802-bc29-60eb9095abfe None admin] REQUEST_METHOD: `POST` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:42:19.855529 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-879d0698-ee81-4802-bc29-60eb9095abfe None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:42:19.855811 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-879d0698-ee81-4802-bc29-60eb9095abfe None admin] PATH_INFO: `/v3/services` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:42:19.856523 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-879d0698-ee81-4802-bc29-60eb9095abfe None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 10:42:19.857994 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-879d0698-ee81-4802-bc29-60eb9095abfe None admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 10:42:19.886952 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 358/732] 10.222.0.25 () {60 vars in 1141 bytes} [Sun Aug 7 10:42:19 2022] POST /identity/v3/services => generated 251 bytes in 187 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 07 10:42:19.886952 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-879d0698-ee81-4802-bc29-60eb9095abfe None admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 10:42:19.952801 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 10:42:20.023188 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-879d0698-ee81-4802-bc29-60eb9095abfe None admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 10:42:20.041866 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9223f45-07b4-46cb-bc76-96baa168352d None admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 10:42:20.075263 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c448cf4-ea63-4075-9791-e0f84748ad85 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 10:42:20.076207 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c448cf4-ea63-4075-9791-e0f84748ad85 None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:42:20.076888 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c448cf4-ea63-4075-9791-e0f84748ad85 None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:42:20.077318 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c448cf4-ea63-4075-9791-e0f84748ad85 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:42:20.082066 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4020c32-d44f-46db-a583-4133ec247f87 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 10:42:20.082735 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4020c32-d44f-46db-a583-4133ec247f87 None admin] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:42:20.083047 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4020c32-d44f-46db-a583-4133ec247f87 None admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:42:20.083323 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4020c32-d44f-46db-a583-4133ec247f87 None admin] PATH_INFO: `/v3/users/swift` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:42:20.092539 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4020c32-d44f-46db-a583-4133ec247f87 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 10:42:20.093736 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c448cf4-ea63-4075-9791-e0f84748ad85 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 10:42:20.094762 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4020c32-d44f-46db-a583-4133ec247f87 None admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 10:42:20.095222 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c448cf4-ea63-4075-9791-e0f84748ad85 None admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 10:42:20.107501 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application [None req-c4020c32-d44f-46db-a583-4133ec247f87 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Aug 07 10:42:20.107501 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 07 10:42:20.107501 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 07 10:42:20.107501 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 07 10:42:20.107501 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 07 10:42:20.107501 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 07 10:42:20.107501 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 07 10:42:20.107501 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 07 10:42:20.107501 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 07 10:42:20.107501 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 07 10:42:20.107501 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 07 10:42:20.107501 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 07 10:42:20.107501 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 07 10:42:20.107501 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 07 10:42:20.107501 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 07 10:42:20.107501 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 07 10:42:20.108449 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 07 10:42:20.108449 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 07 10:42:20.108449 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 07 10:42:20.108449 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 07 10:42:20.108449 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 07 10:42:20.108449 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 07 10:42:20.108449 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 07 10:42:20.108449 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 07 10:42:20.108449 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 07 10:42:20.108449 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self.get_or_create( Aug 07 10:42:20.108449 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 07 10:42:20.108449 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application with Lock( Aug 07 10:42:20.108449 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 07 10:42:20.108449 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self._enter() Aug 07 10:42:20.108449 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 07 10:42:20.108449 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 07 10:42:20.108449 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 07 10:42:20.108449 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self.creator() Aug 07 10:42:20.109412 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 07 10:42:20.109412 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application created_value = creator( Aug 07 10:42:20.109412 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 07 10:42:20.109412 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 07 10:42:20.109412 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 07 10:42:20.109412 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 07 10:42:20.109412 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 07 10:42:20.109412 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 07 10:42:20.109412 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Aug 07 10:42:20.109412 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application Aug 07 10:42:20.109412 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 375/733] 10.222.0.25 () {56 vars in 1096 bytes} [Sun Aug 7 10:42:19 2022] GET /identity/v3/users/swift => generated 83 bytes in 264 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 07 10:42:20.110502 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e23a2b2-4ab8-4844-8b52-0d40b3cd213f None None] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:42:20.110502 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e23a2b2-4ab8-4844-8b52-0d40b3cd213f None None] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:42:20.110852 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e23a2b2-4ab8-4844-8b52-0d40b3cd213f None None] PATH_INFO: `/` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:42:20.112671 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application [None req-6c448cf4-ea63-4075-9791-e0f84748ad85 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 07 10:42:20.112671 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 07 10:42:20.112671 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 07 10:42:20.112671 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 07 10:42:20.112671 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 07 10:42:20.112671 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 07 10:42:20.112671 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 07 10:42:20.112671 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 07 10:42:20.112671 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 07 10:42:20.112671 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 07 10:42:20.112671 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 07 10:42:20.112671 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 07 10:42:20.112671 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 07 10:42:20.112671 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 07 10:42:20.112671 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 07 10:42:20.112671 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 07 10:42:20.114754 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 07 10:42:20.114754 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 07 10:42:20.114754 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 07 10:42:20.114754 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 07 10:42:20.114754 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 07 10:42:20.114754 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.get_or_create( Aug 07 10:42:20.114754 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 07 10:42:20.114754 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application with Lock( Aug 07 10:42:20.114754 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 07 10:42:20.114754 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self._enter() Aug 07 10:42:20.114754 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 07 10:42:20.114754 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 07 10:42:20.114754 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 07 10:42:20.114754 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.creator() Aug 07 10:42:20.114754 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 07 10:42:20.114754 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application created_value = creator( Aug 07 10:42:20.114754 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 07 10:42:20.117033 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 07 10:42:20.117033 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 07 10:42:20.117033 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application Aug 07 10:42:20.117033 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 359/734] 10.222.0.25 () {56 vars in 1108 bytes} [Sun Aug 7 10:42:19 2022] GET /identity/v3/domains/Default => generated 87 bytes in 233 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 07 10:42:20.117476 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 376/735] 10.222.0.25 () {54 vars in 927 bytes} [Sun Aug 7 10:42:20 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 07 10:42:20.118272 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c448cf4-ea63-4075-9791-e0f84748ad85 None admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 10:42:20.118591 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0dd3158-b9d5-4376-b493-7d1747826ee0 None None] REQUEST_METHOD: `POST` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:42:20.119469 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0dd3158-b9d5-4376-b493-7d1747826ee0 None None] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:42:20.119754 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0dd3158-b9d5-4376-b493-7d1747826ee0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:42:20.141030 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.auth.core [None req-c0dd3158-b9d5-4376-b493-7d1747826ee0 None None] MFA Rules not processed for user `bac4e6b76fe64217ac8fa44e71cffe6a`. Rule list: `[]` (Enabled: `True`). {{(pid=80144) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 10:42:20.143191 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.fernet_utils [None req-c0dd3158-b9d5-4376-b493-7d1747826ee0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80144) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 10:42:20.199731 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=100 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 10:42:20.227032 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 377/736] 10.222.0.25 () {58 vars in 1024 bytes} [Sun Aug 7 10:42:20 2022] POST /identity/v3/auth/tokens => generated 903 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 07 10:42:20.227980 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0dd3158-b9d5-4376-b493-7d1747826ee0 None None] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 10:42:20.241741 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c448cf4-ea63-4075-9791-e0f84748ad85 None admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 10:42:20.255269 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=100 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 10:42:20.287391 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b4716cf-b78d-4b03-a0b0-2c8ed9f4b92f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 10:42:20.288011 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b4716cf-b78d-4b03-a0b0-2c8ed9f4b92f None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:42:20.288309 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b4716cf-b78d-4b03-a0b0-2c8ed9f4b92f None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:42:20.288591 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b4716cf-b78d-4b03-a0b0-2c8ed9f4b92f None admin] PATH_INFO: `/v3/users` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:42:20.289311 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b4716cf-b78d-4b03-a0b0-2c8ed9f4b92f None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80143) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 10:42:20.289618 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b4716cf-b78d-4b03-a0b0-2c8ed9f4b92f None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 10:42:20.291143 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b4716cf-b78d-4b03-a0b0-2c8ed9f4b92f None admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 10:42:20.314560 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 360/737] 10.222.0.25 () {56 vars in 1135 bytes} [Sun Aug 7 10:42:20 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 10:42:20.314560 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b4716cf-b78d-4b03-a0b0-2c8ed9f4b92f None admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 10:42:20.349382 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0dd3158-b9d5-4376-b493-7d1747826ee0 None None] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 10:42:20.388117 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d77ea0a5-70f9-4caf-aeb7-7c25fbc9127a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 10:42:20.389078 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d77ea0a5-70f9-4caf-aeb7-7c25fbc9127a None admin] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:42:20.389556 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d77ea0a5-70f9-4caf-aeb7-7c25fbc9127a None admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:42:20.390041 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d77ea0a5-70f9-4caf-aeb7-7c25fbc9127a None admin] PATH_INFO: `/v3/domains` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:42:20.390903 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d77ea0a5-70f9-4caf-aeb7-7c25fbc9127a None admin] RBAC: Adding query filter params (name=Default) {{(pid=80144) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 10:42:20.391500 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d77ea0a5-70f9-4caf-aeb7-7c25fbc9127a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 10:42:20.393134 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d77ea0a5-70f9-4caf-aeb7-7c25fbc9127a None admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 10:42:20.406027 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 378/738] 10.222.0.25 () {56 vars in 1109 bytes} [Sun Aug 7 10:42:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 10:42:20.407362 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d77ea0a5-70f9-4caf-aeb7-7c25fbc9127a None admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 10:42:20.417832 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b4716cf-b78d-4b03-a0b0-2c8ed9f4b92f None admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 10:42:20.452086 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=100 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 10:42:20.454132 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c1ceef7-a4c7-474c-a513-8aa0f15bb5a4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 10:42:20.454792 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c1ceef7-a4c7-474c-a513-8aa0f15bb5a4 None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:42:20.455101 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c1ceef7-a4c7-474c-a513-8aa0f15bb5a4 None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:42:20.455378 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c1ceef7-a4c7-474c-a513-8aa0f15bb5a4 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:42:20.466370 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c1ceef7-a4c7-474c-a513-8aa0f15bb5a4 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 07 10:42:20.471217 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application [None req-6c1ceef7-a4c7-474c-a513-8aa0f15bb5a4 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 07 10:42:20.471217 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 07 10:42:20.471217 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 07 10:42:20.471217 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 07 10:42:20.471217 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 07 10:42:20.471217 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 07 10:42:20.471217 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 07 10:42:20.471217 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 07 10:42:20.471217 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 07 10:42:20.471217 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 07 10:42:20.471217 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 07 10:42:20.471217 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 07 10:42:20.471217 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 07 10:42:20.471217 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 07 10:42:20.471217 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 07 10:42:20.471217 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 07 10:42:20.472128 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 07 10:42:20.472128 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 07 10:42:20.472128 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 07 10:42:20.472128 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 07 10:42:20.472128 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 07 10:42:20.472128 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 07 10:42:20.472128 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 07 10:42:20.472128 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 07 10:42:20.472128 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 07 10:42:20.472128 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.get_or_create( Aug 07 10:42:20.472128 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 07 10:42:20.472128 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application with Lock( Aug 07 10:42:20.472128 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 07 10:42:20.472128 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self._enter() Aug 07 10:42:20.472128 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 07 10:42:20.472128 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 07 10:42:20.473556 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 07 10:42:20.473556 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.creator() Aug 07 10:42:20.473556 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 07 10:42:20.473556 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application created_value = creator( Aug 07 10:42:20.473556 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 07 10:42:20.473556 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 07 10:42:20.473556 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 07 10:42:20.473556 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 07 10:42:20.473556 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 07 10:42:20.473556 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 07 10:42:20.473556 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 07 10:42:20.473556 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application Aug 07 10:42:20.473556 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 361/739] 10.222.0.25 () {56 vars in 1102 bytes} [Sun Aug 7 10:42:20 2022] GET /identity/v3/roles/service => generated 85 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 07 10:42:20.473556 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c1ceef7-a4c7-474c-a513-8aa0f15bb5a4 None admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 10:42:20.497022 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=100 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 10:42:20.526647 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d77ea0a5-70f9-4caf-aeb7-7c25fbc9127a None admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 10:42:20.562673 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01d0cd30-e707-4c7e-9ced-727dd7abb7e4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 10:42:20.563427 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01d0cd30-e707-4c7e-9ced-727dd7abb7e4 None admin] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:42:20.563797 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01d0cd30-e707-4c7e-9ced-727dd7abb7e4 None admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:42:20.564080 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01d0cd30-e707-4c7e-9ced-727dd7abb7e4 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:42:20.584343 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c1ceef7-a4c7-474c-a513-8aa0f15bb5a4 None admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 10:42:20.623362 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a02d8ff-54b5-496c-a481-94396341ea49 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 10:42:20.624171 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a02d8ff-54b5-496c-a481-94396341ea49 None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:42:20.624395 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a02d8ff-54b5-496c-a481-94396341ea49 None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:42:20.624668 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a02d8ff-54b5-496c-a481-94396341ea49 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:42:20.632283 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a02d8ff-54b5-496c-a481-94396341ea49 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 10:42:20.633730 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a02d8ff-54b5-496c-a481-94396341ea49 None admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 10:42:20.634350 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01d0cd30-e707-4c7e-9ced-727dd7abb7e4 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 10:42:20.636426 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01d0cd30-e707-4c7e-9ced-727dd7abb7e4 None admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 10:42:20.645664 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application [None req-5a02d8ff-54b5-496c-a481-94396341ea49 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Aug 07 10:42:20.645664 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 07 10:42:20.645664 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 07 10:42:20.645664 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 07 10:42:20.645664 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 07 10:42:20.645664 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 07 10:42:20.645664 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 07 10:42:20.645664 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 07 10:42:20.645664 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 07 10:42:20.645664 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 07 10:42:20.645664 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 07 10:42:20.645664 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 07 10:42:20.645664 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 07 10:42:20.645664 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 07 10:42:20.645664 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 07 10:42:20.645664 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 07 10:42:20.646758 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 07 10:42:20.646758 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 07 10:42:20.646758 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 07 10:42:20.646758 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 07 10:42:20.646758 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 07 10:42:20.646758 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 07 10:42:20.646758 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 07 10:42:20.646758 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 07 10:42:20.646758 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 07 10:42:20.646758 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.get_or_create( Aug 07 10:42:20.646758 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 07 10:42:20.646758 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application with Lock( Aug 07 10:42:20.646758 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 07 10:42:20.646758 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self._enter() Aug 07 10:42:20.646758 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 07 10:42:20.646758 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 07 10:42:20.646758 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 07 10:42:20.646758 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.creator() Aug 07 10:42:20.647724 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 07 10:42:20.647724 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application created_value = creator( Aug 07 10:42:20.647724 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 07 10:42:20.647724 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 07 10:42:20.647724 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 07 10:42:20.647724 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 07 10:42:20.647724 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 07 10:42:20.647724 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 07 10:42:20.647724 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Aug 07 10:42:20.647724 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application Aug 07 10:42:20.648605 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 362/740] 10.222.0.25 () {56 vars in 1093 bytes} [Sun Aug 7 10:42:20 2022] GET /identity/v3/users/nova => generated 82 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 07 10:42:20.649112 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a02d8ff-54b5-496c-a481-94396341ea49 None admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 10:42:20.678705 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=100 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 10:42:20.743778 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 379/741] 10.222.0.25 () {58 vars in 1282 bytes} [Sun Aug 7 10:42:20 2022] GET /identity/v3/auth/tokens => generated 903 bytes in 337 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 07 10:42:20.745609 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31059cc9-c369-4d45-88b2-cce71f1fad2c None None] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:42:20.746147 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31059cc9-c369-4d45-88b2-cce71f1fad2c None None] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:42:20.746657 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31059cc9-c369-4d45-88b2-cce71f1fad2c None None] PATH_INFO: `/` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:42:20.748521 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 380/742] 10.222.0.25 () {54 vars in 927 bytes} [Sun Aug 7 10:42:20 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 07 10:42:20.749510 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=100 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 10:42:20.749752 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31059cc9-c369-4d45-88b2-cce71f1fad2c None None] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 10:42:20.768966 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a02d8ff-54b5-496c-a481-94396341ea49 None admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 10:42:20.804743 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-788c6df2-e79f-42af-8e82-72b24709c0e4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 10:42:20.805348 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-788c6df2-e79f-42af-8e82-72b24709c0e4 None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:42:20.805626 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-788c6df2-e79f-42af-8e82-72b24709c0e4 None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:42:20.805904 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-788c6df2-e79f-42af-8e82-72b24709c0e4 None admin] PATH_INFO: `/v3/roles` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:42:20.807157 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-788c6df2-e79f-42af-8e82-72b24709c0e4 None admin] RBAC: Adding query filter params (name=service) {{(pid=80143) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 10:42:20.807458 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-788c6df2-e79f-42af-8e82-72b24709c0e4 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 10:42:20.808813 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-788c6df2-e79f-42af-8e82-72b24709c0e4 None admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 10:42:20.821473 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 363/743] 10.222.0.25 () {56 vars in 1103 bytes} [Sun Aug 7 10:42:20 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 10:42:20.821473 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-788c6df2-e79f-42af-8e82-72b24709c0e4 None admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 10:42:20.869535 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31059cc9-c369-4d45-88b2-cce71f1fad2c None None] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 10:42:20.903845 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2b882bf-272d-4134-8eeb-c8699fac5dc1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 10:42:20.904445 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2b882bf-272d-4134-8eeb-c8699fac5dc1 None admin] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:42:20.904729 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2b882bf-272d-4134-8eeb-c8699fac5dc1 None admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:42:20.905004 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2b882bf-272d-4134-8eeb-c8699fac5dc1 None admin] PATH_INFO: `/v3/users` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:42:20.905722 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2b882bf-272d-4134-8eeb-c8699fac5dc1 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80144) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 10:42:20.906025 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2b882bf-272d-4134-8eeb-c8699fac5dc1 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 10:42:20.907461 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2b882bf-272d-4134-8eeb-c8699fac5dc1 None admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 10:42:20.920651 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=100 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 10:42:20.928679 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-788c6df2-e79f-42af-8e82-72b24709c0e4 None admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 10:42:20.931669 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 381/744] 10.222.0.25 () {56 vars in 1133 bytes} [Sun Aug 7 10:42:20 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 10:42:20.932893 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-523bf1f8-1708-4b36-9c58-5f61b4b594fc None None] REQUEST_METHOD: `POST` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:42:20.933181 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-523bf1f8-1708-4b36-9c58-5f61b4b594fc None None] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:42:20.933455 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-523bf1f8-1708-4b36-9c58-5f61b4b594fc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:42:20.950957 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.auth.core [None req-523bf1f8-1708-4b36-9c58-5f61b4b594fc None None] MFA Rules not processed for user `bac4e6b76fe64217ac8fa44e71cffe6a`. Rule list: `[]` (Enabled: `True`). {{(pid=80144) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 10:42:20.952401 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.fernet_utils [None req-523bf1f8-1708-4b36-9c58-5f61b4b594fc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80144) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 10:42:20.968390 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb0ce424-b5d5-4661-a90d-8473d16ad37a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 10:42:20.969023 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb0ce424-b5d5-4661-a90d-8473d16ad37a None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:42:20.969300 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb0ce424-b5d5-4661-a90d-8473d16ad37a None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:42:20.969597 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb0ce424-b5d5-4661-a90d-8473d16ad37a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:42:20.983496 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb0ce424-b5d5-4661-a90d-8473d16ad37a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 10:42:20.984923 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb0ce424-b5d5-4661-a90d-8473d16ad37a None admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 10:42:20.993583 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=100 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 10:42:21.002917 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application [None req-fb0ce424-b5d5-4661-a90d-8473d16ad37a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 07 10:42:21.002917 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 07 10:42:21.002917 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 07 10:42:21.002917 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 07 10:42:21.002917 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 07 10:42:21.002917 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 07 10:42:21.002917 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 07 10:42:21.002917 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 07 10:42:21.002917 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 07 10:42:21.002917 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 07 10:42:21.002917 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 07 10:42:21.002917 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 07 10:42:21.002917 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 07 10:42:21.002917 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 07 10:42:21.002917 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 07 10:42:21.002917 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 07 10:42:21.004055 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 07 10:42:21.004055 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 07 10:42:21.004055 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 07 10:42:21.004055 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 07 10:42:21.004055 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 07 10:42:21.004055 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.get_or_create( Aug 07 10:42:21.004055 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 07 10:42:21.004055 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application with Lock( Aug 07 10:42:21.004055 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 07 10:42:21.004055 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self._enter() Aug 07 10:42:21.004055 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 07 10:42:21.004055 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 07 10:42:21.004055 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 07 10:42:21.004055 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.creator() Aug 07 10:42:21.004055 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 07 10:42:21.004055 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application created_value = creator( Aug 07 10:42:21.004055 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 07 10:42:21.004955 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 07 10:42:21.004955 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 07 10:42:21.004955 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application Aug 07 10:42:21.004955 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 364/745] 10.222.0.25 () {56 vars in 1108 bytes} [Sun Aug 7 10:42:20 2022] GET /identity/v3/domains/Default => generated 87 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 07 10:42:21.004955 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb0ce424-b5d5-4661-a90d-8473d16ad37a None admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 10:42:21.038964 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 382/746] 10.222.0.25 () {58 vars in 1024 bytes} [Sun Aug 7 10:42:20 2022] POST /identity/v3/auth/tokens => generated 903 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 07 10:42:21.038964 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-523bf1f8-1708-4b36-9c58-5f61b4b594fc None None] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 10:42:21.116676 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb0ce424-b5d5-4661-a90d-8473d16ad37a None admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 10:42:21.151517 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-523bf1f8-1708-4b36-9c58-5f61b4b594fc None None] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 10:42:21.153325 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e34ffc9-4107-49a7-93e1-93880fd7e8b3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 10:42:21.153960 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e34ffc9-4107-49a7-93e1-93880fd7e8b3 None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:42:21.154236 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e34ffc9-4107-49a7-93e1-93880fd7e8b3 None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:42:21.154505 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e34ffc9-4107-49a7-93e1-93880fd7e8b3 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:42:21.166861 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e34ffc9-4107-49a7-93e1-93880fd7e8b3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 10:42:21.168381 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e34ffc9-4107-49a7-93e1-93880fd7e8b3 None admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 10:42:21.179908 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=100 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 10:42:21.184557 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application [None req-9e34ffc9-4107-49a7-93e1-93880fd7e8b3 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 07 10:42:21.184557 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 07 10:42:21.184557 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 07 10:42:21.184557 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 07 10:42:21.184557 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 07 10:42:21.184557 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 07 10:42:21.184557 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 07 10:42:21.184557 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 07 10:42:21.184557 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 07 10:42:21.184557 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 07 10:42:21.184557 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 07 10:42:21.184557 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 07 10:42:21.184557 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 07 10:42:21.184557 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 07 10:42:21.184557 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 07 10:42:21.184557 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 07 10:42:21.185800 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 07 10:42:21.185800 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 07 10:42:21.185800 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 07 10:42:21.185800 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 07 10:42:21.185800 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 07 10:42:21.185800 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.get_or_create( Aug 07 10:42:21.185800 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 07 10:42:21.185800 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application with Lock( Aug 07 10:42:21.185800 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 07 10:42:21.185800 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self._enter() Aug 07 10:42:21.185800 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 07 10:42:21.185800 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 07 10:42:21.185800 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 07 10:42:21.185800 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.creator() Aug 07 10:42:21.185800 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 07 10:42:21.185800 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application created_value = creator( Aug 07 10:42:21.185800 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 07 10:42:21.186686 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 07 10:42:21.186686 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 07 10:42:21.186686 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application Aug 07 10:42:21.186686 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 365/747] 10.222.0.25 () {56 vars in 1108 bytes} [Sun Aug 7 10:42:21 2022] GET /identity/v3/domains/Default => generated 87 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 07 10:42:21.186906 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e34ffc9-4107-49a7-93e1-93880fd7e8b3 None admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 10:42:21.194841 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98d1fb1f-8d34-49ca-920b-48be5ad2ab34 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 10:42:21.195495 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98d1fb1f-8d34-49ca-920b-48be5ad2ab34 None admin] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:42:21.195909 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98d1fb1f-8d34-49ca-920b-48be5ad2ab34 None admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:42:21.196169 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98d1fb1f-8d34-49ca-920b-48be5ad2ab34 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:42:21.208066 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98d1fb1f-8d34-49ca-920b-48be5ad2ab34 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 10:42:21.209623 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98d1fb1f-8d34-49ca-920b-48be5ad2ab34 None admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 10:42:21.225371 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application [None req-98d1fb1f-8d34-49ca-920b-48be5ad2ab34 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 07 10:42:21.225371 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 07 10:42:21.225371 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 07 10:42:21.225371 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 07 10:42:21.225371 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 07 10:42:21.225371 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 07 10:42:21.225371 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 07 10:42:21.225371 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 07 10:42:21.225371 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 07 10:42:21.225371 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 07 10:42:21.225371 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 07 10:42:21.225371 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 07 10:42:21.225371 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 07 10:42:21.225371 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 07 10:42:21.225371 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 07 10:42:21.225371 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 07 10:42:21.226335 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 07 10:42:21.226335 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 07 10:42:21.226335 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 07 10:42:21.226335 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 07 10:42:21.226335 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 07 10:42:21.226335 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self.get_or_create( Aug 07 10:42:21.226335 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 07 10:42:21.226335 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application with Lock( Aug 07 10:42:21.226335 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 07 10:42:21.226335 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self._enter() Aug 07 10:42:21.226335 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 07 10:42:21.226335 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 07 10:42:21.226335 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 07 10:42:21.226335 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self.creator() Aug 07 10:42:21.226335 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 07 10:42:21.226335 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application created_value = creator( Aug 07 10:42:21.226335 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 07 10:42:21.227262 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 07 10:42:21.227262 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 07 10:42:21.227262 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application Aug 07 10:42:21.227262 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 383/748] 10.222.0.25 () {56 vars in 1108 bytes} [Sun Aug 7 10:42:21 2022] GET /identity/v3/domains/Default => generated 87 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 07 10:42:21.227894 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98d1fb1f-8d34-49ca-920b-48be5ad2ab34 None admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 10:42:21.239865 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=100 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 10:42:21.296503 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e34ffc9-4107-49a7-93e1-93880fd7e8b3 None admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 10:42:21.330740 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db4c6855-5a20-4e2a-967e-e14b5a0b2276 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 10:42:21.331389 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db4c6855-5a20-4e2a-967e-e14b5a0b2276 None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:42:21.331667 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db4c6855-5a20-4e2a-967e-e14b5a0b2276 None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:42:21.331936 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db4c6855-5a20-4e2a-967e-e14b5a0b2276 None admin] PATH_INFO: `/v3/domains` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:42:21.332569 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db4c6855-5a20-4e2a-967e-e14b5a0b2276 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80143) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 10:42:21.332870 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db4c6855-5a20-4e2a-967e-e14b5a0b2276 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 10:42:21.334239 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db4c6855-5a20-4e2a-967e-e14b5a0b2276 None admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 10:42:21.342311 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98d1fb1f-8d34-49ca-920b-48be5ad2ab34 None admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 10:42:21.345592 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 366/749] 10.222.0.25 () {56 vars in 1109 bytes} [Sun Aug 7 10:42:21 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 10:42:21.350216 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db4c6855-5a20-4e2a-967e-e14b5a0b2276 None admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 10:42:21.379544 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c356f70-dff8-4b2d-80a4-f63d8c672cc5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 10:42:21.380224 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c356f70-dff8-4b2d-80a4-f63d8c672cc5 None admin] REQUEST_METHOD: `POST` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:42:21.380673 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c356f70-dff8-4b2d-80a4-f63d8c672cc5 None admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:42:21.381015 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c356f70-dff8-4b2d-80a4-f63d8c672cc5 None admin] PATH_INFO: `/v3/services` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:42:21.381822 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c356f70-dff8-4b2d-80a4-f63d8c672cc5 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 10:42:21.383252 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c356f70-dff8-4b2d-80a4-f63d8c672cc5 None admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 10:42:21.411203 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 384/750] 10.222.0.25 () {60 vars in 1141 bytes} [Sun Aug 7 10:42:21 2022] POST /identity/v3/services => generated 240 bytes in 184 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 07 10:42:21.412237 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c356f70-dff8-4b2d-80a4-f63d8c672cc5 None admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 10:42:21.463409 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db4c6855-5a20-4e2a-967e-e14b5a0b2276 None admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 10:42:21.473929 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=100 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 10:42:21.486167 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 10:42:21.519407 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e46f521-eee7-42c2-8fc0-9aca2f93ed3a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 10:42:21.520270 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e46f521-eee7-42c2-8fc0-9aca2f93ed3a None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:42:21.520717 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e46f521-eee7-42c2-8fc0-9aca2f93ed3a None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:42:21.521148 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e46f521-eee7-42c2-8fc0-9aca2f93ed3a None admin] PATH_INFO: `/v3/domains` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:42:21.521935 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e46f521-eee7-42c2-8fc0-9aca2f93ed3a None admin] RBAC: Adding query filter params (name=Default) {{(pid=80143) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 10:42:21.522383 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e46f521-eee7-42c2-8fc0-9aca2f93ed3a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 10:42:21.523990 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e46f521-eee7-42c2-8fc0-9aca2f93ed3a None admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 10:42:21.542656 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 367/751] 10.222.0.25 () {56 vars in 1109 bytes} [Sun Aug 7 10:42:21 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 10:42:21.543926 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e46f521-eee7-42c2-8fc0-9aca2f93ed3a None admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 10:42:21.576076 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c356f70-dff8-4b2d-80a4-f63d8c672cc5 None admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 10:42:21.622266 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-043a9cf3-d538-404f-b1e8-b94fa1adf67f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 10:42:21.622889 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-043a9cf3-d538-404f-b1e8-b94fa1adf67f None admin] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:42:21.623204 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-043a9cf3-d538-404f-b1e8-b94fa1adf67f None admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:42:21.623477 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-043a9cf3-d538-404f-b1e8-b94fa1adf67f None admin] PATH_INFO: `/v3/domains` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:42:21.624128 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-043a9cf3-d538-404f-b1e8-b94fa1adf67f None admin] RBAC: Adding query filter params (name=Default) {{(pid=80144) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 10:42:21.624430 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-043a9cf3-d538-404f-b1e8-b94fa1adf67f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 10:42:21.625782 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-043a9cf3-d538-404f-b1e8-b94fa1adf67f None admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 10:42:21.639593 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 385/752] 10.222.0.25 () {56 vars in 1109 bytes} [Sun Aug 7 10:42:21 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 228 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 10:42:21.640422 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-043a9cf3-d538-404f-b1e8-b94fa1adf67f None admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 10:42:21.664525 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e46f521-eee7-42c2-8fc0-9aca2f93ed3a None admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 10:42:21.709448 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a687b4d0-2420-43e9-985a-f437335d54ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 10:42:21.710690 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a687b4d0-2420-43e9-985a-f437335d54ed None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:42:21.711372 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a687b4d0-2420-43e9-985a-f437335d54ed None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:42:21.711844 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a687b4d0-2420-43e9-985a-f437335d54ed None admin] PATH_INFO: `/v3/projects/service` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:42:21.721864 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a687b4d0-2420-43e9-985a-f437335d54ed None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 10:42:21.724235 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a687b4d0-2420-43e9-985a-f437335d54ed None admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 10:42:21.739288 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application [None req-a687b4d0-2420-43e9-985a-f437335d54ed None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 07 10:42:21.739288 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 07 10:42:21.739288 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 07 10:42:21.739288 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 07 10:42:21.739288 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 07 10:42:21.739288 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 07 10:42:21.739288 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 07 10:42:21.739288 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 07 10:42:21.739288 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 07 10:42:21.739288 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 07 10:42:21.739288 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 07 10:42:21.739288 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 07 10:42:21.739288 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 07 10:42:21.739288 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 07 10:42:21.739288 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 07 10:42:21.740219 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 07 10:42:21.740219 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 07 10:42:21.740219 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 07 10:42:21.740219 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 07 10:42:21.740219 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 07 10:42:21.740219 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 07 10:42:21.740219 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.get_or_create( Aug 07 10:42:21.740219 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 07 10:42:21.740219 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application with Lock( Aug 07 10:42:21.740219 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 07 10:42:21.740219 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self._enter() Aug 07 10:42:21.740219 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 07 10:42:21.740219 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 07 10:42:21.740219 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 07 10:42:21.740219 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.creator() Aug 07 10:42:21.740219 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 07 10:42:21.740219 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application created_value = creator( Aug 07 10:42:21.741140 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 07 10:42:21.741140 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 07 10:42:21.741140 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 07 10:42:21.741140 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 07 10:42:21.741140 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 07 10:42:21.741140 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 07 10:42:21.741140 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 07 10:42:21.741140 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application Aug 07 10:42:21.744718 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 368/753] 10.222.0.25 () {56 vars in 1111 bytes} [Sun Aug 7 10:42:21 2022] GET /identity/v3/projects/service => generated 88 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 07 10:42:21.745893 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a687b4d0-2420-43e9-985a-f437335d54ed None admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 10:42:21.765254 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=100 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 10:42:21.771564 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-043a9cf3-d538-404f-b1e8-b94fa1adf67f None admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 10:42:21.781660 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=100 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 10:42:21.812360 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21fd678f-7bbb-4821-958e-f46a96650610 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 10:42:21.813274 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21fd678f-7bbb-4821-958e-f46a96650610 None admin] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:42:21.813771 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21fd678f-7bbb-4821-958e-f46a96650610 None admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:42:21.814185 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21fd678f-7bbb-4821-958e-f46a96650610 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:42:21.823397 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21fd678f-7bbb-4821-958e-f46a96650610 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 10:42:21.825224 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21fd678f-7bbb-4821-958e-f46a96650610 None admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 10:42:21.837941 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application [None req-21fd678f-7bbb-4821-958e-f46a96650610 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 07 10:42:21.837941 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 07 10:42:21.837941 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 07 10:42:21.837941 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 07 10:42:21.837941 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 07 10:42:21.837941 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 07 10:42:21.837941 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 07 10:42:21.837941 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 07 10:42:21.837941 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 07 10:42:21.837941 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 07 10:42:21.837941 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 07 10:42:21.837941 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 07 10:42:21.837941 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 07 10:42:21.837941 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 07 10:42:21.837941 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 07 10:42:21.837941 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 07 10:42:21.839090 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 07 10:42:21.839090 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 07 10:42:21.839090 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 07 10:42:21.839090 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 07 10:42:21.839090 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 07 10:42:21.839090 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 07 10:42:21.839090 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 07 10:42:21.839090 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 07 10:42:21.839090 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 07 10:42:21.839090 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self.get_or_create( Aug 07 10:42:21.839090 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 07 10:42:21.839090 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application with Lock( Aug 07 10:42:21.839090 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 07 10:42:21.839090 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self._enter() Aug 07 10:42:21.839090 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 07 10:42:21.839090 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 07 10:42:21.839090 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 07 10:42:21.839090 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self.creator() Aug 07 10:42:21.840066 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 07 10:42:21.840066 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application created_value = creator( Aug 07 10:42:21.840066 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 07 10:42:21.840066 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 07 10:42:21.840066 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 07 10:42:21.840066 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 07 10:42:21.840066 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 07 10:42:21.840066 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 07 10:42:21.840066 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 07 10:42:21.840066 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application Aug 07 10:42:21.842242 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 386/754] 10.222.0.25 () {56 vars in 1117 bytes} [Sun Aug 7 10:42:21 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 07 10:42:21.843195 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21fd678f-7bbb-4821-958e-f46a96650610 None admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 10:42:21.865763 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a687b4d0-2420-43e9-985a-f437335d54ed None admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 10:42:21.901053 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16602fca-86f6-4132-a268-756a49ab62e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 10:42:21.901715 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16602fca-86f6-4132-a268-756a49ab62e6 None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:42:21.901993 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16602fca-86f6-4132-a268-756a49ab62e6 None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:42:21.902266 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16602fca-86f6-4132-a268-756a49ab62e6 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:42:21.909479 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16602fca-86f6-4132-a268-756a49ab62e6 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 10:42:21.910937 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16602fca-86f6-4132-a268-756a49ab62e6 None admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 10:42:21.923702 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application [None req-16602fca-86f6-4132-a268-756a49ab62e6 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 07 10:42:21.923702 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 07 10:42:21.923702 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 07 10:42:21.923702 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 07 10:42:21.923702 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 07 10:42:21.923702 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 07 10:42:21.923702 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 07 10:42:21.923702 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 07 10:42:21.923702 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 07 10:42:21.923702 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 07 10:42:21.923702 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 07 10:42:21.923702 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 07 10:42:21.923702 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 07 10:42:21.923702 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 07 10:42:21.923702 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 07 10:42:21.924865 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 07 10:42:21.924865 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 07 10:42:21.924865 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 07 10:42:21.924865 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 07 10:42:21.924865 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 07 10:42:21.924865 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 07 10:42:21.924865 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.get_or_create( Aug 07 10:42:21.924865 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 07 10:42:21.924865 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application with Lock( Aug 07 10:42:21.924865 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 07 10:42:21.924865 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self._enter() Aug 07 10:42:21.924865 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 07 10:42:21.924865 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 07 10:42:21.924865 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 07 10:42:21.924865 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.creator() Aug 07 10:42:21.924865 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 07 10:42:21.924865 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application created_value = creator( Aug 07 10:42:21.925927 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 07 10:42:21.925927 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 07 10:42:21.925927 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 07 10:42:21.925927 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 07 10:42:21.925927 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 07 10:42:21.925927 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 07 10:42:21.925927 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 07 10:42:21.925927 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application Aug 07 10:42:21.929412 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 369/755] 10.222.0.25 () {56 vars in 1111 bytes} [Sun Aug 7 10:42:21 2022] GET /identity/v3/projects/service => generated 88 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 07 10:42:21.930264 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16602fca-86f6-4132-a268-756a49ab62e6 None admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 10:42:21.960494 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21fd678f-7bbb-4821-958e-f46a96650610 None admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 10:42:21.999909 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceda3c92-2cdf-45b3-ae38-bae1d9848441 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 10:42:22.000772 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceda3c92-2cdf-45b3-ae38-bae1d9848441 None admin] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:42:22.001789 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceda3c92-2cdf-45b3-ae38-bae1d9848441 None admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:42:22.002216 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceda3c92-2cdf-45b3-ae38-bae1d9848441 None admin] PATH_INFO: `/v3/projects` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:42:22.003367 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ceda3c92-2cdf-45b3-ae38-bae1d9848441 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80144) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 10:42:22.003836 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ceda3c92-2cdf-45b3-ae38-bae1d9848441 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 10:42:22.006310 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ceda3c92-2cdf-45b3-ae38-bae1d9848441 None admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 10:42:22.029289 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 387/756] 10.222.0.25 () {56 vars in 1148 bytes} [Sun Aug 7 10:42:21 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 10:42:22.029289 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceda3c92-2cdf-45b3-ae38-bae1d9848441 None admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 10:42:22.037129 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=100 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 10:42:22.057364 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=100 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 10:42:22.058239 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16602fca-86f6-4132-a268-756a49ab62e6 None admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 10:42:22.106956 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a380c83-dcc5-4951-9932-55393b24f754 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 10:42:22.108363 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a380c83-dcc5-4951-9932-55393b24f754 None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:42:22.109081 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a380c83-dcc5-4951-9932-55393b24f754 None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:42:22.109559 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a380c83-dcc5-4951-9932-55393b24f754 None admin] PATH_INFO: `/v3/users` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:42:22.111131 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a380c83-dcc5-4951-9932-55393b24f754 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80143) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 10:42:22.111927 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a380c83-dcc5-4951-9932-55393b24f754 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 10:42:22.113864 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a380c83-dcc5-4951-9932-55393b24f754 None admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 10:42:22.138340 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 370/757] 10.222.0.25 () {56 vars in 1149 bytes} [Sun Aug 7 10:42:21 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 10:42:22.139568 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a380c83-dcc5-4951-9932-55393b24f754 None admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 10:42:22.151875 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceda3c92-2cdf-45b3-ae38-bae1d9848441 None admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 10:42:22.190670 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23854472-a5ee-4275-b6cb-182c2467f4ea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 10:42:22.191613 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23854472-a5ee-4275-b6cb-182c2467f4ea None admin] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:42:22.191907 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23854472-a5ee-4275-b6cb-182c2467f4ea None admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:42:22.192308 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23854472-a5ee-4275-b6cb-182c2467f4ea None admin] PATH_INFO: `/v3/projects` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:42:22.193138 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23854472-a5ee-4275-b6cb-182c2467f4ea None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80144) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 10:42:22.193451 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23854472-a5ee-4275-b6cb-182c2467f4ea None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 10:42:22.195358 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23854472-a5ee-4275-b6cb-182c2467f4ea None admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 10:42:22.216167 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 388/758] 10.222.0.25 () {56 vars in 1148 bytes} [Sun Aug 7 10:42:22 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 10:42:22.216167 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b6c7f54-6bf1-46c3-abe3-478b4ee1a16d None None] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:42:22.216167 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b6c7f54-6bf1-46c3-abe3-478b4ee1a16d None None] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:42:22.216167 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b6c7f54-6bf1-46c3-abe3-478b4ee1a16d None None] PATH_INFO: `/` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:42:22.219047 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 389/759] 10.222.0.25 () {54 vars in 927 bytes} [Sun Aug 7 10:42:22 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 07 10:42:22.220038 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b6c7f54-6bf1-46c3-abe3-478b4ee1a16d None None] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 10:42:22.252792 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a380c83-dcc5-4951-9932-55393b24f754 None admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 10:42:22.289705 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49f157d5-1cc3-4836-8763-aebd02b9990c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 10:42:22.290314 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49f157d5-1cc3-4836-8763-aebd02b9990c None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:42:22.290657 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49f157d5-1cc3-4836-8763-aebd02b9990c None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:42:22.291291 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49f157d5-1cc3-4836-8763-aebd02b9990c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:42:22.291663 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49f157d5-1cc3-4836-8763-aebd02b9990c None admin] RBAC: Adding query filter params (role.id=81b1f1bcee2a473088fbbd327c1c5b23, scope.project.id=6de1f3cac02c4bcdb05dd2337176033d, user.id=69490ea2a9a340bf91aca38bf5655b45) {{(pid=80143) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 10:42:22.291971 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49f157d5-1cc3-4836-8763-aebd02b9990c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 10:42:22.293386 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49f157d5-1cc3-4836-8763-aebd02b9990c None admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 10:42:22.305470 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=100 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 10:42:22.307544 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 371/760] 10.222.0.25 () {56 vars in 1374 bytes} [Sun Aug 7 10:42:22 2022] GET /identity/v3/role_assignments?user.id=69490ea2a9a340bf91aca38bf5655b45&scope.project.id=6de1f3cac02c4bcdb05dd2337176033d&role.id=81b1f1bcee2a473088fbbd327c1c5b23 => generated 259 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 10:42:22.308352 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49f157d5-1cc3-4836-8763-aebd02b9990c None admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 10:42:22.315627 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=100 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 10:42:22.340606 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b6c7f54-6bf1-46c3-abe3-478b4ee1a16d None None] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 10:42:22.381174 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef06ef7b-0a07-48bf-bf06-5a9ad16b505e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 10:42:22.381803 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef06ef7b-0a07-48bf-bf06-5a9ad16b505e None admin] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:42:22.382135 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef06ef7b-0a07-48bf-bf06-5a9ad16b505e None admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:42:22.382417 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef06ef7b-0a07-48bf-bf06-5a9ad16b505e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:42:22.462501 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49f157d5-1cc3-4836-8763-aebd02b9990c None admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 10:42:22.475843 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef06ef7b-0a07-48bf-bf06-5a9ad16b505e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 10:42:22.477393 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef06ef7b-0a07-48bf-bf06-5a9ad16b505e None admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 10:42:22.511845 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2b0df95-d5cb-4e93-984c-537ac8becb34 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 10:42:22.512804 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2b0df95-d5cb-4e93-984c-537ac8becb34 None admin] REQUEST_METHOD: `PUT` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:42:22.513259 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2b0df95-d5cb-4e93-984c-537ac8becb34 None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:42:22.513732 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2b0df95-d5cb-4e93-984c-537ac8becb34 None admin] PATH_INFO: `/v3/projects/6de1f3cac02c4bcdb05dd2337176033d/users/31d16d1b96104b4eb690398432b9829e/roles/81b1f1bcee2a473088fbbd327c1c5b23` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:42:22.515839 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2b0df95-d5cb-4e93-984c-537ac8becb34 None admin] RBAC: Authorizing `identity:create_grant(project_id=6de1f3cac02c4bcdb05dd2337176033d, user_id=31d16d1b96104b4eb690398432b9829e, role_id=81b1f1bcee2a473088fbbd327c1c5b23)` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 10:42:22.517436 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2b0df95-d5cb-4e93-984c-537ac8becb34 None admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 10:42:22.536955 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 372/761] 10.222.0.25 () {58 vars in 1439 bytes} [Sun Aug 7 10:42:22 2022] PUT /identity/v3/projects/6de1f3cac02c4bcdb05dd2337176033d/users/31d16d1b96104b4eb690398432b9829e/roles/81b1f1bcee2a473088fbbd327c1c5b23 => generated 0 bytes in 229 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 07 10:42:22.538645 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70f4ec1f-68a7-4108-b03b-48e63a01d325 None None] REQUEST_METHOD: `POST` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:42:22.539754 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70f4ec1f-68a7-4108-b03b-48e63a01d325 None None] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:42:22.540276 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70f4ec1f-68a7-4108-b03b-48e63a01d325 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:42:22.574304 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.auth.core [None req-70f4ec1f-68a7-4108-b03b-48e63a01d325 None None] MFA Rules not processed for user `bac4e6b76fe64217ac8fa44e71cffe6a`. Rule list: `[]` (Enabled: `True`). {{(pid=80143) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 10:42:22.577317 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-70f4ec1f-68a7-4108-b03b-48e63a01d325 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 10:42:22.613531 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=100 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 10:42:22.665297 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 390/762] 10.222.0.25 () {58 vars in 1282 bytes} [Sun Aug 7 10:42:22 2022] GET /identity/v3/auth/tokens => generated 1002 bytes in 441 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 07 10:42:22.669949 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef06ef7b-0a07-48bf-bf06-5a9ad16b505e None admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 10:42:22.703222 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 10:42:22.727164 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 373/763] 10.222.0.25 () {58 vars in 1024 bytes} [Sun Aug 7 10:42:22 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 182 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 07 10:42:22.743651 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70f4ec1f-68a7-4108-b03b-48e63a01d325 None None] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 10:42:22.834877 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef06ef7b-0a07-48bf-bf06-5a9ad16b505e None admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 10:42:22.859233 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70f4ec1f-68a7-4108-b03b-48e63a01d325 None None] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 10:42:22.874296 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91102d90-b983-4fa7-8dd1-d31bb6b06e8b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 10:42:22.875436 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91102d90-b983-4fa7-8dd1-d31bb6b06e8b None admin] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:42:22.875966 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91102d90-b983-4fa7-8dd1-d31bb6b06e8b None admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:42:22.876462 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91102d90-b983-4fa7-8dd1-d31bb6b06e8b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:42:22.887967 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91102d90-b983-4fa7-8dd1-d31bb6b06e8b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 10:42:22.889832 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91102d90-b983-4fa7-8dd1-d31bb6b06e8b None admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 10:42:22.896907 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b4d0650-7fee-47cd-af28-8f11ab10ca29 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 10:42:22.897593 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b4d0650-7fee-47cd-af28-8f11ab10ca29 None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:42:22.897873 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b4d0650-7fee-47cd-af28-8f11ab10ca29 None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:42:22.898143 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b4d0650-7fee-47cd-af28-8f11ab10ca29 None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:42:22.898730 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b4d0650-7fee-47cd-af28-8f11ab10ca29 None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 10:42:22.900329 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b4d0650-7fee-47cd-af28-8f11ab10ca29 None admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 10:42:22.908055 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application [None req-91102d90-b983-4fa7-8dd1-d31bb6b06e8b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 07 10:42:22.908055 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 07 10:42:22.908055 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 07 10:42:22.908055 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 07 10:42:22.908055 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 07 10:42:22.908055 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 07 10:42:22.908055 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 07 10:42:22.908055 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 07 10:42:22.908055 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 07 10:42:22.908055 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 07 10:42:22.908055 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 07 10:42:22.908055 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 07 10:42:22.908055 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 07 10:42:22.908055 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 07 10:42:22.908055 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 07 10:42:22.908055 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 07 10:42:22.908978 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 07 10:42:22.908978 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 07 10:42:22.908978 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 07 10:42:22.908978 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 07 10:42:22.908978 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 07 10:42:22.908978 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self.get_or_create( Aug 07 10:42:22.908978 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 07 10:42:22.908978 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application with Lock( Aug 07 10:42:22.908978 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 07 10:42:22.908978 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self._enter() Aug 07 10:42:22.908978 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 07 10:42:22.908978 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 07 10:42:22.908978 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 07 10:42:22.908978 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self.creator() Aug 07 10:42:22.908978 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 07 10:42:22.908978 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application created_value = creator( Aug 07 10:42:22.908978 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 07 10:42:22.909860 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 07 10:42:22.909860 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 07 10:42:22.909860 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application Aug 07 10:42:22.910031 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application [None req-8b4d0650-7fee-47cd-af28-8f11ab10ca29 None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Aug 07 10:42:22.910031 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 07 10:42:22.910031 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 07 10:42:22.910031 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 07 10:42:22.910031 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 07 10:42:22.910031 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 07 10:42:22.910031 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 07 10:42:22.910031 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 07 10:42:22.910031 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Aug 07 10:42:22.910031 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application Aug 07 10:42:22.910031 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 07 10:42:22.910031 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application Aug 07 10:42:22.910031 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 07 10:42:22.910031 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 07 10:42:22.910031 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 07 10:42:22.910031 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 07 10:42:22.910031 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 07 10:42:22.910912 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 07 10:42:22.910912 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 07 10:42:22.910912 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 07 10:42:22.910912 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 07 10:42:22.910912 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 07 10:42:22.910912 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 07 10:42:22.910912 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 07 10:42:22.910912 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 07 10:42:22.910912 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 07 10:42:22.910912 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 07 10:42:22.910912 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 07 10:42:22.910912 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 07 10:42:22.910912 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 07 10:42:22.910912 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 07 10:42:22.910912 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 07 10:42:22.910912 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.get_or_create( Aug 07 10:42:22.912383 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 07 10:42:22.912383 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application with Lock( Aug 07 10:42:22.912383 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 07 10:42:22.912383 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self._enter() Aug 07 10:42:22.912383 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 07 10:42:22.912383 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 07 10:42:22.912383 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 07 10:42:22.912383 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.creator() Aug 07 10:42:22.912383 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 07 10:42:22.912383 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application created_value = creator( Aug 07 10:42:22.912383 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 07 10:42:22.912383 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 07 10:42:22.912383 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Aug 07 10:42:22.912383 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application Aug 07 10:42:22.914647 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 374/764] 10.222.0.25 () {56 vars in 1129 bytes} [Sun Aug 7 10:42:22 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 07 10:42:22.916730 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 391/765] 10.222.0.25 () {56 vars in 1108 bytes} [Sun Aug 7 10:42:22 2022] GET /identity/v3/domains/Default => generated 87 bytes in 247 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 07 10:42:22.917496 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b4d0650-7fee-47cd-af28-8f11ab10ca29 None admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 10:42:22.944641 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91102d90-b983-4fa7-8dd1-d31bb6b06e8b None admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 10:42:22.986461 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=100 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 10:42:23.001896 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=100 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 10:42:23.043382 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b4d0650-7fee-47cd-af28-8f11ab10ca29 None admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 10:42:23.082102 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f616491-3bec-4972-b136-25dc280e1b1a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 10:42:23.082879 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f616491-3bec-4972-b136-25dc280e1b1a None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:42:23.083287 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f616491-3bec-4972-b136-25dc280e1b1a None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:42:23.083583 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f616491-3bec-4972-b136-25dc280e1b1a None admin] PATH_INFO: `/v3/services` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:42:23.084665 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f616491-3bec-4972-b136-25dc280e1b1a None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=80143) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 10:42:23.084665 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f616491-3bec-4972-b136-25dc280e1b1a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 10:42:23.088554 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f616491-3bec-4972-b136-25dc280e1b1a None admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 10:42:23.099241 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 375/766] 10.222.0.25 () {56 vars in 1124 bytes} [Sun Aug 7 10:42:22 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 07 10:42:23.108687 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f616491-3bec-4972-b136-25dc280e1b1a None admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 10:42:23.144568 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91102d90-b983-4fa7-8dd1-d31bb6b06e8b None admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 10:42:23.223325 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bce6f774-a61f-44a9-9dfb-dfc723bd269d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 10:42:23.225378 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bce6f774-a61f-44a9-9dfb-dfc723bd269d None admin] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:42:23.225665 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bce6f774-a61f-44a9-9dfb-dfc723bd269d None admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:42:23.230954 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bce6f774-a61f-44a9-9dfb-dfc723bd269d None admin] PATH_INFO: `/v3/domains` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:42:23.230954 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bce6f774-a61f-44a9-9dfb-dfc723bd269d None admin] RBAC: Adding query filter params (name=Default) {{(pid=80144) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 10:42:23.230954 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bce6f774-a61f-44a9-9dfb-dfc723bd269d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 10:42:23.230954 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bce6f774-a61f-44a9-9dfb-dfc723bd269d None admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 10:42:23.264635 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 392/767] 10.222.0.25 () {56 vars in 1109 bytes} [Sun Aug 7 10:42:22 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 320 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 07 10:42:23.267158 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bce6f774-a61f-44a9-9dfb-dfc723bd269d None admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 10:42:23.325675 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f616491-3bec-4972-b136-25dc280e1b1a None admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 10:42:23.379278 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=100 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 10:42:23.404238 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c99d649d-8853-42a2-b34e-6481bc473d9d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 10:42:23.404882 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c99d649d-8853-42a2-b34e-6481bc473d9d None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:42:23.405158 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c99d649d-8853-42a2-b34e-6481bc473d9d None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:42:23.405425 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c99d649d-8853-42a2-b34e-6481bc473d9d None admin] PATH_INFO: `/v3/services` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:42:23.406060 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c99d649d-8853-42a2-b34e-6481bc473d9d None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=80143) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 10:42:23.406362 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c99d649d-8853-42a2-b34e-6481bc473d9d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 10:42:23.411081 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c99d649d-8853-42a2-b34e-6481bc473d9d None admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 10:42:23.426865 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 376/768] 10.222.0.25 () {56 vars in 1124 bytes} [Sun Aug 7 10:42:23 2022] GET /identity/v3/services?type=block-storage => generated 367 bytes in 312 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 10:42:23.435727 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c99d649d-8853-42a2-b34e-6481bc473d9d None admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 10:42:23.443457 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=100 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 10:42:23.473612 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bce6f774-a61f-44a9-9dfb-dfc723bd269d None admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 10:42:23.581604 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ccd5f43-a927-4f61-86b8-309e213cd332 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 10:42:23.581604 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ccd5f43-a927-4f61-86b8-309e213cd332 None admin] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:42:23.581604 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ccd5f43-a927-4f61-86b8-309e213cd332 None admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:42:23.581604 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ccd5f43-a927-4f61-86b8-309e213cd332 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:42:23.609502 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ccd5f43-a927-4f61-86b8-309e213cd332 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 10:42:23.610986 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ccd5f43-a927-4f61-86b8-309e213cd332 None admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 10:42:23.636426 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application [None req-6ccd5f43-a927-4f61-86b8-309e213cd332 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 07 10:42:23.636426 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 07 10:42:23.636426 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 07 10:42:23.636426 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 07 10:42:23.636426 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 07 10:42:23.636426 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 07 10:42:23.636426 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 07 10:42:23.636426 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 07 10:42:23.636426 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 07 10:42:23.636426 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 07 10:42:23.636426 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 07 10:42:23.636426 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 07 10:42:23.636426 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 07 10:42:23.636426 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 07 10:42:23.636426 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 07 10:42:23.637358 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 07 10:42:23.637358 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 07 10:42:23.637358 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 07 10:42:23.637358 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 07 10:42:23.637358 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 07 10:42:23.637358 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 07 10:42:23.637358 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self.get_or_create( Aug 07 10:42:23.637358 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 07 10:42:23.637358 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application with Lock( Aug 07 10:42:23.637358 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 07 10:42:23.637358 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self._enter() Aug 07 10:42:23.637358 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 07 10:42:23.637358 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 07 10:42:23.637358 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 07 10:42:23.637358 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self.creator() Aug 07 10:42:23.637358 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 07 10:42:23.637358 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application created_value = creator( Aug 07 10:42:23.638223 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 07 10:42:23.638223 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 07 10:42:23.638223 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 07 10:42:23.638223 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 07 10:42:23.638223 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 07 10:42:23.638223 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 07 10:42:23.638223 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 07 10:42:23.638223 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application Aug 07 10:42:23.638223 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 393/769] 10.222.0.25 () {56 vars in 1111 bytes} [Sun Aug 7 10:42:23 2022] GET /identity/v3/projects/service => generated 88 bytes in 371 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 07 10:42:23.639022 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9ca2f95-3197-46b4-a8b2-86efd7830039 None None] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:42:23.640383 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9ca2f95-3197-46b4-a8b2-86efd7830039 None None] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:42:23.640999 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9ca2f95-3197-46b4-a8b2-86efd7830039 None None] PATH_INFO: `/` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:42:23.644457 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c99d649d-8853-42a2-b34e-6481bc473d9d None admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 10:42:23.648685 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 394/770] 10.222.0.25 () {54 vars in 927 bytes} [Sun Aug 7 10:42:23 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 07 10:42:23.648685 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9ca2f95-3197-46b4-a8b2-86efd7830039 None None] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 10:42:23.719801 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6faa246b-9b14-4f8f-9f89-d725f1c9caee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 10:42:23.721209 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6faa246b-9b14-4f8f-9f89-d725f1c9caee None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:42:23.721728 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6faa246b-9b14-4f8f-9f89-d725f1c9caee None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:42:23.722248 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6faa246b-9b14-4f8f-9f89-d725f1c9caee None admin] PATH_INFO: `/v3/endpoints` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:42:23.723292 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6faa246b-9b14-4f8f-9f89-d725f1c9caee None admin] RBAC: Adding query filter params (interface=public, service_id=c57808f48573433893449786258f6d17, region_id=RegionOne) {{(pid=80143) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 10:42:23.723922 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6faa246b-9b14-4f8f-9f89-d725f1c9caee None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 10:42:23.725828 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6faa246b-9b14-4f8f-9f89-d725f1c9caee None admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 10:42:23.741009 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 377/771] 10.222.0.25 () {56 vars in 1251 bytes} [Sun Aug 7 10:42:23 2022] GET /identity/v3/endpoints?service_id=c57808f48573433893449786258f6d17&interface=public®ion_id=RegionOne => generated 194 bytes in 306 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 07 10:42:23.742363 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94d446d2-45ce-434d-a22d-70531b451896 None None] REQUEST_METHOD: `POST` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:42:23.742659 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94d446d2-45ce-434d-a22d-70531b451896 None None] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:42:23.742964 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94d446d2-45ce-434d-a22d-70531b451896 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:42:23.764099 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.auth.core [None req-94d446d2-45ce-434d-a22d-70531b451896 None None] MFA Rules not processed for user `bac4e6b76fe64217ac8fa44e71cffe6a`. Rule list: `[]` (Enabled: `True`). {{(pid=80143) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 10:42:23.765871 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-94d446d2-45ce-434d-a22d-70531b451896 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 10:42:23.823288 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=100 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 10:42:23.827010 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9ca2f95-3197-46b4-a8b2-86efd7830039 None None] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 10:42:23.855395 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 378/772] 10.222.0.25 () {58 vars in 1024 bytes} [Sun Aug 7 10:42:23 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 07 10:42:23.855395 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94d446d2-45ce-434d-a22d-70531b451896 None None] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 10:42:23.886372 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2f270ca-3508-40f1-b132-156ebe8b67f7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 10:42:23.887067 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2f270ca-3508-40f1-b132-156ebe8b67f7 None admin] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:42:23.887322 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2f270ca-3508-40f1-b132-156ebe8b67f7 None admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:42:23.887599 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2f270ca-3508-40f1-b132-156ebe8b67f7 None admin] PATH_INFO: `/v3/projects` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:42:23.888301 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2f270ca-3508-40f1-b132-156ebe8b67f7 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80144) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 10:42:23.888620 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2f270ca-3508-40f1-b132-156ebe8b67f7 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 10:42:23.890066 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2f270ca-3508-40f1-b132-156ebe8b67f7 None admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 10:42:23.908029 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 395/773] 10.222.0.25 () {56 vars in 1148 bytes} [Sun Aug 7 10:42:23 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 262 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 10:42:23.908894 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2f270ca-3508-40f1-b132-156ebe8b67f7 None admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 10:42:23.924291 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=100 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 10:42:23.992352 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94d446d2-45ce-434d-a22d-70531b451896 None None] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 10:42:24.028622 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3183a0ed-b13a-4da8-b6eb-59e3a7a051f9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 10:42:24.029336 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3183a0ed-b13a-4da8-b6eb-59e3a7a051f9 None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:42:24.029542 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3183a0ed-b13a-4da8-b6eb-59e3a7a051f9 None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:42:24.029785 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3183a0ed-b13a-4da8-b6eb-59e3a7a051f9 None admin] PATH_INFO: `/v3/services` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:42:24.030400 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3183a0ed-b13a-4da8-b6eb-59e3a7a051f9 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 10:42:24.031796 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3183a0ed-b13a-4da8-b6eb-59e3a7a051f9 None admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 10:42:24.037890 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 379/774] 10.222.0.25 () {56 vars in 1087 bytes} [Sun Aug 7 10:42:23 2022] GET /identity/v3/services => generated 772 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 10:42:24.038773 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3183a0ed-b13a-4da8-b6eb-59e3a7a051f9 None admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 10:42:24.049466 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2f270ca-3508-40f1-b132-156ebe8b67f7 None admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 10:42:24.097560 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=100 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 10:42:24.099132 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9629e181-78ac-4d2f-a90d-293200e69288 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 10:42:24.100014 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9629e181-78ac-4d2f-a90d-293200e69288 None admin] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:42:24.101205 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9629e181-78ac-4d2f-a90d-293200e69288 None admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:42:24.101738 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9629e181-78ac-4d2f-a90d-293200e69288 None admin] PATH_INFO: `/v3/services/network` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:42:24.102545 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9629e181-78ac-4d2f-a90d-293200e69288 None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 10:42:24.108823 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9629e181-78ac-4d2f-a90d-293200e69288 None admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 10:42:24.128137 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application [None req-9629e181-78ac-4d2f-a90d-293200e69288 None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Aug 07 10:42:24.128137 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 07 10:42:24.128137 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 07 10:42:24.128137 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 07 10:42:24.128137 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 07 10:42:24.128137 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 07 10:42:24.128137 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 07 10:42:24.128137 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 07 10:42:24.128137 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Aug 07 10:42:24.128137 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application Aug 07 10:42:24.128137 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 07 10:42:24.128137 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application Aug 07 10:42:24.128137 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 07 10:42:24.128137 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 07 10:42:24.128137 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 07 10:42:24.128137 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 07 10:42:24.128137 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 07 10:42:24.129195 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 07 10:42:24.129195 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 07 10:42:24.129195 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 07 10:42:24.129195 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 07 10:42:24.129195 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 07 10:42:24.129195 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 07 10:42:24.129195 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 07 10:42:24.129195 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 07 10:42:24.129195 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 07 10:42:24.129195 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 07 10:42:24.129195 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 07 10:42:24.129195 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 07 10:42:24.129195 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 07 10:42:24.129195 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 07 10:42:24.129195 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 07 10:42:24.129195 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self.get_or_create( Aug 07 10:42:24.130043 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 07 10:42:24.130043 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application with Lock( Aug 07 10:42:24.130043 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 07 10:42:24.130043 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self._enter() Aug 07 10:42:24.130043 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 07 10:42:24.130043 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 07 10:42:24.130043 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 07 10:42:24.130043 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self.creator() Aug 07 10:42:24.130043 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 07 10:42:24.130043 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application created_value = creator( Aug 07 10:42:24.130043 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 07 10:42:24.130043 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 07 10:42:24.130043 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Aug 07 10:42:24.130043 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application Aug 07 10:42:24.132698 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 396/775] 10.222.0.25 () {56 vars in 1111 bytes} [Sun Aug 7 10:42:23 2022] GET /identity/v3/services/network => generated 88 bytes in 224 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 07 10:42:24.140412 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9629e181-78ac-4d2f-a90d-293200e69288 None admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 10:42:24.161377 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3183a0ed-b13a-4da8-b6eb-59e3a7a051f9 None admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 10:42:24.200608 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ba80a4c-14c5-4aba-81f6-38a998b2c5df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 10:42:24.201226 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ba80a4c-14c5-4aba-81f6-38a998b2c5df None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:42:24.201508 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ba80a4c-14c5-4aba-81f6-38a998b2c5df None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:42:24.201783 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ba80a4c-14c5-4aba-81f6-38a998b2c5df None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:42:24.202639 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ba80a4c-14c5-4aba-81f6-38a998b2c5df None admin] RBAC: Adding query filter params (role.id=fbf375e9760340c9aa9529333eeb25f9, scope.project.id=6de1f3cac02c4bcdb05dd2337176033d, user.id=de1d6c24b1a14d10a20d82a4c65793d8) {{(pid=80143) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 10:42:24.202983 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ba80a4c-14c5-4aba-81f6-38a998b2c5df None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 10:42:24.204587 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ba80a4c-14c5-4aba-81f6-38a998b2c5df None admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 10:42:24.210898 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 380/776] 10.222.0.25 () {56 vars in 1374 bytes} [Sun Aug 7 10:42:24 2022] GET /identity/v3/role_assignments?user.id=de1d6c24b1a14d10a20d82a4c65793d8&scope.project.id=6de1f3cac02c4bcdb05dd2337176033d&role.id=fbf375e9760340c9aa9529333eeb25f9 => generated 259 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 10:42:24.253232 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=100 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 10:42:24.286684 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9629e181-78ac-4d2f-a90d-293200e69288 None admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 10:42:24.325522 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6443b16-6778-486d-b46c-b13c93b68db1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 10:42:24.326526 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6443b16-6778-486d-b46c-b13c93b68db1 None admin] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:42:24.327092 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6443b16-6778-486d-b46c-b13c93b68db1 None admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:42:24.327565 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6443b16-6778-486d-b46c-b13c93b68db1 None admin] PATH_INFO: `/v3/services` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:42:24.328402 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6443b16-6778-486d-b46c-b13c93b68db1 None admin] RBAC: Adding query filter params (name=network) {{(pid=80144) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 10:42:24.328921 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6443b16-6778-486d-b46c-b13c93b68db1 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 10:42:24.330494 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6443b16-6778-486d-b46c-b13c93b68db1 None admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 10:42:24.336324 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 397/777] 10.222.0.25 () {56 vars in 1112 bytes} [Sun Aug 7 10:42:24 2022] GET /identity/v3/services?name=network => generated 124 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 10:42:24.342444 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ba80a4c-14c5-4aba-81f6-38a998b2c5df None admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 10:42:24.458389 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ba80a4c-14c5-4aba-81f6-38a998b2c5df None admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 10:42:24.467927 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=100 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 10:42:24.495587 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-637e5ac6-db98-49e2-b877-1b5b56732cf9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 10:42:24.496207 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-637e5ac6-db98-49e2-b877-1b5b56732cf9 None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:42:24.496492 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-637e5ac6-db98-49e2-b877-1b5b56732cf9 None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:42:24.496776 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-637e5ac6-db98-49e2-b877-1b5b56732cf9 None admin] PATH_INFO: `/v3/services` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:42:24.497414 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-637e5ac6-db98-49e2-b877-1b5b56732cf9 None admin] RBAC: Adding query filter params (type=network) {{(pid=80143) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 10:42:24.497720 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-637e5ac6-db98-49e2-b877-1b5b56732cf9 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 10:42:24.499119 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-637e5ac6-db98-49e2-b877-1b5b56732cf9 None admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 10:42:24.504686 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 381/778] 10.222.0.25 () {56 vars in 1112 bytes} [Sun Aug 7 10:42:24 2022] GET /identity/v3/services?type=network => generated 350 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 10:42:24.511237 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6443b16-6778-486d-b46c-b13c93b68db1 None admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 10:42:24.565338 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aef98c7b-6086-4026-9fde-bacfebb42e90 None None] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:42:24.565624 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aef98c7b-6086-4026-9fde-bacfebb42e90 None None] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:42:24.565900 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aef98c7b-6086-4026-9fde-bacfebb42e90 None None] PATH_INFO: `/` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:42:24.570980 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 382/779] 10.222.0.25 () {54 vars in 927 bytes} [Sun Aug 7 10:42:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 07 10:42:24.572704 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3940e602-72aa-49a8-b400-3f03f2e47188 None None] REQUEST_METHOD: `POST` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:42:24.573025 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3940e602-72aa-49a8-b400-3f03f2e47188 None None] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:42:24.573270 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3940e602-72aa-49a8-b400-3f03f2e47188 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:42:24.606510 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.auth.core [None req-3940e602-72aa-49a8-b400-3f03f2e47188 None None] MFA Rules not processed for user `bac4e6b76fe64217ac8fa44e71cffe6a`. Rule list: `[]` (Enabled: `True`). {{(pid=80143) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 10:42:24.607791 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-3940e602-72aa-49a8-b400-3f03f2e47188 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 10:42:24.631674 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6443b16-6778-486d-b46c-b13c93b68db1 None admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 10:42:24.700423 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f59d198-9563-4679-94b6-3bbeabab3f19 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 10:42:24.700961 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f59d198-9563-4679-94b6-3bbeabab3f19 None admin] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:42:24.701238 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f59d198-9563-4679-94b6-3bbeabab3f19 None admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:42:24.701505 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f59d198-9563-4679-94b6-3bbeabab3f19 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:42:24.702206 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f59d198-9563-4679-94b6-3bbeabab3f19 None admin] RBAC: Adding query filter params (interface=public, service_id=40159e7f266e4aa3adc8c157c11eab0c, region_id=RegionOne) {{(pid=80144) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 10:42:24.702513 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f59d198-9563-4679-94b6-3bbeabab3f19 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 10:42:24.703877 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f59d198-9563-4679-94b6-3bbeabab3f19 None admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 10:42:24.719956 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 398/780] 10.222.0.25 () {56 vars in 1251 bytes} [Sun Aug 7 10:42:24 2022] GET /identity/v3/endpoints?service_id=40159e7f266e4aa3adc8c157c11eab0c&interface=public®ion_id=RegionOne => generated 194 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 10:42:24.726735 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f59d198-9563-4679-94b6-3bbeabab3f19 None admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 10:42:24.731239 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=100 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 10:42:24.759861 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 383/781] 10.222.0.25 () {58 vars in 1024 bytes} [Sun Aug 7 10:42:24 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 188 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 07 10:42:24.800296 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3940e602-72aa-49a8-b400-3f03f2e47188 None None] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 10:42:24.867348 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f59d198-9563-4679-94b6-3bbeabab3f19 None admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 10:42:24.902105 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f47de83b-8b5b-4e55-a24c-f3f8ce708e69 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 10:42:24.902723 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f47de83b-8b5b-4e55-a24c-f3f8ce708e69 None admin] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:42:24.903011 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f47de83b-8b5b-4e55-a24c-f3f8ce708e69 None admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:42:24.903285 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f47de83b-8b5b-4e55-a24c-f3f8ce708e69 None admin] PATH_INFO: `/v3/services` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:42:24.903883 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f47de83b-8b5b-4e55-a24c-f3f8ce708e69 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 10:42:24.905229 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f47de83b-8b5b-4e55-a24c-f3f8ce708e69 None admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 10:42:24.910720 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 399/782] 10.222.0.25 () {56 vars in 1087 bytes} [Sun Aug 7 10:42:24 2022] GET /identity/v3/services => generated 772 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 10:42:25.033951 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3940e602-72aa-49a8-b400-3f03f2e47188 None None] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 10:42:25.035756 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=100 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 10:42:25.088116 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72bac9f8-3714-4d72-9440-f68fa7f786c4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 10:42:25.088877 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72bac9f8-3714-4d72-9440-f68fa7f786c4 None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:42:25.089161 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72bac9f8-3714-4d72-9440-f68fa7f786c4 None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:42:25.089439 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72bac9f8-3714-4d72-9440-f68fa7f786c4 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:42:25.105211 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72bac9f8-3714-4d72-9440-f68fa7f786c4 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 07 10:42:25.109948 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application [None req-72bac9f8-3714-4d72-9440-f68fa7f786c4 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 07 10:42:25.109948 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 07 10:42:25.109948 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 07 10:42:25.109948 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 07 10:42:25.109948 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 07 10:42:25.109948 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 07 10:42:25.109948 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 07 10:42:25.109948 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 07 10:42:25.109948 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 07 10:42:25.109948 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 07 10:42:25.109948 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 07 10:42:25.109948 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 07 10:42:25.109948 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 07 10:42:25.109948 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 07 10:42:25.109948 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 07 10:42:25.109948 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 07 10:42:25.110888 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 07 10:42:25.110888 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 07 10:42:25.110888 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 07 10:42:25.110888 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 07 10:42:25.110888 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 07 10:42:25.110888 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 07 10:42:25.110888 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 07 10:42:25.110888 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 07 10:42:25.110888 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 07 10:42:25.110888 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.get_or_create( Aug 07 10:42:25.110888 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 07 10:42:25.110888 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application with Lock( Aug 07 10:42:25.110888 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 07 10:42:25.110888 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self._enter() Aug 07 10:42:25.110888 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 07 10:42:25.110888 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 07 10:42:25.111747 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 07 10:42:25.111747 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.creator() Aug 07 10:42:25.111747 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 07 10:42:25.111747 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application created_value = creator( Aug 07 10:42:25.111747 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 07 10:42:25.111747 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 07 10:42:25.111747 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 07 10:42:25.111747 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 07 10:42:25.111747 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 07 10:42:25.111747 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 07 10:42:25.111747 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 07 10:42:25.111747 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application Aug 07 10:42:25.111747 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 384/783] 10.222.0.25 () {56 vars in 1096 bytes} [Sun Aug 7 10:42:24 2022] GET /identity/v3/roles/admin => generated 83 bytes in 311 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 07 10:42:25.119811 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f47de83b-8b5b-4e55-a24c-f3f8ce708e69 None admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 10:42:25.173718 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8086dd68-4617-4bc1-ac06-71267674c4dd None None] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:42:25.174023 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8086dd68-4617-4bc1-ac06-71267674c4dd None None] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:42:25.174382 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8086dd68-4617-4bc1-ac06-71267674c4dd None None] PATH_INFO: `/` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:42:25.176114 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 385/784] 10.222.0.25 () {54 vars in 927 bytes} [Sun Aug 7 10:42:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 07 10:42:25.180701 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-690ca931-2097-446a-ab5b-4f387a11576e None None] REQUEST_METHOD: `POST` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:42:25.180988 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-690ca931-2097-446a-ab5b-4f387a11576e None None] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:42:25.181347 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-690ca931-2097-446a-ab5b-4f387a11576e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:42:25.196449 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=100 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 10:42:25.207705 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.auth.core [None req-690ca931-2097-446a-ab5b-4f387a11576e None None] MFA Rules not processed for user `bac4e6b76fe64217ac8fa44e71cffe6a`. Rule list: `[]` (Enabled: `True`). {{(pid=80143) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 10:42:25.209373 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-690ca931-2097-446a-ab5b-4f387a11576e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 10:42:25.233058 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f47de83b-8b5b-4e55-a24c-f3f8ce708e69 None admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 10:42:25.268619 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c668c68-e167-4a4e-bcf0-1a083effdec0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 10:42:25.269228 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c668c68-e167-4a4e-bcf0-1a083effdec0 None admin] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:42:25.269506 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c668c68-e167-4a4e-bcf0-1a083effdec0 None admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:42:25.269777 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c668c68-e167-4a4e-bcf0-1a083effdec0 None admin] PATH_INFO: `/v3/roles` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:42:25.270433 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c668c68-e167-4a4e-bcf0-1a083effdec0 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80144) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 10:42:25.270734 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c668c68-e167-4a4e-bcf0-1a083effdec0 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 10:42:25.273032 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c668c68-e167-4a4e-bcf0-1a083effdec0 None admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 10:42:25.281088 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 400/785] 10.222.0.25 () {56 vars in 1099 bytes} [Sun Aug 7 10:42:25 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 07 10:42:25.287380 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c668c68-e167-4a4e-bcf0-1a083effdec0 None admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 10:42:25.326691 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 386/786] 10.222.0.25 () {58 vars in 1024 bytes} [Sun Aug 7 10:42:25 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 146 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 07 10:42:25.357154 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-690ca931-2097-446a-ab5b-4f387a11576e None None] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 10:42:25.466868 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c668c68-e167-4a4e-bcf0-1a083effdec0 None admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 10:42:25.490816 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=100 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 10:42:25.496382 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=100 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 10:42:25.504631 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85e83444-e29b-4df4-84d9-dfef0ab2cb38 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 10:42:25.505359 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85e83444-e29b-4df4-84d9-dfef0ab2cb38 None admin] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:42:25.505646 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85e83444-e29b-4df4-84d9-dfef0ab2cb38 None admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:42:25.505751 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-690ca931-2097-446a-ab5b-4f387a11576e None None] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 10:42:25.505912 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85e83444-e29b-4df4-84d9-dfef0ab2cb38 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:42:25.518641 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85e83444-e29b-4df4-84d9-dfef0ab2cb38 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 10:42:25.520164 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85e83444-e29b-4df4-84d9-dfef0ab2cb38 None admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 10:42:25.535786 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application [None req-85e83444-e29b-4df4-84d9-dfef0ab2cb38 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 07 10:42:25.535786 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 07 10:42:25.535786 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 07 10:42:25.535786 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 07 10:42:25.535786 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 07 10:42:25.535786 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 07 10:42:25.535786 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 07 10:42:25.535786 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 07 10:42:25.535786 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 07 10:42:25.535786 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 07 10:42:25.535786 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 07 10:42:25.535786 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 07 10:42:25.535786 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 07 10:42:25.535786 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 07 10:42:25.535786 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 07 10:42:25.535786 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 07 10:42:25.536763 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 07 10:42:25.536763 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 07 10:42:25.536763 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 07 10:42:25.536763 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 07 10:42:25.536763 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 07 10:42:25.536763 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self.get_or_create( Aug 07 10:42:25.536763 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 07 10:42:25.536763 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application with Lock( Aug 07 10:42:25.536763 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 07 10:42:25.536763 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self._enter() Aug 07 10:42:25.536763 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 07 10:42:25.536763 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 07 10:42:25.536763 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 07 10:42:25.536763 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self.creator() Aug 07 10:42:25.536763 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 07 10:42:25.536763 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application created_value = creator( Aug 07 10:42:25.536763 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 07 10:42:25.537684 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 07 10:42:25.537684 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 07 10:42:25.537684 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application Aug 07 10:42:25.541970 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 401/787] 10.222.0.25 () {56 vars in 1108 bytes} [Sun Aug 7 10:42:25 2022] GET /identity/v3/domains/Default => generated 87 bytes in 250 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 07 10:42:25.543223 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85e83444-e29b-4df4-84d9-dfef0ab2cb38 None admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 10:42:25.584647 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08d55f75-c7ce-4747-a393-a6364bd4ace1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 10:42:25.585306 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08d55f75-c7ce-4747-a393-a6364bd4ace1 None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:42:25.585595 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08d55f75-c7ce-4747-a393-a6364bd4ace1 None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:42:25.585925 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08d55f75-c7ce-4747-a393-a6364bd4ace1 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:42:25.600039 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08d55f75-c7ce-4747-a393-a6364bd4ace1 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 07 10:42:25.605049 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application [None req-08d55f75-c7ce-4747-a393-a6364bd4ace1 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 07 10:42:25.605049 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 07 10:42:25.605049 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 07 10:42:25.605049 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 07 10:42:25.605049 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 07 10:42:25.605049 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 07 10:42:25.605049 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 07 10:42:25.605049 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 07 10:42:25.605049 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 07 10:42:25.605049 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 07 10:42:25.605049 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 07 10:42:25.605049 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 07 10:42:25.605049 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 07 10:42:25.605049 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 07 10:42:25.605049 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 07 10:42:25.605049 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 07 10:42:25.605961 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 07 10:42:25.605961 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 07 10:42:25.605961 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 07 10:42:25.605961 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 07 10:42:25.605961 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 07 10:42:25.605961 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 07 10:42:25.605961 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 07 10:42:25.605961 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 07 10:42:25.605961 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 07 10:42:25.605961 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.get_or_create( Aug 07 10:42:25.605961 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 07 10:42:25.605961 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application with Lock( Aug 07 10:42:25.605961 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 07 10:42:25.605961 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self._enter() Aug 07 10:42:25.605961 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 07 10:42:25.605961 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 07 10:42:25.606780 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 07 10:42:25.606780 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.creator() Aug 07 10:42:25.606780 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 07 10:42:25.606780 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application created_value = creator( Aug 07 10:42:25.606780 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 07 10:42:25.606780 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 07 10:42:25.606780 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 07 10:42:25.606780 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 07 10:42:25.606780 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 07 10:42:25.606780 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 07 10:42:25.606780 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 07 10:42:25.606780 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application Aug 07 10:42:25.606780 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 387/788] 10.222.0.25 () {56 vars in 1096 bytes} [Sun Aug 7 10:42:25 2022] GET /identity/v3/roles/admin => generated 83 bytes in 250 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 07 10:42:25.615992 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08d55f75-c7ce-4747-a393-a6364bd4ace1 None admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 10:42:25.663550 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85e83444-e29b-4df4-84d9-dfef0ab2cb38 None admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 10:42:25.724672 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8461342-8a32-4a51-ae15-23bf5bc2b53b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 10:42:25.725288 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8461342-8a32-4a51-ae15-23bf5bc2b53b None admin] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:42:25.725616 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8461342-8a32-4a51-ae15-23bf5bc2b53b None admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:42:25.725897 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8461342-8a32-4a51-ae15-23bf5bc2b53b None admin] PATH_INFO: `/v3/domains` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:42:25.726540 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8461342-8a32-4a51-ae15-23bf5bc2b53b None admin] RBAC: Adding query filter params (name=Default) {{(pid=80144) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 10:42:25.726843 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8461342-8a32-4a51-ae15-23bf5bc2b53b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 10:42:25.728792 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8461342-8a32-4a51-ae15-23bf5bc2b53b None admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 10:42:25.749152 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08d55f75-c7ce-4747-a393-a6364bd4ace1 None admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 10:42:25.762013 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 402/789] 10.222.0.25 () {56 vars in 1109 bytes} [Sun Aug 7 10:42:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 219 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 10:42:25.767867 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8461342-8a32-4a51-ae15-23bf5bc2b53b None admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 10:42:25.784779 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98e86954-501e-4038-8d7e-a1ab2f71d921 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 10:42:25.785404 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98e86954-501e-4038-8d7e-a1ab2f71d921 None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:42:25.785685 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98e86954-501e-4038-8d7e-a1ab2f71d921 None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:42:25.785966 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98e86954-501e-4038-8d7e-a1ab2f71d921 None admin] PATH_INFO: `/v3/roles` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:42:25.786647 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98e86954-501e-4038-8d7e-a1ab2f71d921 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80143) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 10:42:25.786981 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98e86954-501e-4038-8d7e-a1ab2f71d921 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 10:42:25.788608 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98e86954-501e-4038-8d7e-a1ab2f71d921 None admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 10:42:25.797972 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 388/790] 10.222.0.25 () {56 vars in 1099 bytes} [Sun Aug 7 10:42:25 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 10:42:25.806726 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98e86954-501e-4038-8d7e-a1ab2f71d921 None admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 10:42:25.835286 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=100 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 10:42:25.839241 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=100 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 10:42:25.934968 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8461342-8a32-4a51-ae15-23bf5bc2b53b None admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 10:42:25.977322 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98e86954-501e-4038-8d7e-a1ab2f71d921 None admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 10:42:26.000235 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b543cbb9-eb6f-4cb8-8dd6-fa66d64a4ec3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 10:42:26.001042 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b543cbb9-eb6f-4cb8-8dd6-fa66d64a4ec3 None admin] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:42:26.001340 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b543cbb9-eb6f-4cb8-8dd6-fa66d64a4ec3 None admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:42:26.001630 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b543cbb9-eb6f-4cb8-8dd6-fa66d64a4ec3 None admin] PATH_INFO: `/v3/users/swift` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:42:26.016276 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64815a54-9a77-43d3-a5fe-fe9020e2af84 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 10:42:26.016554 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b543cbb9-eb6f-4cb8-8dd6-fa66d64a4ec3 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 10:42:26.017410 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64815a54-9a77-43d3-a5fe-fe9020e2af84 None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:42:26.017911 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64815a54-9a77-43d3-a5fe-fe9020e2af84 None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:42:26.018017 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b543cbb9-eb6f-4cb8-8dd6-fa66d64a4ec3 None admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 10:42:26.018584 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64815a54-9a77-43d3-a5fe-fe9020e2af84 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:42:26.030082 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64815a54-9a77-43d3-a5fe-fe9020e2af84 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 10:42:26.031855 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64815a54-9a77-43d3-a5fe-fe9020e2af84 None admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 10:42:26.039880 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application [None req-b543cbb9-eb6f-4cb8-8dd6-fa66d64a4ec3 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Aug 07 10:42:26.039880 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 07 10:42:26.039880 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 07 10:42:26.039880 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 07 10:42:26.039880 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 07 10:42:26.039880 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 07 10:42:26.039880 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 07 10:42:26.039880 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 07 10:42:26.039880 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 07 10:42:26.039880 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 07 10:42:26.039880 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 07 10:42:26.039880 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 07 10:42:26.039880 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 07 10:42:26.039880 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 07 10:42:26.039880 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 07 10:42:26.039880 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 07 10:42:26.040969 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 07 10:42:26.040969 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 07 10:42:26.040969 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 07 10:42:26.040969 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 07 10:42:26.040969 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 07 10:42:26.040969 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 07 10:42:26.040969 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 07 10:42:26.040969 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 07 10:42:26.040969 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 07 10:42:26.040969 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self.get_or_create( Aug 07 10:42:26.040969 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 07 10:42:26.040969 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application with Lock( Aug 07 10:42:26.040969 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 07 10:42:26.040969 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self._enter() Aug 07 10:42:26.040969 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 07 10:42:26.040969 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 07 10:42:26.040969 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 07 10:42:26.040969 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self.creator() Aug 07 10:42:26.041950 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 07 10:42:26.041950 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application created_value = creator( Aug 07 10:42:26.041950 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 07 10:42:26.041950 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 07 10:42:26.041950 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 07 10:42:26.041950 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 07 10:42:26.041950 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 07 10:42:26.041950 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 07 10:42:26.041950 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Aug 07 10:42:26.041950 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application Aug 07 10:42:26.047382 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 403/791] 10.222.0.25 () {56 vars in 1096 bytes} [Sun Aug 7 10:42:25 2022] GET /identity/v3/users/swift => generated 83 bytes in 280 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 07 10:42:26.049845 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application [None req-64815a54-9a77-43d3-a5fe-fe9020e2af84 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 07 10:42:26.049845 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 07 10:42:26.049845 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 07 10:42:26.049845 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 07 10:42:26.049845 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 07 10:42:26.049845 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 07 10:42:26.049845 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 07 10:42:26.049845 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 07 10:42:26.049845 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 07 10:42:26.049845 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 07 10:42:26.049845 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 07 10:42:26.049845 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 07 10:42:26.049845 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 07 10:42:26.049845 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 07 10:42:26.049845 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 07 10:42:26.049845 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 07 10:42:26.050760 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 07 10:42:26.050760 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 07 10:42:26.050760 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 07 10:42:26.050760 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 07 10:42:26.050760 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 07 10:42:26.050760 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.get_or_create( Aug 07 10:42:26.050760 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 07 10:42:26.050760 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application with Lock( Aug 07 10:42:26.050760 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 07 10:42:26.050760 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self._enter() Aug 07 10:42:26.050760 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 07 10:42:26.050760 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 07 10:42:26.050760 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 07 10:42:26.050760 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.creator() Aug 07 10:42:26.050760 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 07 10:42:26.050760 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application created_value = creator( Aug 07 10:42:26.050760 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 07 10:42:26.051682 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 07 10:42:26.051682 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 07 10:42:26.051682 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application Aug 07 10:42:26.052678 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b543cbb9-eb6f-4cb8-8dd6-fa66d64a4ec3 None admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 10:42:26.054040 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 389/792] 10.222.0.25 () {56 vars in 1108 bytes} [Sun Aug 7 10:42:25 2022] GET /identity/v3/domains/Default => generated 87 bytes in 248 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 07 10:42:26.059864 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64815a54-9a77-43d3-a5fe-fe9020e2af84 None admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 10:42:26.186755 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=100 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 10:42:26.215270 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=100 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 10:42:26.218256 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b543cbb9-eb6f-4cb8-8dd6-fa66d64a4ec3 None admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 10:42:26.256571 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64815a54-9a77-43d3-a5fe-fe9020e2af84 None admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 10:42:26.285132 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5286b2e-1461-4ea9-9d44-bcaeabc64f37 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 10:42:26.285738 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5286b2e-1461-4ea9-9d44-bcaeabc64f37 None admin] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:42:26.286015 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5286b2e-1461-4ea9-9d44-bcaeabc64f37 None admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:42:26.286281 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5286b2e-1461-4ea9-9d44-bcaeabc64f37 None admin] PATH_INFO: `/v3/users` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:42:26.287032 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5286b2e-1461-4ea9-9d44-bcaeabc64f37 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80144) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 10:42:26.287339 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5286b2e-1461-4ea9-9d44-bcaeabc64f37 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 10:42:26.288738 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5286b2e-1461-4ea9-9d44-bcaeabc64f37 None admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 10:42:26.315557 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 404/793] 10.222.0.25 () {56 vars in 1135 bytes} [Sun Aug 7 10:42:26 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 263 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 07 10:42:26.322311 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5286b2e-1461-4ea9-9d44-bcaeabc64f37 None admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 10:42:26.336816 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70182034-2a7c-4a02-b125-a94eb803b45d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 10:42:26.338114 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70182034-2a7c-4a02-b125-a94eb803b45d None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:42:26.338361 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70182034-2a7c-4a02-b125-a94eb803b45d None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:42:26.338605 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70182034-2a7c-4a02-b125-a94eb803b45d None admin] PATH_INFO: `/v3/domains` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:42:26.339270 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70182034-2a7c-4a02-b125-a94eb803b45d None admin] RBAC: Adding query filter params (name=Default) {{(pid=80143) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 10:42:26.339567 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70182034-2a7c-4a02-b125-a94eb803b45d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 10:42:26.340934 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70182034-2a7c-4a02-b125-a94eb803b45d None admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 10:42:26.373445 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 390/794] 10.222.0.25 () {56 vars in 1109 bytes} [Sun Aug 7 10:42:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 313 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 10:42:26.389235 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70182034-2a7c-4a02-b125-a94eb803b45d None admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 10:42:26.535119 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5286b2e-1461-4ea9-9d44-bcaeabc64f37 None admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 10:42:26.570310 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70182034-2a7c-4a02-b125-a94eb803b45d None admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 10:42:26.579921 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6069544-b423-4200-9358-909c4e3d7436 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 10:42:26.580911 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6069544-b423-4200-9358-909c4e3d7436 None admin] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:42:26.581444 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6069544-b423-4200-9358-909c4e3d7436 None admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:42:26.581919 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6069544-b423-4200-9358-909c4e3d7436 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:42:26.593475 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6069544-b423-4200-9358-909c4e3d7436 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 10:42:26.595309 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6069544-b423-4200-9358-909c4e3d7436 None admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 10:42:26.596255 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=100 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 10:42:26.605853 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb94e8e5-3531-4391-ac17-8c0e8a3857ba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 10:42:26.606847 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb94e8e5-3531-4391-ac17-8c0e8a3857ba None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:42:26.607695 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb94e8e5-3531-4391-ac17-8c0e8a3857ba None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:42:26.607695 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb94e8e5-3531-4391-ac17-8c0e8a3857ba None admin] PATH_INFO: `/v3/users/nova` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:42:26.614879 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application [None req-a6069544-b423-4200-9358-909c4e3d7436 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 07 10:42:26.614879 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 07 10:42:26.614879 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 07 10:42:26.614879 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 07 10:42:26.614879 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 07 10:42:26.614879 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 07 10:42:26.614879 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 07 10:42:26.614879 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 07 10:42:26.614879 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 07 10:42:26.614879 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 07 10:42:26.614879 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 07 10:42:26.614879 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 07 10:42:26.614879 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 07 10:42:26.614879 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 07 10:42:26.614879 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 07 10:42:26.614879 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 07 10:42:26.615871 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 07 10:42:26.615871 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 07 10:42:26.615871 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 07 10:42:26.615871 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 07 10:42:26.615871 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 07 10:42:26.615871 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self.get_or_create( Aug 07 10:42:26.615871 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 07 10:42:26.615871 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application with Lock( Aug 07 10:42:26.615871 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 07 10:42:26.615871 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self._enter() Aug 07 10:42:26.615871 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 07 10:42:26.615871 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 07 10:42:26.615871 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 07 10:42:26.615871 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self.creator() Aug 07 10:42:26.615871 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 07 10:42:26.615871 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application created_value = creator( Aug 07 10:42:26.615871 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 07 10:42:26.616755 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 07 10:42:26.616755 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 07 10:42:26.616755 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application Aug 07 10:42:26.616925 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb94e8e5-3531-4391-ac17-8c0e8a3857ba None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 10:42:26.617679 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb94e8e5-3531-4391-ac17-8c0e8a3857ba None admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 10:42:26.619030 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 405/795] 10.222.0.25 () {56 vars in 1108 bytes} [Sun Aug 7 10:42:26 2022] GET /identity/v3/domains/Default => generated 87 bytes in 297 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 07 10:42:26.620575 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a3e69ff-2e9b-4b99-b110-de4cd208e957 None None] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:42:26.621007 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a3e69ff-2e9b-4b99-b110-de4cd208e957 None None] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:42:26.621533 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a3e69ff-2e9b-4b99-b110-de4cd208e957 None None] PATH_INFO: `/` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:42:26.626959 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 406/796] 10.222.0.25 () {54 vars in 927 bytes} [Sun Aug 7 10:42:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 07 10:42:26.627978 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a3e69ff-2e9b-4b99-b110-de4cd208e957 None None] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 10:42:26.631456 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application [None req-cb94e8e5-3531-4391-ac17-8c0e8a3857ba None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Aug 07 10:42:26.631456 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 07 10:42:26.631456 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 07 10:42:26.631456 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 07 10:42:26.631456 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 07 10:42:26.631456 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 07 10:42:26.631456 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 07 10:42:26.631456 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 07 10:42:26.631456 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 07 10:42:26.631456 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 07 10:42:26.631456 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 07 10:42:26.631456 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 07 10:42:26.631456 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 07 10:42:26.631456 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 07 10:42:26.631456 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 07 10:42:26.631456 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 07 10:42:26.632366 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 07 10:42:26.632366 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 07 10:42:26.632366 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 07 10:42:26.632366 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 07 10:42:26.632366 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 07 10:42:26.632366 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 07 10:42:26.632366 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 07 10:42:26.632366 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 07 10:42:26.632366 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 07 10:42:26.632366 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.get_or_create( Aug 07 10:42:26.632366 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 07 10:42:26.632366 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application with Lock( Aug 07 10:42:26.632366 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 07 10:42:26.632366 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self._enter() Aug 07 10:42:26.632366 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 07 10:42:26.632366 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 07 10:42:26.632366 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 07 10:42:26.632366 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.creator() Aug 07 10:42:26.633330 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 07 10:42:26.633330 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application created_value = creator( Aug 07 10:42:26.633330 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 07 10:42:26.633330 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 07 10:42:26.633330 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 07 10:42:26.633330 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 07 10:42:26.633330 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 07 10:42:26.633330 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 07 10:42:26.633330 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Aug 07 10:42:26.633330 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application Aug 07 10:42:26.633330 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 391/797] 10.222.0.25 () {56 vars in 1093 bytes} [Sun Aug 7 10:42:26 2022] GET /identity/v3/users/nova => generated 82 bytes in 244 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 07 10:42:26.639072 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af5a0856-4677-4d5d-ae2c-5b6d46f4ecf6 None None] REQUEST_METHOD: `POST` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:42:26.639357 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af5a0856-4677-4d5d-ae2c-5b6d46f4ecf6 None None] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:42:26.639633 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af5a0856-4677-4d5d-ae2c-5b6d46f4ecf6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:42:26.646021 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=100 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 10:42:26.658493 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.auth.core [None req-af5a0856-4677-4d5d-ae2c-5b6d46f4ecf6 None None] MFA Rules not processed for user `bac4e6b76fe64217ac8fa44e71cffe6a`. Rule list: `[]` (Enabled: `True`). {{(pid=80143) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 10:42:26.659726 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-af5a0856-4677-4d5d-ae2c-5b6d46f4ecf6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 10:42:26.735038 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 392/798] 10.222.0.25 () {58 vars in 1024 bytes} [Sun Aug 7 10:42:26 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 07 10:42:26.735038 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af5a0856-4677-4d5d-ae2c-5b6d46f4ecf6 None None] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 10:42:26.756118 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a3e69ff-2e9b-4b99-b110-de4cd208e957 None None] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 10:42:26.794241 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ebc6215-38c4-4667-b9b4-4b5decb3a8b2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 10:42:26.794989 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ebc6215-38c4-4667-b9b4-4b5decb3a8b2 None admin] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:42:26.795327 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ebc6215-38c4-4667-b9b4-4b5decb3a8b2 None admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:42:26.795606 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ebc6215-38c4-4667-b9b4-4b5decb3a8b2 None admin] PATH_INFO: `/v3/domains` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:42:26.796249 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ebc6215-38c4-4667-b9b4-4b5decb3a8b2 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80144) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 10:42:26.796555 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ebc6215-38c4-4667-b9b4-4b5decb3a8b2 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 10:42:26.798249 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ebc6215-38c4-4667-b9b4-4b5decb3a8b2 None admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 10:42:26.815580 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 407/799] 10.222.0.25 () {56 vars in 1109 bytes} [Sun Aug 7 10:42:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 10:42:26.816995 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd7bf207-139d-43f6-a63d-4d9313d005d5 None None] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:42:26.817287 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd7bf207-139d-43f6-a63d-4d9313d005d5 None None] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:42:26.817568 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd7bf207-139d-43f6-a63d-4d9313d005d5 None None] PATH_INFO: `/` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:42:26.818969 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 408/800] 10.222.0.25 () {54 vars in 927 bytes} [Sun Aug 7 10:42:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 07 10:42:26.819520 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd7bf207-139d-43f6-a63d-4d9313d005d5 None None] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 10:42:26.843493 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af5a0856-4677-4d5d-ae2c-5b6d46f4ecf6 None None] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 10:42:26.861345 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=100 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 10:42:26.880866 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dc4e18f-a26c-4104-86b1-7bedf0a3c127 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 10:42:26.881513 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dc4e18f-a26c-4104-86b1-7bedf0a3c127 None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:42:26.881741 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dc4e18f-a26c-4104-86b1-7bedf0a3c127 None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:42:26.882009 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dc4e18f-a26c-4104-86b1-7bedf0a3c127 None admin] PATH_INFO: `/v3/users` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:42:26.882730 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dc4e18f-a26c-4104-86b1-7bedf0a3c127 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80143) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 10:42:26.883060 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dc4e18f-a26c-4104-86b1-7bedf0a3c127 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 10:42:26.884584 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dc4e18f-a26c-4104-86b1-7bedf0a3c127 None admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 10:42:26.899448 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=100 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 10:42:26.903791 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 393/801] 10.222.0.25 () {56 vars in 1133 bytes} [Sun Aug 7 10:42:26 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 10:42:26.904839 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dc4e18f-a26c-4104-86b1-7bedf0a3c127 None admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 10:42:26.937969 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd7bf207-139d-43f6-a63d-4d9313d005d5 None None] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 10:42:26.976940 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8870069c-3d93-473c-b856-717aa9fe5161 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 10:42:26.977581 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8870069c-3d93-473c-b856-717aa9fe5161 None admin] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:42:26.977969 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8870069c-3d93-473c-b856-717aa9fe5161 None admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:42:26.978892 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8870069c-3d93-473c-b856-717aa9fe5161 None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:42:26.979513 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8870069c-3d93-473c-b856-717aa9fe5161 None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 10:42:26.981653 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8870069c-3d93-473c-b856-717aa9fe5161 None admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 10:42:26.993224 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application [None req-8870069c-3d93-473c-b856-717aa9fe5161 None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Aug 07 10:42:26.993224 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 07 10:42:26.993224 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 07 10:42:26.993224 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 07 10:42:26.993224 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 07 10:42:26.993224 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 07 10:42:26.993224 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 07 10:42:26.993224 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 07 10:42:26.993224 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Aug 07 10:42:26.993224 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application Aug 07 10:42:26.993224 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 07 10:42:26.993224 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application Aug 07 10:42:26.993224 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 07 10:42:26.993224 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 07 10:42:26.993224 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 07 10:42:26.993224 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 07 10:42:26.993224 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 07 10:42:26.994788 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 07 10:42:26.994788 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 07 10:42:26.994788 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 07 10:42:26.994788 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 07 10:42:26.994788 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 07 10:42:26.994788 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 07 10:42:26.994788 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 07 10:42:26.994788 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 07 10:42:26.994788 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 07 10:42:26.994788 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 07 10:42:26.994788 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 07 10:42:26.994788 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 07 10:42:26.994788 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 07 10:42:26.994788 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 07 10:42:26.994788 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 07 10:42:26.994788 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self.get_or_create( Aug 07 10:42:26.995676 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 07 10:42:26.995676 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application with Lock( Aug 07 10:42:26.995676 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 07 10:42:26.995676 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self._enter() Aug 07 10:42:26.995676 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 07 10:42:26.995676 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 07 10:42:26.995676 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 07 10:42:26.995676 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self.creator() Aug 07 10:42:26.995676 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 07 10:42:26.995676 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application created_value = creator( Aug 07 10:42:26.995676 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 07 10:42:26.995676 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 07 10:42:26.995676 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Aug 07 10:42:26.995676 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application Aug 07 10:42:26.999079 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 409/802] 10.222.0.25 () {56 vars in 1129 bytes} [Sun Aug 7 10:42:26 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 07 10:42:27.000395 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaeb4805-fa39-4351-8d9f-cf1f5c9245ad None None] REQUEST_METHOD: `POST` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:42:27.000686 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaeb4805-fa39-4351-8d9f-cf1f5c9245ad None None] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:42:27.000966 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaeb4805-fa39-4351-8d9f-cf1f5c9245ad None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:42:27.019182 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.auth.core [None req-eaeb4805-fa39-4351-8d9f-cf1f5c9245ad None None] MFA Rules not processed for user `bac4e6b76fe64217ac8fa44e71cffe6a`. Rule list: `[]` (Enabled: `True`). {{(pid=80144) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 10:42:27.020378 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.fernet_utils [None req-eaeb4805-fa39-4351-8d9f-cf1f5c9245ad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80144) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 10:42:27.021703 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dc4e18f-a26c-4104-86b1-7bedf0a3c127 None admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 10:42:27.059935 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07321d93-5f00-49fc-8eaa-e92c0bafab92 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 10:42:27.060611 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07321d93-5f00-49fc-8eaa-e92c0bafab92 None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:42:27.060891 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07321d93-5f00-49fc-8eaa-e92c0bafab92 None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:42:27.061166 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07321d93-5f00-49fc-8eaa-e92c0bafab92 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:42:27.068850 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07321d93-5f00-49fc-8eaa-e92c0bafab92 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 10:42:27.070680 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07321d93-5f00-49fc-8eaa-e92c0bafab92 None admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 10:42:27.083018 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application [None req-07321d93-5f00-49fc-8eaa-e92c0bafab92 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 07 10:42:27.083018 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 07 10:42:27.083018 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 07 10:42:27.083018 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 07 10:42:27.083018 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 07 10:42:27.083018 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 07 10:42:27.083018 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 07 10:42:27.083018 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 07 10:42:27.083018 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 07 10:42:27.083018 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 07 10:42:27.083018 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 07 10:42:27.083018 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 07 10:42:27.083018 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 07 10:42:27.083018 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 07 10:42:27.083018 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 07 10:42:27.084108 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 07 10:42:27.084108 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 07 10:42:27.084108 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 07 10:42:27.084108 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 07 10:42:27.084108 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 07 10:42:27.084108 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 07 10:42:27.084108 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.get_or_create( Aug 07 10:42:27.084108 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 07 10:42:27.084108 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application with Lock( Aug 07 10:42:27.084108 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 07 10:42:27.084108 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self._enter() Aug 07 10:42:27.084108 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 07 10:42:27.084108 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 07 10:42:27.084108 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 07 10:42:27.084108 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.creator() Aug 07 10:42:27.084108 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 07 10:42:27.084108 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application created_value = creator( Aug 07 10:42:27.085212 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 07 10:42:27.085212 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 07 10:42:27.085212 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 07 10:42:27.085212 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 07 10:42:27.085212 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 07 10:42:27.085212 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 07 10:42:27.085212 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 07 10:42:27.085212 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application Aug 07 10:42:27.087072 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 394/803] 10.222.0.25 () {56 vars in 1111 bytes} [Sun Aug 7 10:42:26 2022] GET /identity/v3/projects/service => generated 88 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 07 10:42:27.088875 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07321d93-5f00-49fc-8eaa-e92c0bafab92 None admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 10:42:27.096821 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 410/804] 10.222.0.25 () {58 vars in 1024 bytes} [Sun Aug 7 10:42:26 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 07 10:42:27.097831 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaeb4805-fa39-4351-8d9f-cf1f5c9245ad None None] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 10:42:27.126295 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=100 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 10:42:27.172354 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=100 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 10:42:27.199442 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07321d93-5f00-49fc-8eaa-e92c0bafab92 None admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 10:42:27.214150 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaeb4805-fa39-4351-8d9f-cf1f5c9245ad None None] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 10:42:27.234748 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c81f3445-6098-4e8d-a0ba-0837f64af1ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 10:42:27.235551 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c81f3445-6098-4e8d-a0ba-0837f64af1ce None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:42:27.235837 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c81f3445-6098-4e8d-a0ba-0837f64af1ce None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:42:27.236110 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c81f3445-6098-4e8d-a0ba-0837f64af1ce None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:42:27.250849 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b7e0b57-e19e-4260-9c97-1b5142922cc7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 10:42:27.251494 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b7e0b57-e19e-4260-9c97-1b5142922cc7 None admin] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:42:27.251777 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b7e0b57-e19e-4260-9c97-1b5142922cc7 None admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:42:27.252054 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b7e0b57-e19e-4260-9c97-1b5142922cc7 None admin] PATH_INFO: `/v3/services` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:42:27.252683 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b7e0b57-e19e-4260-9c97-1b5142922cc7 None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=80144) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 10:42:27.252994 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b7e0b57-e19e-4260-9c97-1b5142922cc7 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 10:42:27.254345 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b7e0b57-e19e-4260-9c97-1b5142922cc7 None admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 10:42:27.264510 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 411/805] 10.222.0.25 () {56 vars in 1124 bytes} [Sun Aug 7 10:42:27 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 10:42:27.264510 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b7e0b57-e19e-4260-9c97-1b5142922cc7 None admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 10:42:27.309159 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c81f3445-6098-4e8d-a0ba-0837f64af1ce None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 10:42:27.310607 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c81f3445-6098-4e8d-a0ba-0837f64af1ce None admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 10:42:27.344147 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=100 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 10:42:27.377359 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b7e0b57-e19e-4260-9c97-1b5142922cc7 None admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 10:42:27.402637 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=100 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 10:42:27.411745 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d46109b-9d54-4d52-b49d-b640d1ed41ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 10:42:27.411991 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 395/806] 10.222.0.25 () {58 vars in 1282 bytes} [Sun Aug 7 10:42:27 2022] GET /identity/v3/auth/tokens => generated 1002 bytes in 324 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 07 10:42:27.412364 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d46109b-9d54-4d52-b49d-b640d1ed41ac None admin] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:42:27.412649 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d46109b-9d54-4d52-b49d-b640d1ed41ac None admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:42:27.412925 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d46109b-9d54-4d52-b49d-b640d1ed41ac None admin] PATH_INFO: `/v3/projects` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:42:27.413443 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c81f3445-6098-4e8d-a0ba-0837f64af1ce None admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 10:42:27.413636 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d46109b-9d54-4d52-b49d-b640d1ed41ac None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80144) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 10:42:27.413923 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d46109b-9d54-4d52-b49d-b640d1ed41ac None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 10:42:27.416618 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d46109b-9d54-4d52-b49d-b640d1ed41ac None admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 10:42:27.428379 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 412/807] 10.222.0.25 () {56 vars in 1148 bytes} [Sun Aug 7 10:42:27 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 10:42:27.429208 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d46109b-9d54-4d52-b49d-b640d1ed41ac None admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 10:42:27.514240 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c81f3445-6098-4e8d-a0ba-0837f64af1ce None admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 10:42:27.528468 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d46109b-9d54-4d52-b49d-b640d1ed41ac None admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 10:42:27.548513 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f97b46d5-a60a-4e9f-894a-064e3e4206e7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 10:42:27.549190 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f97b46d5-a60a-4e9f-894a-064e3e4206e7 None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:42:27.549473 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f97b46d5-a60a-4e9f-894a-064e3e4206e7 None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:42:27.549744 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f97b46d5-a60a-4e9f-894a-064e3e4206e7 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:42:27.551865 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=100 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 10:42:27.561884 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f97b46d5-a60a-4e9f-894a-064e3e4206e7 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 07 10:42:27.562824 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f9780f6-07ec-4892-8d3a-8fc88bd86bc0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 10:42:27.563465 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f9780f6-07ec-4892-8d3a-8fc88bd86bc0 None admin] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:42:27.563742 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f9780f6-07ec-4892-8d3a-8fc88bd86bc0 None admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:42:27.564013 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f9780f6-07ec-4892-8d3a-8fc88bd86bc0 None admin] PATH_INFO: `/v3/services` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:42:27.564634 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f9780f6-07ec-4892-8d3a-8fc88bd86bc0 None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=80144) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 10:42:27.564934 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f9780f6-07ec-4892-8d3a-8fc88bd86bc0 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 10:42:27.566268 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f9780f6-07ec-4892-8d3a-8fc88bd86bc0 None admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 10:42:27.566575 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application [None req-f97b46d5-a60a-4e9f-894a-064e3e4206e7 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 07 10:42:27.566575 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 07 10:42:27.566575 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 07 10:42:27.566575 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 07 10:42:27.566575 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 07 10:42:27.566575 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 07 10:42:27.566575 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 07 10:42:27.566575 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 07 10:42:27.566575 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 07 10:42:27.566575 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 07 10:42:27.566575 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 07 10:42:27.566575 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 07 10:42:27.566575 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 07 10:42:27.566575 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 07 10:42:27.566575 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 07 10:42:27.566575 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 07 10:42:27.567592 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 07 10:42:27.567592 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 07 10:42:27.567592 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 07 10:42:27.567592 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 07 10:42:27.567592 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 07 10:42:27.567592 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 07 10:42:27.567592 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 07 10:42:27.567592 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 07 10:42:27.567592 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 07 10:42:27.567592 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.get_or_create( Aug 07 10:42:27.567592 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 07 10:42:27.567592 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application with Lock( Aug 07 10:42:27.567592 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 07 10:42:27.567592 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self._enter() Aug 07 10:42:27.567592 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 07 10:42:27.567592 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 07 10:42:27.568436 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 07 10:42:27.568436 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.creator() Aug 07 10:42:27.568436 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 07 10:42:27.568436 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application created_value = creator( Aug 07 10:42:27.568436 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 07 10:42:27.568436 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 07 10:42:27.568436 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 07 10:42:27.568436 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 07 10:42:27.568436 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 07 10:42:27.568436 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 07 10:42:27.568436 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 07 10:42:27.568436 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application Aug 07 10:42:27.568436 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 396/808] 10.222.0.25 () {56 vars in 1102 bytes} [Sun Aug 7 10:42:27 2022] GET /identity/v3/roles/service => generated 85 bytes in 155 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 07 10:42:27.569121 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-366a1a68-a7c0-45e0-8283-67b21666d7a4 None None] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:42:27.569121 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-366a1a68-a7c0-45e0-8283-67b21666d7a4 None None] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:42:27.569810 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-366a1a68-a7c0-45e0-8283-67b21666d7a4 None None] PATH_INFO: `/` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:42:27.570706 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 397/809] 10.222.0.25 () {54 vars in 927 bytes} [Sun Aug 7 10:42:27 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 07 10:42:27.571331 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-366a1a68-a7c0-45e0-8283-67b21666d7a4 None None] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 10:42:27.579110 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 413/810] 10.222.0.25 () {56 vars in 1124 bytes} [Sun Aug 7 10:42:27 2022] GET /identity/v3/services?type=block-storage => generated 367 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 10:42:27.579110 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f9780f6-07ec-4892-8d3a-8fc88bd86bc0 None admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 10:42:27.629973 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=100 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 10:42:27.674087 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-366a1a68-a7c0-45e0-8283-67b21666d7a4 None None] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 10:42:27.676318 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f9780f6-07ec-4892-8d3a-8fc88bd86bc0 None admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 10:42:27.707202 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b7f94c6-ade5-41a9-85a0-b938e9dac4a8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 10:42:27.707833 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b7f94c6-ade5-41a9-85a0-b938e9dac4a8 None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:42:27.708116 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b7f94c6-ade5-41a9-85a0-b938e9dac4a8 None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:42:27.708392 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b7f94c6-ade5-41a9-85a0-b938e9dac4a8 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:42:27.710118 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eae98029-33d1-4103-9d22-01533c246e34 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 10:42:27.710782 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eae98029-33d1-4103-9d22-01533c246e34 None admin] REQUEST_METHOD: `PUT` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:42:27.711098 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eae98029-33d1-4103-9d22-01533c246e34 None admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:42:27.711375 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eae98029-33d1-4103-9d22-01533c246e34 None admin] PATH_INFO: `/v3/projects/6de1f3cac02c4bcdb05dd2337176033d/users/69490ea2a9a340bf91aca38bf5655b45/roles/81b1f1bcee2a473088fbbd327c1c5b23` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:42:27.713823 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eae98029-33d1-4103-9d22-01533c246e34 None admin] RBAC: Authorizing `identity:create_grant(project_id=6de1f3cac02c4bcdb05dd2337176033d, user_id=69490ea2a9a340bf91aca38bf5655b45, role_id=81b1f1bcee2a473088fbbd327c1c5b23)` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 10:42:27.715269 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eae98029-33d1-4103-9d22-01533c246e34 None admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 10:42:27.719465 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b7f94c6-ade5-41a9-85a0-b938e9dac4a8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 10:42:27.721268 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b7f94c6-ade5-41a9-85a0-b938e9dac4a8 None admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 10:42:27.728745 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 414/811] 10.222.0.25 () {58 vars in 1439 bytes} [Sun Aug 7 10:42:27 2022] PUT /identity/v3/projects/6de1f3cac02c4bcdb05dd2337176033d/users/69490ea2a9a340bf91aca38bf5655b45/roles/81b1f1bcee2a473088fbbd327c1c5b23 => generated 0 bytes in 154 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 07 10:42:27.729923 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1af36b8-4291-489c-9469-dc52eb62e63b None None] REQUEST_METHOD: `POST` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:42:27.730210 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1af36b8-4291-489c-9469-dc52eb62e63b None None] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:42:27.730488 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1af36b8-4291-489c-9469-dc52eb62e63b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:42:27.745496 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application [None req-7b7f94c6-ade5-41a9-85a0-b938e9dac4a8 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 07 10:42:27.745496 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 07 10:42:27.745496 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 07 10:42:27.745496 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 07 10:42:27.745496 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 07 10:42:27.745496 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 07 10:42:27.745496 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 07 10:42:27.745496 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 07 10:42:27.745496 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 07 10:42:27.745496 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 07 10:42:27.745496 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 07 10:42:27.745496 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 07 10:42:27.745496 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 07 10:42:27.745496 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 07 10:42:27.745496 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 07 10:42:27.745496 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 07 10:42:27.746725 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 07 10:42:27.746725 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 07 10:42:27.746725 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 07 10:42:27.746725 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 07 10:42:27.746725 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 07 10:42:27.746725 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.get_or_create( Aug 07 10:42:27.746725 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 07 10:42:27.746725 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application with Lock( Aug 07 10:42:27.746725 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 07 10:42:27.746725 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self._enter() Aug 07 10:42:27.746725 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 07 10:42:27.746725 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 07 10:42:27.746725 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 07 10:42:27.746725 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.creator() Aug 07 10:42:27.746725 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 07 10:42:27.746725 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application created_value = creator( Aug 07 10:42:27.746725 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 07 10:42:27.747828 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 07 10:42:27.747828 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 07 10:42:27.747828 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application Aug 07 10:42:27.747828 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 398/812] 10.222.0.25 () {56 vars in 1108 bytes} [Sun Aug 7 10:42:27 2022] GET /identity/v3/domains/Default => generated 87 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 07 10:42:27.747828 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b7f94c6-ade5-41a9-85a0-b938e9dac4a8 None admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 10:42:27.756755 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.auth.core [None req-f1af36b8-4291-489c-9469-dc52eb62e63b None None] MFA Rules not processed for user `bac4e6b76fe64217ac8fa44e71cffe6a`. Rule list: `[]` (Enabled: `True`). {{(pid=80144) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 10:42:27.758029 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.fernet_utils [None req-f1af36b8-4291-489c-9469-dc52eb62e63b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80144) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 10:42:27.803349 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=100 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 10:42:27.846959 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 415/813] 10.222.0.25 () {58 vars in 1024 bytes} [Sun Aug 7 10:42:27 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 07 10:42:27.846959 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1af36b8-4291-489c-9469-dc52eb62e63b None None] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 10:42:27.881593 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b7f94c6-ade5-41a9-85a0-b938e9dac4a8 None admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 10:42:27.899411 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 10:42:27.923152 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-339fcc85-e159-4126-a5d3-eab1f5c54ef3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 10:42:27.923768 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-339fcc85-e159-4126-a5d3-eab1f5c54ef3 None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:42:27.924050 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-339fcc85-e159-4126-a5d3-eab1f5c54ef3 None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:42:27.924326 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-339fcc85-e159-4126-a5d3-eab1f5c54ef3 None admin] PATH_INFO: `/v3/roles` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:42:27.925250 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-339fcc85-e159-4126-a5d3-eab1f5c54ef3 None admin] RBAC: Adding query filter params (name=service) {{(pid=80143) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 10:42:27.925555 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-339fcc85-e159-4126-a5d3-eab1f5c54ef3 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 10:42:27.927833 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-339fcc85-e159-4126-a5d3-eab1f5c54ef3 None admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 10:42:27.943494 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 399/814] 10.222.0.25 () {56 vars in 1103 bytes} [Sun Aug 7 10:42:27 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 10:42:27.943494 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-339fcc85-e159-4126-a5d3-eab1f5c54ef3 None admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 10:42:27.960424 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1af36b8-4291-489c-9469-dc52eb62e63b None None] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 10:42:27.998202 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-907d1fa9-91d6-401d-a5d4-d3d48cae50ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 10:42:27.998843 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-907d1fa9-91d6-401d-a5d4-d3d48cae50ef None admin] REQUEST_METHOD: `POST` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:42:27.999155 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-907d1fa9-91d6-401d-a5d4-d3d48cae50ef None admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:42:27.999431 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-907d1fa9-91d6-401d-a5d4-d3d48cae50ef None admin] PATH_INFO: `/v3/endpoints` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:42:28.000091 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-907d1fa9-91d6-401d-a5d4-d3d48cae50ef None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 10:42:28.001441 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-907d1fa9-91d6-401d-a5d4-d3d48cae50ef None admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 10:42:28.030961 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 416/815] 10.222.0.25 () {60 vars in 1144 bytes} [Sun Aug 7 10:42:27 2022] POST /identity/v3/endpoints => generated 346 bytes in 186 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 07 10:42:28.030961 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-907d1fa9-91d6-401d-a5d4-d3d48cae50ef None admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 10:42:28.061188 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-339fcc85-e159-4126-a5d3-eab1f5c54ef3 None admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 10:42:28.073442 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=100 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 10:42:28.106235 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77c641da-b5dc-4d7f-89c4-50a7624bbf31 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 10:42:28.106850 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77c641da-b5dc-4d7f-89c4-50a7624bbf31 None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:42:28.107154 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77c641da-b5dc-4d7f-89c4-50a7624bbf31 None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:42:28.107423 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77c641da-b5dc-4d7f-89c4-50a7624bbf31 None admin] PATH_INFO: `/v3/domains` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:42:28.108064 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77c641da-b5dc-4d7f-89c4-50a7624bbf31 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80143) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 10:42:28.108363 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77c641da-b5dc-4d7f-89c4-50a7624bbf31 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 10:42:28.109722 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77c641da-b5dc-4d7f-89c4-50a7624bbf31 None admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 10:42:28.123583 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 400/816] 10.222.0.25 () {56 vars in 1109 bytes} [Sun Aug 7 10:42:27 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 10:42:28.124592 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77c641da-b5dc-4d7f-89c4-50a7624bbf31 None admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 10:42:28.175762 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 10:42:28.185959 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-907d1fa9-91d6-401d-a5d4-d3d48cae50ef None admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 10:42:28.232424 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cd53d32-65e3-4930-982d-1fec049d52ca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 10:42:28.233554 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cd53d32-65e3-4930-982d-1fec049d52ca None admin] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:42:28.234008 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cd53d32-65e3-4930-982d-1fec049d52ca None admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:42:28.234556 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cd53d32-65e3-4930-982d-1fec049d52ca None admin] PATH_INFO: `/v3/services/network` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:42:28.235506 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cd53d32-65e3-4930-982d-1fec049d52ca None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 10:42:28.237317 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cd53d32-65e3-4930-982d-1fec049d52ca None admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 10:42:28.244944 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77c641da-b5dc-4d7f-89c4-50a7624bbf31 None admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 10:42:28.249095 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application [None req-1cd53d32-65e3-4930-982d-1fec049d52ca None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Aug 07 10:42:28.249095 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 07 10:42:28.249095 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 07 10:42:28.249095 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 07 10:42:28.249095 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 07 10:42:28.249095 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 07 10:42:28.249095 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 07 10:42:28.249095 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 07 10:42:28.249095 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Aug 07 10:42:28.249095 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application Aug 07 10:42:28.249095 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 07 10:42:28.249095 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application Aug 07 10:42:28.249095 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 07 10:42:28.249095 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 07 10:42:28.249095 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 07 10:42:28.249095 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 07 10:42:28.249095 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 07 10:42:28.250069 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 07 10:42:28.250069 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 07 10:42:28.250069 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 07 10:42:28.250069 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 07 10:42:28.250069 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 07 10:42:28.250069 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 07 10:42:28.250069 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 07 10:42:28.250069 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 07 10:42:28.250069 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 07 10:42:28.250069 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 07 10:42:28.250069 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 07 10:42:28.250069 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 07 10:42:28.250069 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 07 10:42:28.250069 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 07 10:42:28.250069 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 07 10:42:28.250069 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self.get_or_create( Aug 07 10:42:28.250912 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 07 10:42:28.250912 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application with Lock( Aug 07 10:42:28.250912 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 07 10:42:28.250912 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self._enter() Aug 07 10:42:28.250912 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 07 10:42:28.250912 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 07 10:42:28.250912 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 07 10:42:28.250912 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self.creator() Aug 07 10:42:28.250912 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 07 10:42:28.250912 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application created_value = creator( Aug 07 10:42:28.250912 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 07 10:42:28.250912 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 07 10:42:28.250912 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Aug 07 10:42:28.250912 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application Aug 07 10:42:28.250912 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 417/817] 10.222.0.25 () {56 vars in 1111 bytes} [Sun Aug 7 10:42:28 2022] GET /identity/v3/services/network => generated 88 bytes in 224 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 07 10:42:28.251747 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cd53d32-65e3-4930-982d-1fec049d52ca None admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 10:42:28.290583 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0ab4508-06c0-48ce-abe2-ec0a263b6968 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 10:42:28.291839 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0ab4508-06c0-48ce-abe2-ec0a263b6968 None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:42:28.292548 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0ab4508-06c0-48ce-abe2-ec0a263b6968 None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:42:28.293470 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0ab4508-06c0-48ce-abe2-ec0a263b6968 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:42:28.305106 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0ab4508-06c0-48ce-abe2-ec0a263b6968 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 10:42:28.307170 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0ab4508-06c0-48ce-abe2-ec0a263b6968 None admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 10:42:28.324086 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application [None req-f0ab4508-06c0-48ce-abe2-ec0a263b6968 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 07 10:42:28.324086 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 07 10:42:28.324086 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 07 10:42:28.324086 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 07 10:42:28.324086 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 07 10:42:28.324086 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 07 10:42:28.324086 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 07 10:42:28.324086 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 07 10:42:28.324086 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 07 10:42:28.324086 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 07 10:42:28.324086 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 07 10:42:28.324086 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 07 10:42:28.324086 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 07 10:42:28.324086 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 07 10:42:28.324086 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 07 10:42:28.324086 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 07 10:42:28.325076 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 07 10:42:28.325076 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 07 10:42:28.325076 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 07 10:42:28.325076 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 07 10:42:28.325076 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 07 10:42:28.325076 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.get_or_create( Aug 07 10:42:28.325076 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 07 10:42:28.325076 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application with Lock( Aug 07 10:42:28.325076 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 07 10:42:28.325076 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self._enter() Aug 07 10:42:28.325076 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 07 10:42:28.325076 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 07 10:42:28.325076 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 07 10:42:28.325076 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.creator() Aug 07 10:42:28.325076 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 07 10:42:28.325076 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application created_value = creator( Aug 07 10:42:28.325076 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 07 10:42:28.325955 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 07 10:42:28.325955 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 07 10:42:28.325955 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application Aug 07 10:42:28.325955 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 401/818] 10.222.0.25 () {56 vars in 1108 bytes} [Sun Aug 7 10:42:28 2022] GET /identity/v3/domains/Default => generated 87 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 07 10:42:28.325955 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0ab4508-06c0-48ce-abe2-ec0a263b6968 None admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 10:42:28.351132 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=100 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 10:42:28.393537 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cd53d32-65e3-4930-982d-1fec049d52ca None admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 10:42:28.442414 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0ab4508-06c0-48ce-abe2-ec0a263b6968 None admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 10:42:28.443999 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fec406d3-3d0f-4c80-847d-96f8cdb70d5b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 10:42:28.445897 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fec406d3-3d0f-4c80-847d-96f8cdb70d5b None admin] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:42:28.446340 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fec406d3-3d0f-4c80-847d-96f8cdb70d5b None admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:42:28.447381 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fec406d3-3d0f-4c80-847d-96f8cdb70d5b None admin] PATH_INFO: `/v3/projects/service` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:42:28.456870 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fec406d3-3d0f-4c80-847d-96f8cdb70d5b None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 10:42:28.458340 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fec406d3-3d0f-4c80-847d-96f8cdb70d5b None admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 10:42:28.474275 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application [None req-fec406d3-3d0f-4c80-847d-96f8cdb70d5b None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 07 10:42:28.474275 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 07 10:42:28.474275 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 07 10:42:28.474275 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 07 10:42:28.474275 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 07 10:42:28.474275 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 07 10:42:28.474275 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 07 10:42:28.474275 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 07 10:42:28.474275 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 07 10:42:28.474275 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 07 10:42:28.474275 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 07 10:42:28.474275 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 07 10:42:28.474275 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 07 10:42:28.474275 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 07 10:42:28.474275 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 07 10:42:28.475265 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 07 10:42:28.475265 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 07 10:42:28.475265 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 07 10:42:28.475265 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 07 10:42:28.475265 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 07 10:42:28.475265 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 07 10:42:28.475265 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self.get_or_create( Aug 07 10:42:28.475265 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 07 10:42:28.475265 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application with Lock( Aug 07 10:42:28.475265 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 07 10:42:28.475265 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self._enter() Aug 07 10:42:28.475265 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 07 10:42:28.475265 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 07 10:42:28.475265 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 07 10:42:28.475265 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self.creator() Aug 07 10:42:28.475265 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 07 10:42:28.475265 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application created_value = creator( Aug 07 10:42:28.476151 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 07 10:42:28.476151 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 07 10:42:28.476151 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 07 10:42:28.476151 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 07 10:42:28.476151 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 07 10:42:28.476151 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 07 10:42:28.476151 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 07 10:42:28.476151 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application Aug 07 10:42:28.480352 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 418/819] 10.222.0.25 () {56 vars in 1111 bytes} [Sun Aug 7 10:42:28 2022] GET /identity/v3/projects/service => generated 88 bytes in 224 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 07 10:42:28.481110 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fec406d3-3d0f-4c80-847d-96f8cdb70d5b None admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 10:42:28.492400 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abd1f827-0007-4895-82a7-b6c327a9bd9b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 10:42:28.493389 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abd1f827-0007-4895-82a7-b6c327a9bd9b None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:42:28.495154 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abd1f827-0007-4895-82a7-b6c327a9bd9b None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:42:28.495665 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abd1f827-0007-4895-82a7-b6c327a9bd9b None admin] PATH_INFO: `/v3/services` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:42:28.497018 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abd1f827-0007-4895-82a7-b6c327a9bd9b None admin] RBAC: Adding query filter params (name=network) {{(pid=80143) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 10:42:28.497651 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abd1f827-0007-4895-82a7-b6c327a9bd9b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 10:42:28.499524 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abd1f827-0007-4895-82a7-b6c327a9bd9b None admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 10:42:28.506809 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 402/820] 10.222.0.25 () {56 vars in 1112 bytes} [Sun Aug 7 10:42:28 2022] GET /identity/v3/services?name=network => generated 124 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 10:42:28.507864 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abd1f827-0007-4895-82a7-b6c327a9bd9b None admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 10:42:28.514622 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=100 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 10:42:28.595669 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fec406d3-3d0f-4c80-847d-96f8cdb70d5b None admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 10:42:28.624629 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=100 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 10:42:28.632485 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed005626-8e8e-4418-8006-96966e73ddcd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 10:42:28.633207 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed005626-8e8e-4418-8006-96966e73ddcd None admin] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:42:28.633552 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed005626-8e8e-4418-8006-96966e73ddcd None admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:42:28.633900 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed005626-8e8e-4418-8006-96966e73ddcd None admin] PATH_INFO: `/v3/domains` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:42:28.635237 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed005626-8e8e-4418-8006-96966e73ddcd None admin] RBAC: Adding query filter params (name=Default) {{(pid=80144) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 10:42:28.635752 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed005626-8e8e-4418-8006-96966e73ddcd None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 10:42:28.637250 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed005626-8e8e-4418-8006-96966e73ddcd None admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 10:42:28.649661 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 419/821] 10.222.0.25 () {56 vars in 1109 bytes} [Sun Aug 7 10:42:28 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 10:42:28.652485 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abd1f827-0007-4895-82a7-b6c327a9bd9b None admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 10:42:28.654967 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed005626-8e8e-4418-8006-96966e73ddcd None admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 10:42:28.699478 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-123bc555-e7a7-4e8a-bfed-4d7bc7326c94 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 10:42:28.700808 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-123bc555-e7a7-4e8a-bfed-4d7bc7326c94 None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:42:28.701287 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-123bc555-e7a7-4e8a-bfed-4d7bc7326c94 None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:42:28.702102 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-123bc555-e7a7-4e8a-bfed-4d7bc7326c94 None admin] PATH_INFO: `/v3/projects` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:42:28.703279 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-123bc555-e7a7-4e8a-bfed-4d7bc7326c94 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80143) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 10:42:28.703765 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-123bc555-e7a7-4e8a-bfed-4d7bc7326c94 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 10:42:28.706186 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-123bc555-e7a7-4e8a-bfed-4d7bc7326c94 None admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 10:42:28.724136 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 403/822] 10.222.0.25 () {56 vars in 1148 bytes} [Sun Aug 7 10:42:28 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 216 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 10:42:28.725238 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-123bc555-e7a7-4e8a-bfed-4d7bc7326c94 None admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 10:42:28.750806 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=100 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 10:42:28.771368 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed005626-8e8e-4418-8006-96966e73ddcd None admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 10:42:28.809828 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-852cd3af-3d49-4864-8a34-b1b507add52d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 10:42:28.810458 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-852cd3af-3d49-4864-8a34-b1b507add52d None admin] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:42:28.810740 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-852cd3af-3d49-4864-8a34-b1b507add52d None admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:42:28.811048 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-852cd3af-3d49-4864-8a34-b1b507add52d None admin] PATH_INFO: `/v3/services` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:42:28.811792 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-852cd3af-3d49-4864-8a34-b1b507add52d None admin] RBAC: Adding query filter params (type=network) {{(pid=80144) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 10:42:28.812108 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-852cd3af-3d49-4864-8a34-b1b507add52d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 10:42:28.813609 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-852cd3af-3d49-4864-8a34-b1b507add52d None admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 10:42:28.819424 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 420/823] 10.222.0.25 () {56 vars in 1112 bytes} [Sun Aug 7 10:42:28 2022] GET /identity/v3/services?type=network => generated 350 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 10:42:28.820557 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-852cd3af-3d49-4864-8a34-b1b507add52d None admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 10:42:28.844705 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-123bc555-e7a7-4e8a-bfed-4d7bc7326c94 None admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 10:42:28.888836 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aed9bb1b-d335-4ba9-8599-6536699f264a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 10:42:28.890263 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aed9bb1b-d335-4ba9-8599-6536699f264a None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:42:28.890723 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aed9bb1b-d335-4ba9-8599-6536699f264a None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:42:28.891211 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aed9bb1b-d335-4ba9-8599-6536699f264a None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:42:28.899578 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aed9bb1b-d335-4ba9-8599-6536699f264a None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 10:42:28.901869 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aed9bb1b-d335-4ba9-8599-6536699f264a None admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 10:42:28.905109 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=100 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 10:42:28.915066 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application [None req-aed9bb1b-d335-4ba9-8599-6536699f264a None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 07 10:42:28.915066 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 07 10:42:28.915066 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 07 10:42:28.915066 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 07 10:42:28.915066 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 07 10:42:28.915066 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 07 10:42:28.915066 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 07 10:42:28.915066 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 07 10:42:28.915066 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 07 10:42:28.915066 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 07 10:42:28.915066 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 07 10:42:28.915066 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 07 10:42:28.915066 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 07 10:42:28.915066 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 07 10:42:28.915066 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 07 10:42:28.915066 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 07 10:42:28.916628 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 07 10:42:28.916628 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 07 10:42:28.916628 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 07 10:42:28.916628 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 07 10:42:28.916628 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 07 10:42:28.916628 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 07 10:42:28.916628 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 07 10:42:28.916628 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 07 10:42:28.916628 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 07 10:42:28.916628 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.get_or_create( Aug 07 10:42:28.916628 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 07 10:42:28.916628 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application with Lock( Aug 07 10:42:28.916628 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 07 10:42:28.916628 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self._enter() Aug 07 10:42:28.916628 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 07 10:42:28.916628 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 07 10:42:28.916628 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 07 10:42:28.916628 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.creator() Aug 07 10:42:28.917668 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 07 10:42:28.917668 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application created_value = creator( Aug 07 10:42:28.917668 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 07 10:42:28.917668 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 07 10:42:28.917668 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 07 10:42:28.917668 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 07 10:42:28.917668 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 07 10:42:28.917668 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 07 10:42:28.917668 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 07 10:42:28.917668 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application Aug 07 10:42:28.918208 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 404/824] 10.222.0.25 () {56 vars in 1117 bytes} [Sun Aug 7 10:42:28 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 07 10:42:28.918915 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aed9bb1b-d335-4ba9-8599-6536699f264a None admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 10:42:28.952567 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-852cd3af-3d49-4864-8a34-b1b507add52d None admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 10:42:28.993472 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55f25574-83fb-4029-bcb8-3bcf804473d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 10:42:28.994398 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55f25574-83fb-4029-bcb8-3bcf804473d9 None admin] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:42:28.994705 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55f25574-83fb-4029-bcb8-3bcf804473d9 None admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:42:28.995242 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55f25574-83fb-4029-bcb8-3bcf804473d9 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:42:28.995978 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55f25574-83fb-4029-bcb8-3bcf804473d9 None admin] RBAC: Adding query filter params (role.id=81b1f1bcee2a473088fbbd327c1c5b23, scope.project.id=6de1f3cac02c4bcdb05dd2337176033d, user.id=31d16d1b96104b4eb690398432b9829e) {{(pid=80144) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 10:42:28.996292 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55f25574-83fb-4029-bcb8-3bcf804473d9 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 10:42:28.998584 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55f25574-83fb-4029-bcb8-3bcf804473d9 None admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 10:42:29.006531 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 421/825] 10.222.0.25 () {56 vars in 1374 bytes} [Sun Aug 7 10:42:28 2022] GET /identity/v3/role_assignments?user.id=31d16d1b96104b4eb690398432b9829e&scope.project.id=6de1f3cac02c4bcdb05dd2337176033d&role.id=81b1f1bcee2a473088fbbd327c1c5b23 => generated 608 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 10:42:29.007384 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55f25574-83fb-4029-bcb8-3bcf804473d9 None admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 10:42:29.027145 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=100 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 10:42:29.038056 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aed9bb1b-d335-4ba9-8599-6536699f264a None admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 10:42:29.089516 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98a6b248-c7ed-41ab-8322-f2202b2e1007 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 10:42:29.090452 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98a6b248-c7ed-41ab-8322-f2202b2e1007 None admin] REQUEST_METHOD: `POST` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:42:29.090741 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98a6b248-c7ed-41ab-8322-f2202b2e1007 None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:42:29.091043 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98a6b248-c7ed-41ab-8322-f2202b2e1007 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:42:29.091719 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98a6b248-c7ed-41ab-8322-f2202b2e1007 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 10:42:29.093772 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98a6b248-c7ed-41ab-8322-f2202b2e1007 None admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 10:42:29.139247 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 405/826] 10.222.0.25 () {60 vars in 1144 bytes} [Sun Aug 7 10:42:28 2022] POST /identity/v3/endpoints => generated 337 bytes in 221 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 07 10:42:29.168738 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55f25574-83fb-4029-bcb8-3bcf804473d9 None admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 10:42:29.253929 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57504369-ed90-4c69-92fa-411f2f493399 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 10:42:29.254532 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57504369-ed90-4c69-92fa-411f2f493399 None admin] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:42:29.254821 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57504369-ed90-4c69-92fa-411f2f493399 None admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:42:29.255125 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57504369-ed90-4c69-92fa-411f2f493399 None admin] PATH_INFO: `/v3/users` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:42:29.255945 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57504369-ed90-4c69-92fa-411f2f493399 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80144) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 10:42:29.256254 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57504369-ed90-4c69-92fa-411f2f493399 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 10:42:29.257715 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57504369-ed90-4c69-92fa-411f2f493399 None admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 10:42:29.285497 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 422/827] 10.222.0.25 () {56 vars in 1149 bytes} [Sun Aug 7 10:42:29 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 278 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 10:42:29.290840 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98a6b248-c7ed-41ab-8322-f2202b2e1007 None admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 10:42:29.362168 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 10:42:29.412486 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98a6b248-c7ed-41ab-8322-f2202b2e1007 None admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 10:42:29.456306 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98bdb783-527a-48af-b60e-83548dc59eeb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 10:42:29.457319 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98bdb783-527a-48af-b60e-83548dc59eeb None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:42:29.457804 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98bdb783-527a-48af-b60e-83548dc59eeb None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:42:29.458278 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98bdb783-527a-48af-b60e-83548dc59eeb None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:42:29.475312 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98bdb783-527a-48af-b60e-83548dc59eeb None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 10:42:29.477129 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98bdb783-527a-48af-b60e-83548dc59eeb None admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 10:42:29.492744 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application [None req-98bdb783-527a-48af-b60e-83548dc59eeb None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 07 10:42:29.492744 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 07 10:42:29.492744 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 07 10:42:29.492744 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 07 10:42:29.492744 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 07 10:42:29.492744 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 07 10:42:29.492744 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 07 10:42:29.492744 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 07 10:42:29.492744 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 07 10:42:29.492744 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 07 10:42:29.492744 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 07 10:42:29.492744 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 07 10:42:29.492744 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 07 10:42:29.492744 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 07 10:42:29.492744 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 07 10:42:29.492744 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 07 10:42:29.494354 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 07 10:42:29.494354 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 07 10:42:29.494354 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 07 10:42:29.494354 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 07 10:42:29.494354 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 07 10:42:29.494354 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.get_or_create( Aug 07 10:42:29.494354 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 07 10:42:29.494354 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application with Lock( Aug 07 10:42:29.494354 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 07 10:42:29.494354 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self._enter() Aug 07 10:42:29.494354 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 07 10:42:29.494354 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 07 10:42:29.494354 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 07 10:42:29.494354 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.creator() Aug 07 10:42:29.494354 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 07 10:42:29.494354 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application created_value = creator( Aug 07 10:42:29.494354 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 07 10:42:29.495768 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 07 10:42:29.495768 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 07 10:42:29.495768 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application Aug 07 10:42:29.497306 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 406/828] 10.222.0.25 () {56 vars in 1108 bytes} [Sun Aug 7 10:42:29 2022] GET /identity/v3/domains/Default => generated 87 bytes in 207 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 07 10:42:29.508021 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57504369-ed90-4c69-92fa-411f2f493399 None admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 10:42:29.581989 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=100 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 10:42:29.623684 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57504369-ed90-4c69-92fa-411f2f493399 None admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 10:42:29.658308 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4ca0122-c65d-426a-9b9f-0793b7924718 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 10:42:29.658932 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4ca0122-c65d-426a-9b9f-0793b7924718 None admin] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:42:29.659242 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4ca0122-c65d-426a-9b9f-0793b7924718 None admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:42:29.659515 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4ca0122-c65d-426a-9b9f-0793b7924718 None admin] PATH_INFO: `/v3/domains` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:42:29.660163 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4ca0122-c65d-426a-9b9f-0793b7924718 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80144) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 10:42:29.660463 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4ca0122-c65d-426a-9b9f-0793b7924718 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 10:42:29.661887 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4ca0122-c65d-426a-9b9f-0793b7924718 None admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 10:42:29.674978 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 423/829] 10.222.0.25 () {56 vars in 1109 bytes} [Sun Aug 7 10:42:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 10:42:29.680400 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98bdb783-527a-48af-b60e-83548dc59eeb None admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 10:42:29.786202 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98bdb783-527a-48af-b60e-83548dc59eeb None admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 10:42:29.795325 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=100 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 10:42:29.821573 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bea2534-6110-4a56-9969-a0c99f9a804b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 10:42:29.822250 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bea2534-6110-4a56-9969-a0c99f9a804b None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:42:29.822600 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bea2534-6110-4a56-9969-a0c99f9a804b None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:42:29.823003 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bea2534-6110-4a56-9969-a0c99f9a804b None admin] PATH_INFO: `/v3/projects/service` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:42:29.829915 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bea2534-6110-4a56-9969-a0c99f9a804b None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 10:42:29.831385 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bea2534-6110-4a56-9969-a0c99f9a804b None admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 10:42:29.843199 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application [None req-4bea2534-6110-4a56-9969-a0c99f9a804b None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 07 10:42:29.843199 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 07 10:42:29.843199 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 07 10:42:29.843199 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 07 10:42:29.843199 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 07 10:42:29.843199 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 07 10:42:29.843199 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 07 10:42:29.843199 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 07 10:42:29.843199 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 07 10:42:29.843199 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 07 10:42:29.843199 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 07 10:42:29.843199 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 07 10:42:29.843199 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 07 10:42:29.843199 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 07 10:42:29.843199 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 07 10:42:29.844074 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 07 10:42:29.844074 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 07 10:42:29.844074 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 07 10:42:29.844074 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 07 10:42:29.844074 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 07 10:42:29.844074 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 07 10:42:29.844074 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.get_or_create( Aug 07 10:42:29.844074 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 07 10:42:29.844074 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application with Lock( Aug 07 10:42:29.844074 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 07 10:42:29.844074 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self._enter() Aug 07 10:42:29.844074 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 07 10:42:29.844074 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 07 10:42:29.844074 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 07 10:42:29.844074 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.creator() Aug 07 10:42:29.844074 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 07 10:42:29.844074 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application created_value = creator( Aug 07 10:42:29.845027 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 07 10:42:29.845027 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 07 10:42:29.845027 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 07 10:42:29.845027 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 07 10:42:29.845027 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 07 10:42:29.845027 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 07 10:42:29.845027 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 07 10:42:29.845027 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application Aug 07 10:42:29.845027 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 407/830] 10.222.0.25 () {56 vars in 1111 bytes} [Sun Aug 7 10:42:29 2022] GET /identity/v3/projects/service => generated 88 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 07 10:42:29.851739 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4ca0122-c65d-426a-9b9f-0793b7924718 None admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 10:42:29.908703 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87258855-c341-4eec-bb2d-d1554cc32bb1 None None] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:42:29.909214 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87258855-c341-4eec-bb2d-d1554cc32bb1 None None] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:42:29.909316 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87258855-c341-4eec-bb2d-d1554cc32bb1 None None] PATH_INFO: `/` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:42:29.911781 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 408/831] 10.222.0.25 () {54 vars in 927 bytes} [Sun Aug 7 10:42:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 07 10:42:29.915821 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-205be694-c790-42f5-a607-81a1e357ae87 None None] REQUEST_METHOD: `POST` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:42:29.916107 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-205be694-c790-42f5-a607-81a1e357ae87 None None] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:42:29.916385 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-205be694-c790-42f5-a607-81a1e357ae87 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:42:29.938076 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.auth.core [None req-205be694-c790-42f5-a607-81a1e357ae87 None None] MFA Rules not processed for user `bac4e6b76fe64217ac8fa44e71cffe6a`. Rule list: `[]` (Enabled: `True`). {{(pid=80143) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 10:42:29.939957 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-205be694-c790-42f5-a607-81a1e357ae87 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 10:42:29.961138 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4ca0122-c65d-426a-9b9f-0793b7924718 None admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 10:42:29.986785 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=100 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 10:42:30.002780 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c28b1614-361c-4e43-abd9-a1cbaa07128c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 10:42:30.003442 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c28b1614-361c-4e43-abd9-a1cbaa07128c None admin] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:42:30.003685 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c28b1614-361c-4e43-abd9-a1cbaa07128c None admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:42:30.003959 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c28b1614-361c-4e43-abd9-a1cbaa07128c None admin] PATH_INFO: `/v3/projects` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:42:30.004647 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c28b1614-361c-4e43-abd9-a1cbaa07128c None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80144) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 10:42:30.005053 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c28b1614-361c-4e43-abd9-a1cbaa07128c None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 10:42:30.006570 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c28b1614-361c-4e43-abd9-a1cbaa07128c None admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 10:42:30.019879 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 424/832] 10.222.0.25 () {56 vars in 1148 bytes} [Sun Aug 7 10:42:29 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 10:42:30.026115 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c28b1614-361c-4e43-abd9-a1cbaa07128c None admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 10:42:30.036195 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 409/833] 10.222.0.25 () {58 vars in 1024 bytes} [Sun Aug 7 10:42:29 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 07 10:42:30.055764 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-205be694-c790-42f5-a607-81a1e357ae87 None None] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 10:42:30.152162 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c28b1614-361c-4e43-abd9-a1cbaa07128c None admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 10:42:30.167464 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=100 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 10:42:30.171594 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-205be694-c790-42f5-a607-81a1e357ae87 None None] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 10:42:30.189424 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4b65acb-650b-4393-bc32-bfded69cd192 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 10:42:30.190393 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4b65acb-650b-4393-bc32-bfded69cd192 None admin] REQUEST_METHOD: `PUT` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:42:30.190829 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4b65acb-650b-4393-bc32-bfded69cd192 None admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:42:30.191362 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4b65acb-650b-4393-bc32-bfded69cd192 None admin] PATH_INFO: `/v3/projects/6de1f3cac02c4bcdb05dd2337176033d/users/de1d6c24b1a14d10a20d82a4c65793d8/roles/fbf375e9760340c9aa9529333eeb25f9` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:42:30.207132 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4b65acb-650b-4393-bc32-bfded69cd192 None admin] RBAC: Authorizing `identity:create_grant(project_id=6de1f3cac02c4bcdb05dd2337176033d, user_id=de1d6c24b1a14d10a20d82a4c65793d8, role_id=fbf375e9760340c9aa9529333eeb25f9)` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 10:42:30.208564 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4b65acb-650b-4393-bc32-bfded69cd192 None admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 10:42:30.210171 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-710c7418-55c6-4e4a-9166-4dbecef6e50c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 10:42:30.211008 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-710c7418-55c6-4e4a-9166-4dbecef6e50c None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:42:30.211456 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-710c7418-55c6-4e4a-9166-4dbecef6e50c None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:42:30.211802 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-710c7418-55c6-4e4a-9166-4dbecef6e50c None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:42:30.224037 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-710c7418-55c6-4e4a-9166-4dbecef6e50c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 07 10:42:30.225280 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 425/834] 10.222.0.25 () {58 vars in 1439 bytes} [Sun Aug 7 10:42:30 2022] PUT /identity/v3/projects/6de1f3cac02c4bcdb05dd2337176033d/users/de1d6c24b1a14d10a20d82a4c65793d8/roles/fbf375e9760340c9aa9529333eeb25f9 => generated 0 bytes in 200 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 07 10:42:30.229917 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application [None req-710c7418-55c6-4e4a-9166-4dbecef6e50c None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 07 10:42:30.229917 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 07 10:42:30.229917 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 07 10:42:30.229917 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 07 10:42:30.229917 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 07 10:42:30.229917 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 07 10:42:30.229917 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 07 10:42:30.229917 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 07 10:42:30.229917 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 07 10:42:30.229917 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 07 10:42:30.229917 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 07 10:42:30.229917 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 07 10:42:30.229917 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 07 10:42:30.229917 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 07 10:42:30.229917 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 07 10:42:30.229917 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 07 10:42:30.230856 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 07 10:42:30.230856 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 07 10:42:30.230856 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 07 10:42:30.230856 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 07 10:42:30.230856 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 07 10:42:30.230856 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 07 10:42:30.230856 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 07 10:42:30.230856 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 07 10:42:30.230856 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 07 10:42:30.230856 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.get_or_create( Aug 07 10:42:30.230856 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 07 10:42:30.230856 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application with Lock( Aug 07 10:42:30.230856 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 07 10:42:30.230856 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self._enter() Aug 07 10:42:30.230856 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 07 10:42:30.230856 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 07 10:42:30.231711 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 07 10:42:30.231711 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.creator() Aug 07 10:42:30.231711 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 07 10:42:30.231711 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application created_value = creator( Aug 07 10:42:30.231711 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 07 10:42:30.231711 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 07 10:42:30.231711 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 07 10:42:30.231711 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 07 10:42:30.231711 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 07 10:42:30.231711 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 07 10:42:30.231711 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 07 10:42:30.231711 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application Aug 07 10:42:30.236146 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 410/835] 10.222.0.25 () {56 vars in 1096 bytes} [Sun Aug 7 10:42:30 2022] GET /identity/v3/roles/admin => generated 83 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 07 10:42:30.240107 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4b65acb-650b-4393-bc32-bfded69cd192 None admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 10:42:30.279367 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 10:42:30.358443 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4b65acb-650b-4393-bc32-bfded69cd192 None admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 10:42:30.366832 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9700339-ecff-459b-b73a-9218595e2681 None None] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:42:30.367154 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9700339-ecff-459b-b73a-9218595e2681 None None] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:42:30.367413 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9700339-ecff-459b-b73a-9218595e2681 None None] PATH_INFO: `/` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:42:30.369528 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 411/836] 10.222.0.25 () {54 vars in 927 bytes} [Sun Aug 7 10:42:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 07 10:42:30.375627 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-facb8818-1955-468c-ac44-1d1c45e2991e None None] REQUEST_METHOD: `POST` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:42:30.376224 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-facb8818-1955-468c-ac44-1d1c45e2991e None None] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:42:30.376781 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-facb8818-1955-468c-ac44-1d1c45e2991e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:42:30.400633 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.auth.core [None req-facb8818-1955-468c-ac44-1d1c45e2991e None None] MFA Rules not processed for user `bac4e6b76fe64217ac8fa44e71cffe6a`. Rule list: `[]` (Enabled: `True`). {{(pid=80143) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 10:42:30.402507 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-facb8818-1955-468c-ac44-1d1c45e2991e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 10:42:30.405500 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02f55883-22d9-49a6-b579-6bdfdaa9ddfc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 10:42:30.407011 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02f55883-22d9-49a6-b579-6bdfdaa9ddfc None admin] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:42:30.407283 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02f55883-22d9-49a6-b579-6bdfdaa9ddfc None admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:42:30.407624 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02f55883-22d9-49a6-b579-6bdfdaa9ddfc None admin] PATH_INFO: `/v3/roles` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:42:30.408310 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02f55883-22d9-49a6-b579-6bdfdaa9ddfc None admin] RBAC: Adding query filter params (name=admin) {{(pid=80144) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 10:42:30.408620 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02f55883-22d9-49a6-b579-6bdfdaa9ddfc None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 10:42:30.409999 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02f55883-22d9-49a6-b579-6bdfdaa9ddfc None admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 10:42:30.426345 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 426/837] 10.222.0.25 () {56 vars in 1099 bytes} [Sun Aug 7 10:42:30 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 10:42:30.431632 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02f55883-22d9-49a6-b579-6bdfdaa9ddfc None admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 10:42:30.488125 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 412/838] 10.222.0.25 () {58 vars in 1024 bytes} [Sun Aug 7 10:42:30 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 07 10:42:30.510648 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-facb8818-1955-468c-ac44-1d1c45e2991e None None] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 10:42:30.543949 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02f55883-22d9-49a6-b579-6bdfdaa9ddfc None admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 10:42:30.547377 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=100 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 10:42:30.582516 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f260137-7a33-4cc6-b2bc-5972c8f96141 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 10:42:30.583161 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f260137-7a33-4cc6-b2bc-5972c8f96141 None admin] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:42:30.583443 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f260137-7a33-4cc6-b2bc-5972c8f96141 None admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:42:30.583717 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f260137-7a33-4cc6-b2bc-5972c8f96141 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:42:30.594931 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f260137-7a33-4cc6-b2bc-5972c8f96141 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 10:42:30.596389 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f260137-7a33-4cc6-b2bc-5972c8f96141 None admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 10:42:30.604868 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=100 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 10:42:30.615630 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application [None req-0f260137-7a33-4cc6-b2bc-5972c8f96141 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 07 10:42:30.615630 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 07 10:42:30.615630 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 07 10:42:30.615630 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 07 10:42:30.615630 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 07 10:42:30.615630 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 07 10:42:30.615630 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 07 10:42:30.615630 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 07 10:42:30.615630 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 07 10:42:30.615630 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 07 10:42:30.615630 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 07 10:42:30.615630 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 07 10:42:30.615630 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 07 10:42:30.615630 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 07 10:42:30.615630 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 07 10:42:30.615630 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 07 10:42:30.616583 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 07 10:42:30.616583 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 07 10:42:30.616583 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 07 10:42:30.616583 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 07 10:42:30.616583 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 07 10:42:30.616583 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self.get_or_create( Aug 07 10:42:30.616583 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 07 10:42:30.616583 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application with Lock( Aug 07 10:42:30.616583 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 07 10:42:30.616583 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self._enter() Aug 07 10:42:30.616583 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 07 10:42:30.616583 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 07 10:42:30.616583 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 07 10:42:30.616583 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self.creator() Aug 07 10:42:30.616583 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 07 10:42:30.616583 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application created_value = creator( Aug 07 10:42:30.616583 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 07 10:42:30.617453 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 07 10:42:30.617453 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 07 10:42:30.617453 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application Aug 07 10:42:30.618333 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-facb8818-1955-468c-ac44-1d1c45e2991e None None] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 10:42:30.627186 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 427/839] 10.222.0.25 () {56 vars in 1108 bytes} [Sun Aug 7 10:42:30 2022] GET /identity/v3/domains/Default => generated 87 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 07 10:42:30.631840 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f260137-7a33-4cc6-b2bc-5972c8f96141 None admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 10:42:30.654908 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16daccd9-5be7-47eb-8488-de4981666d08 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 10:42:30.655574 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16daccd9-5be7-47eb-8488-de4981666d08 None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:42:30.655848 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16daccd9-5be7-47eb-8488-de4981666d08 None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:42:30.656128 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16daccd9-5be7-47eb-8488-de4981666d08 None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:42:30.656717 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16daccd9-5be7-47eb-8488-de4981666d08 None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 10:42:30.658366 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16daccd9-5be7-47eb-8488-de4981666d08 None admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 10:42:30.667166 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application [None req-16daccd9-5be7-47eb-8488-de4981666d08 None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Aug 07 10:42:30.667166 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 07 10:42:30.667166 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 07 10:42:30.667166 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 07 10:42:30.667166 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 07 10:42:30.667166 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 07 10:42:30.667166 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 07 10:42:30.667166 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 07 10:42:30.667166 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Aug 07 10:42:30.667166 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application Aug 07 10:42:30.667166 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 07 10:42:30.667166 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application Aug 07 10:42:30.667166 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 07 10:42:30.667166 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 07 10:42:30.667166 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 07 10:42:30.667166 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 07 10:42:30.667166 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 07 10:42:30.668415 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 07 10:42:30.668415 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 07 10:42:30.668415 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 07 10:42:30.668415 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 07 10:42:30.668415 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 07 10:42:30.668415 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 07 10:42:30.668415 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 07 10:42:30.668415 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 07 10:42:30.668415 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 07 10:42:30.668415 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 07 10:42:30.668415 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 07 10:42:30.668415 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 07 10:42:30.668415 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 07 10:42:30.668415 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 07 10:42:30.668415 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 07 10:42:30.668415 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.get_or_create( Aug 07 10:42:30.669461 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 07 10:42:30.669461 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application with Lock( Aug 07 10:42:30.669461 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 07 10:42:30.669461 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self._enter() Aug 07 10:42:30.669461 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 07 10:42:30.669461 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 07 10:42:30.669461 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 07 10:42:30.669461 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.creator() Aug 07 10:42:30.669461 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 07 10:42:30.669461 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application created_value = creator( Aug 07 10:42:30.669461 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 07 10:42:30.669461 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 07 10:42:30.669461 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Aug 07 10:42:30.669461 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application Aug 07 10:42:30.676600 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 413/840] 10.222.0.25 () {56 vars in 1114 bytes} [Sun Aug 7 10:42:30 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 07 10:42:30.677128 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16daccd9-5be7-47eb-8488-de4981666d08 None admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 10:42:30.748996 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f260137-7a33-4cc6-b2bc-5972c8f96141 None admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 10:42:30.789005 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49d4b260-c017-4c1b-a55f-2a7030e8a0ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 10:42:30.789772 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49d4b260-c017-4c1b-a55f-2a7030e8a0ee None admin] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:42:30.791155 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49d4b260-c017-4c1b-a55f-2a7030e8a0ee None admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:42:30.791678 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49d4b260-c017-4c1b-a55f-2a7030e8a0ee None admin] PATH_INFO: `/v3/domains` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:42:30.792717 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49d4b260-c017-4c1b-a55f-2a7030e8a0ee None admin] RBAC: Adding query filter params (name=Default) {{(pid=80144) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 10:42:30.793149 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49d4b260-c017-4c1b-a55f-2a7030e8a0ee None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 10:42:30.794783 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49d4b260-c017-4c1b-a55f-2a7030e8a0ee None admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 10:42:30.803612 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16daccd9-5be7-47eb-8488-de4981666d08 None admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 10:42:30.815760 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 428/841] 10.222.0.25 () {56 vars in 1109 bytes} [Sun Aug 7 10:42:30 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 10:42:30.821323 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49d4b260-c017-4c1b-a55f-2a7030e8a0ee None admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 10:42:30.832909 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=100 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 10:42:30.850602 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae2848d1-9711-46ec-a87e-068c9252dfe4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 10:42:30.851766 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae2848d1-9711-46ec-a87e-068c9252dfe4 None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:42:30.852539 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae2848d1-9711-46ec-a87e-068c9252dfe4 None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:42:30.853591 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae2848d1-9711-46ec-a87e-068c9252dfe4 None admin] PATH_INFO: `/v3/services` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:42:30.854708 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae2848d1-9711-46ec-a87e-068c9252dfe4 None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=80143) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 10:42:30.855244 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae2848d1-9711-46ec-a87e-068c9252dfe4 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 10:42:30.857725 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae2848d1-9711-46ec-a87e-068c9252dfe4 None admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 10:42:30.869729 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 414/842] 10.222.0.25 () {56 vars in 1114 bytes} [Sun Aug 7 10:42:30 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 07 10:42:30.870320 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae2848d1-9711-46ec-a87e-068c9252dfe4 None admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 10:42:30.892604 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=100 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 10:42:30.953722 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49d4b260-c017-4c1b-a55f-2a7030e8a0ee None admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 10:42:30.991090 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d551c89-4af5-47d9-b019-42517e124f76 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 10:42:30.991732 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d551c89-4af5-47d9-b019-42517e124f76 None admin] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:42:30.992015 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d551c89-4af5-47d9-b019-42517e124f76 None admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:42:30.992289 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d551c89-4af5-47d9-b019-42517e124f76 None admin] PATH_INFO: `/v3/users/swift` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:42:31.002637 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae2848d1-9711-46ec-a87e-068c9252dfe4 None admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 10:42:31.004663 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d551c89-4af5-47d9-b019-42517e124f76 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 10:42:31.007748 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d551c89-4af5-47d9-b019-42517e124f76 None admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 10:42:31.021806 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application [None req-2d551c89-4af5-47d9-b019-42517e124f76 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Aug 07 10:42:31.021806 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 07 10:42:31.021806 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 07 10:42:31.021806 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 07 10:42:31.021806 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 07 10:42:31.021806 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 07 10:42:31.021806 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 07 10:42:31.021806 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 07 10:42:31.021806 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 07 10:42:31.021806 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 07 10:42:31.021806 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 07 10:42:31.021806 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 07 10:42:31.021806 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 07 10:42:31.021806 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 07 10:42:31.021806 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 07 10:42:31.021806 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 07 10:42:31.022788 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 07 10:42:31.022788 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 07 10:42:31.022788 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 07 10:42:31.022788 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 07 10:42:31.022788 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 07 10:42:31.022788 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 07 10:42:31.022788 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 07 10:42:31.022788 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 07 10:42:31.022788 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 07 10:42:31.022788 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self.get_or_create( Aug 07 10:42:31.022788 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 07 10:42:31.022788 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application with Lock( Aug 07 10:42:31.022788 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 07 10:42:31.022788 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self._enter() Aug 07 10:42:31.022788 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 07 10:42:31.022788 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 07 10:42:31.022788 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 07 10:42:31.022788 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self.creator() Aug 07 10:42:31.023817 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 07 10:42:31.023817 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application created_value = creator( Aug 07 10:42:31.023817 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 07 10:42:31.023817 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 07 10:42:31.023817 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 07 10:42:31.023817 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 07 10:42:31.023817 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 07 10:42:31.023817 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 07 10:42:31.023817 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Aug 07 10:42:31.023817 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application Aug 07 10:42:31.027812 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 429/843] 10.222.0.25 () {56 vars in 1096 bytes} [Sun Aug 7 10:42:30 2022] GET /identity/v3/users/swift => generated 83 bytes in 207 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 07 10:42:31.029158 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d551c89-4af5-47d9-b019-42517e124f76 None admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 10:42:31.050851 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ace24a8-ad5a-42a7-bd95-8940311729f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 10:42:31.051853 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ace24a8-ad5a-42a7-bd95-8940311729f4 None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:42:31.052846 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ace24a8-ad5a-42a7-bd95-8940311729f4 None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:42:31.053336 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ace24a8-ad5a-42a7-bd95-8940311729f4 None admin] PATH_INFO: `/v3/services` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:42:31.054446 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ace24a8-ad5a-42a7-bd95-8940311729f4 None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=80143) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 10:42:31.055234 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ace24a8-ad5a-42a7-bd95-8940311729f4 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 10:42:31.058172 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ace24a8-ad5a-42a7-bd95-8940311729f4 None admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 10:42:31.071405 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 415/844] 10.222.0.25 () {56 vars in 1114 bytes} [Sun Aug 7 10:42:30 2022] GET /identity/v3/services?type=volumev3 => generated 125 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 10:42:31.114781 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=100 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 10:42:31.153057 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d551c89-4af5-47d9-b019-42517e124f76 None admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 10:42:31.191015 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f685a74-4049-4647-8de0-2901cf11e6cb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 10:42:31.191642 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f685a74-4049-4647-8de0-2901cf11e6cb None admin] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:42:31.191922 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f685a74-4049-4647-8de0-2901cf11e6cb None admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:42:31.192194 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f685a74-4049-4647-8de0-2901cf11e6cb None admin] PATH_INFO: `/v3/users` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:42:31.192926 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f685a74-4049-4647-8de0-2901cf11e6cb None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80144) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 10:42:31.193233 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f685a74-4049-4647-8de0-2901cf11e6cb None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 10:42:31.194807 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f685a74-4049-4647-8de0-2901cf11e6cb None admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 10:42:31.219125 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 430/845] 10.222.0.25 () {56 vars in 1135 bytes} [Sun Aug 7 10:42:31 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 10:42:31.221162 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ace24a8-ad5a-42a7-bd95-8940311729f4 None admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 10:42:31.309936 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=100 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 10:42:31.331452 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ace24a8-ad5a-42a7-bd95-8940311729f4 None admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 10:42:31.366847 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45b480d3-2dbb-405e-9f4b-69b0c9326e3c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 10:42:31.367513 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45b480d3-2dbb-405e-9f4b-69b0c9326e3c None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:42:31.367786 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45b480d3-2dbb-405e-9f4b-69b0c9326e3c None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:42:31.368063 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45b480d3-2dbb-405e-9f4b-69b0c9326e3c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:42:31.397787 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1a16bb4-77e9-4467-803c-f3eb7a994f74 None None] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:42:31.398073 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1a16bb4-77e9-4467-803c-f3eb7a994f74 None None] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:42:31.398350 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1a16bb4-77e9-4467-803c-f3eb7a994f74 None None] PATH_INFO: `/` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:42:31.400613 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 431/846] 10.222.0.25 () {54 vars in 927 bytes} [Sun Aug 7 10:42:31 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 07 10:42:31.405701 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50bc0bfb-67b9-4f7f-a87c-a0cedc720b1c None None] REQUEST_METHOD: `POST` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:42:31.405914 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50bc0bfb-67b9-4f7f-a87c-a0cedc720b1c None None] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:42:31.406188 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50bc0bfb-67b9-4f7f-a87c-a0cedc720b1c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:42:31.435335 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.auth.core [None req-50bc0bfb-67b9-4f7f-a87c-a0cedc720b1c None None] MFA Rules not processed for user `bac4e6b76fe64217ac8fa44e71cffe6a`. Rule list: `[]` (Enabled: `True`). {{(pid=80144) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 10:42:31.436838 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.fernet_utils [None req-50bc0bfb-67b9-4f7f-a87c-a0cedc720b1c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80144) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 10:42:31.452631 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45b480d3-2dbb-405e-9f4b-69b0c9326e3c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 10:42:31.454512 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45b480d3-2dbb-405e-9f4b-69b0c9326e3c None admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 10:42:31.527957 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 432/847] 10.222.0.25 () {58 vars in 1024 bytes} [Sun Aug 7 10:42:31 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 07 10:42:31.551964 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50bc0bfb-67b9-4f7f-a87c-a0cedc720b1c None None] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 10:42:31.560205 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=100 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 10:42:31.582425 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 416/848] 10.222.0.25 () {58 vars in 1282 bytes} [Sun Aug 7 10:42:31 2022] GET /identity/v3/auth/tokens => generated 1161 bytes in 362 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 07 10:42:31.589102 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45b480d3-2dbb-405e-9f4b-69b0c9326e3c None admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 10:42:31.625715 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=100 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 10:42:31.676373 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50bc0bfb-67b9-4f7f-a87c-a0cedc720b1c None None] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 10:42:31.721212 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45b480d3-2dbb-405e-9f4b-69b0c9326e3c None admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 10:42:31.722015 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c7617c3-655d-46b6-a1b1-a9916106afbd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 10:42:31.722701 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c7617c3-655d-46b6-a1b1-a9916106afbd None admin] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:42:31.723012 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c7617c3-655d-46b6-a1b1-a9916106afbd None admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:42:31.723287 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c7617c3-655d-46b6-a1b1-a9916106afbd None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:42:31.724131 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c7617c3-655d-46b6-a1b1-a9916106afbd None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 10:42:31.726422 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c7617c3-655d-46b6-a1b1-a9916106afbd None admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 10:42:31.738840 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application [None req-7c7617c3-655d-46b6-a1b1-a9916106afbd None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Aug 07 10:42:31.738840 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 07 10:42:31.738840 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 07 10:42:31.738840 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 07 10:42:31.738840 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 07 10:42:31.738840 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 07 10:42:31.738840 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 07 10:42:31.738840 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 07 10:42:31.738840 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Aug 07 10:42:31.738840 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application Aug 07 10:42:31.738840 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 07 10:42:31.738840 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application Aug 07 10:42:31.738840 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 07 10:42:31.738840 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 07 10:42:31.738840 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 07 10:42:31.738840 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 07 10:42:31.739838 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 07 10:42:31.739838 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 07 10:42:31.739838 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 07 10:42:31.739838 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 07 10:42:31.739838 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 07 10:42:31.739838 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 07 10:42:31.739838 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 07 10:42:31.739838 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 07 10:42:31.739838 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 07 10:42:31.739838 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 07 10:42:31.739838 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 07 10:42:31.739838 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 07 10:42:31.739838 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 07 10:42:31.739838 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 07 10:42:31.739838 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 07 10:42:31.739838 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 07 10:42:31.739838 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self.get_or_create( Aug 07 10:42:31.740708 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 07 10:42:31.740708 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application with Lock( Aug 07 10:42:31.740708 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 07 10:42:31.740708 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self._enter() Aug 07 10:42:31.740708 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 07 10:42:31.740708 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 07 10:42:31.740708 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 07 10:42:31.740708 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self.creator() Aug 07 10:42:31.740708 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 07 10:42:31.740708 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application created_value = creator( Aug 07 10:42:31.740708 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 07 10:42:31.740708 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 07 10:42:31.740708 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Aug 07 10:42:31.740708 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application Aug 07 10:42:31.740708 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 433/849] 10.222.0.25 () {56 vars in 1132 bytes} [Sun Aug 7 10:42:31 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 07 10:42:31.746118 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c7617c3-655d-46b6-a1b1-a9916106afbd None admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 10:42:31.764906 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae39d61d-3e4e-471e-ae5c-c0f2f954cead None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 10:42:31.765895 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae39d61d-3e4e-471e-ae5c-c0f2f954cead None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:42:31.766398 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae39d61d-3e4e-471e-ae5c-c0f2f954cead None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:42:31.766885 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae39d61d-3e4e-471e-ae5c-c0f2f954cead None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:42:31.777800 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae39d61d-3e4e-471e-ae5c-c0f2f954cead None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 10:42:31.779618 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae39d61d-3e4e-471e-ae5c-c0f2f954cead None admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 10:42:31.795374 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application [None req-ae39d61d-3e4e-471e-ae5c-c0f2f954cead None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 07 10:42:31.795374 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 07 10:42:31.795374 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 07 10:42:31.795374 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 07 10:42:31.795374 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 07 10:42:31.795374 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 07 10:42:31.795374 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 07 10:42:31.795374 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 07 10:42:31.795374 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 07 10:42:31.795374 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 07 10:42:31.795374 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 07 10:42:31.795374 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 07 10:42:31.795374 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 07 10:42:31.795374 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 07 10:42:31.795374 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 07 10:42:31.795374 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 07 10:42:31.796335 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 07 10:42:31.796335 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 07 10:42:31.796335 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 07 10:42:31.796335 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 07 10:42:31.796335 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 07 10:42:31.796335 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.get_or_create( Aug 07 10:42:31.796335 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 07 10:42:31.796335 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application with Lock( Aug 07 10:42:31.796335 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 07 10:42:31.796335 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self._enter() Aug 07 10:42:31.796335 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 07 10:42:31.796335 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 07 10:42:31.796335 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 07 10:42:31.796335 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.creator() Aug 07 10:42:31.796335 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 07 10:42:31.796335 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application created_value = creator( Aug 07 10:42:31.796335 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 07 10:42:31.797227 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 07 10:42:31.797227 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 07 10:42:31.797227 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application Aug 07 10:42:31.799281 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 417/850] 10.222.0.25 () {56 vars in 1108 bytes} [Sun Aug 7 10:42:31 2022] GET /identity/v3/domains/Default => generated 87 bytes in 210 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 07 10:42:31.805113 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae39d61d-3e4e-471e-ae5c-c0f2f954cead None admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 10:42:31.844111 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=100 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 10:42:31.875788 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c7617c3-655d-46b6-a1b1-a9916106afbd None admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 10:42:31.902973 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=100 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 10:42:31.924680 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aba0bd9e-d3b7-4b48-845b-d8f32b7f7077 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 10:42:31.925295 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aba0bd9e-d3b7-4b48-845b-d8f32b7f7077 None admin] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:42:31.925575 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aba0bd9e-d3b7-4b48-845b-d8f32b7f7077 None admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:42:31.925847 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aba0bd9e-d3b7-4b48-845b-d8f32b7f7077 None admin] PATH_INFO: `/v3/services` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:42:31.926532 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aba0bd9e-d3b7-4b48-845b-d8f32b7f7077 None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=80144) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 10:42:31.926780 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aba0bd9e-d3b7-4b48-845b-d8f32b7f7077 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 10:42:31.927973 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae39d61d-3e4e-471e-ae5c-c0f2f954cead None admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 10:42:31.928281 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aba0bd9e-d3b7-4b48-845b-d8f32b7f7077 None admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 10:42:31.934457 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 434/851] 10.222.0.25 () {56 vars in 1126 bytes} [Sun Aug 7 10:42:31 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 07 10:42:31.940508 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aba0bd9e-d3b7-4b48-845b-d8f32b7f7077 None admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 10:42:31.968698 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36731ac1-571a-49b0-85dc-73bbf167fa62 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 10:42:31.969735 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36731ac1-571a-49b0-85dc-73bbf167fa62 None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:42:31.970220 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36731ac1-571a-49b0-85dc-73bbf167fa62 None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:42:31.970762 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36731ac1-571a-49b0-85dc-73bbf167fa62 None admin] PATH_INFO: `/v3/domains` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:42:31.971660 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36731ac1-571a-49b0-85dc-73bbf167fa62 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80143) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 10:42:31.972166 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36731ac1-571a-49b0-85dc-73bbf167fa62 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 10:42:31.973793 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36731ac1-571a-49b0-85dc-73bbf167fa62 None admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 10:42:31.991992 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 418/852] 10.222.0.25 () {56 vars in 1109 bytes} [Sun Aug 7 10:42:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 10:42:31.993173 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36731ac1-571a-49b0-85dc-73bbf167fa62 None admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 10:42:32.050478 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aba0bd9e-d3b7-4b48-845b-d8f32b7f7077 None admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 10:42:32.089028 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df03cd94-d676-4259-8be0-539d054856cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 10:42:32.089623 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df03cd94-d676-4259-8be0-539d054856cc None admin] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:42:32.089908 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df03cd94-d676-4259-8be0-539d054856cc None admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:42:32.090171 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df03cd94-d676-4259-8be0-539d054856cc None admin] PATH_INFO: `/v3/services` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:42:32.091003 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=100 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 10:42:32.091621 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df03cd94-d676-4259-8be0-539d054856cc None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=80144) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 10:42:32.092396 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df03cd94-d676-4259-8be0-539d054856cc None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 10:42:32.094040 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df03cd94-d676-4259-8be0-539d054856cc None admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 10:42:32.104223 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36731ac1-571a-49b0-85dc-73bbf167fa62 None admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 10:42:32.104593 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 435/853] 10.222.0.25 () {56 vars in 1126 bytes} [Sun Aug 7 10:42:31 2022] GET /identity/v3/services?type=compute_legacy => generated 131 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 07 10:42:32.141877 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55004e7b-de83-4c37-bda9-4f9eaac7bb2b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 10:42:32.142595 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55004e7b-de83-4c37-bda9-4f9eaac7bb2b None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:42:32.142882 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55004e7b-de83-4c37-bda9-4f9eaac7bb2b None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:42:32.143211 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55004e7b-de83-4c37-bda9-4f9eaac7bb2b None admin] PATH_INFO: `/v3/projects/service` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:42:32.151866 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55004e7b-de83-4c37-bda9-4f9eaac7bb2b None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 10:42:32.153476 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55004e7b-de83-4c37-bda9-4f9eaac7bb2b None admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 10:42:32.168528 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application [None req-55004e7b-de83-4c37-bda9-4f9eaac7bb2b None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 07 10:42:32.168528 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 07 10:42:32.168528 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 07 10:42:32.168528 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 07 10:42:32.168528 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 07 10:42:32.168528 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 07 10:42:32.168528 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 07 10:42:32.168528 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 07 10:42:32.168528 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 07 10:42:32.168528 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 07 10:42:32.168528 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 07 10:42:32.168528 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 07 10:42:32.168528 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 07 10:42:32.168528 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 07 10:42:32.168528 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 07 10:42:32.169468 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 07 10:42:32.169468 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 07 10:42:32.169468 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 07 10:42:32.169468 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 07 10:42:32.169468 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 07 10:42:32.169468 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 07 10:42:32.169468 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.get_or_create( Aug 07 10:42:32.169468 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 07 10:42:32.169468 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application with Lock( Aug 07 10:42:32.169468 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 07 10:42:32.169468 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self._enter() Aug 07 10:42:32.169468 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 07 10:42:32.169468 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 07 10:42:32.169468 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 07 10:42:32.169468 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.creator() Aug 07 10:42:32.169468 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 07 10:42:32.169468 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application created_value = creator( Aug 07 10:42:32.170330 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 07 10:42:32.170330 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 07 10:42:32.170330 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 07 10:42:32.170330 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 07 10:42:32.170330 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 07 10:42:32.170330 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 07 10:42:32.170330 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 07 10:42:32.170330 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application Aug 07 10:42:32.170330 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 419/854] 10.222.0.25 () {56 vars in 1111 bytes} [Sun Aug 7 10:42:31 2022] GET /identity/v3/projects/service => generated 88 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 07 10:42:32.176211 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df03cd94-d676-4259-8be0-539d054856cc None admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 10:42:32.237080 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=100 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 10:42:32.285679 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f38e3460-3c90-4e5c-b7c3-50f18cae2d9f None None] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:42:32.285993 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f38e3460-3c90-4e5c-b7c3-50f18cae2d9f None None] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:42:32.286227 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f38e3460-3c90-4e5c-b7c3-50f18cae2d9f None None] PATH_INFO: `/` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:42:32.287958 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 420/855] 10.222.0.25 () {54 vars in 927 bytes} [Sun Aug 7 10:42:32 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 07 10:42:32.295407 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41023c49-2f28-43a7-abb0-d476ecf414fb None None] REQUEST_METHOD: `POST` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:42:32.295691 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41023c49-2f28-43a7-abb0-d476ecf414fb None None] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:42:32.295962 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41023c49-2f28-43a7-abb0-d476ecf414fb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:42:32.298695 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df03cd94-d676-4259-8be0-539d054856cc None admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 10:42:32.332318 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.auth.core [None req-41023c49-2f28-43a7-abb0-d476ecf414fb None None] MFA Rules not processed for user `bac4e6b76fe64217ac8fa44e71cffe6a`. Rule list: `[]` (Enabled: `True`). {{(pid=80143) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 10:42:32.333817 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-41023c49-2f28-43a7-abb0-d476ecf414fb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 10:42:32.333982 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f747488-078e-43e0-b8ce-991c43630f06 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 10:42:32.334586 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f747488-078e-43e0-b8ce-991c43630f06 None admin] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:42:32.334868 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f747488-078e-43e0-b8ce-991c43630f06 None admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:42:32.335177 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f747488-078e-43e0-b8ce-991c43630f06 None admin] PATH_INFO: `/v3/projects` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:42:32.335878 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f747488-078e-43e0-b8ce-991c43630f06 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80144) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 10:42:32.336187 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f747488-078e-43e0-b8ce-991c43630f06 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 10:42:32.338127 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f747488-078e-43e0-b8ce-991c43630f06 None admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 10:42:32.350573 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 436/856] 10.222.0.25 () {56 vars in 1148 bytes} [Sun Aug 7 10:42:32 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 10:42:32.357759 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f747488-078e-43e0-b8ce-991c43630f06 None admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 10:42:32.422946 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 421/857] 10.222.0.25 () {58 vars in 1024 bytes} [Sun Aug 7 10:42:32 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 07 10:42:32.440903 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41023c49-2f28-43a7-abb0-d476ecf414fb None None] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 10:42:32.477365 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f747488-078e-43e0-b8ce-991c43630f06 None admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 10:42:32.522654 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=100 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 10:42:32.528500 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a593096b-aaa1-4b44-bfef-814e974f29a8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 10:42:32.529113 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a593096b-aaa1-4b44-bfef-814e974f29a8 None admin] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:42:32.529399 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a593096b-aaa1-4b44-bfef-814e974f29a8 None admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:42:32.529659 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a593096b-aaa1-4b44-bfef-814e974f29a8 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:42:32.530375 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a593096b-aaa1-4b44-bfef-814e974f29a8 None admin] RBAC: Adding query filter params (role.id=81b1f1bcee2a473088fbbd327c1c5b23, scope.project.id=6de1f3cac02c4bcdb05dd2337176033d, user.id=69490ea2a9a340bf91aca38bf5655b45) {{(pid=80144) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 10:42:32.530681 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a593096b-aaa1-4b44-bfef-814e974f29a8 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 10:42:32.532531 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a593096b-aaa1-4b44-bfef-814e974f29a8 None admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 10:42:32.539187 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=100 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 10:42:32.540017 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 437/858] 10.222.0.25 () {56 vars in 1374 bytes} [Sun Aug 7 10:42:32 2022] GET /identity/v3/role_assignments?user.id=69490ea2a9a340bf91aca38bf5655b45&scope.project.id=6de1f3cac02c4bcdb05dd2337176033d&role.id=81b1f1bcee2a473088fbbd327c1c5b23 => generated 608 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 10:42:32.566605 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41023c49-2f28-43a7-abb0-d476ecf414fb None None] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 10:42:32.607344 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f19e5e1b-6005-43a5-84e0-c26b9b03a884 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 10:42:32.608257 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f19e5e1b-6005-43a5-84e0-c26b9b03a884 None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:42:32.608724 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f19e5e1b-6005-43a5-84e0-c26b9b03a884 None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:42:32.609153 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f19e5e1b-6005-43a5-84e0-c26b9b03a884 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:42:32.622150 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f19e5e1b-6005-43a5-84e0-c26b9b03a884 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 07 10:42:32.627478 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application [None req-f19e5e1b-6005-43a5-84e0-c26b9b03a884 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 07 10:42:32.627478 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 07 10:42:32.627478 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 07 10:42:32.627478 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 07 10:42:32.627478 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 07 10:42:32.627478 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 07 10:42:32.627478 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 07 10:42:32.627478 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 07 10:42:32.627478 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 07 10:42:32.627478 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 07 10:42:32.627478 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 07 10:42:32.627478 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 07 10:42:32.627478 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 07 10:42:32.627478 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 07 10:42:32.627478 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 07 10:42:32.627478 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 07 10:42:32.628663 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 07 10:42:32.628663 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 07 10:42:32.628663 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 07 10:42:32.628663 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 07 10:42:32.628663 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 07 10:42:32.628663 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 07 10:42:32.628663 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 07 10:42:32.628663 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 07 10:42:32.628663 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 07 10:42:32.628663 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.get_or_create( Aug 07 10:42:32.628663 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 07 10:42:32.628663 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application with Lock( Aug 07 10:42:32.628663 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 07 10:42:32.628663 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self._enter() Aug 07 10:42:32.628663 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 07 10:42:32.628663 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 07 10:42:32.629683 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 07 10:42:32.629683 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.creator() Aug 07 10:42:32.629683 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 07 10:42:32.629683 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application created_value = creator( Aug 07 10:42:32.629683 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 07 10:42:32.629683 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 07 10:42:32.629683 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 07 10:42:32.629683 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 07 10:42:32.629683 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 07 10:42:32.629683 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 07 10:42:32.629683 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 07 10:42:32.629683 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application Aug 07 10:42:32.637648 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 422/859] 10.222.0.25 () {56 vars in 1102 bytes} [Sun Aug 7 10:42:32 2022] GET /identity/v3/roles/service => generated 85 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 07 10:42:32.639129 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a593096b-aaa1-4b44-bfef-814e974f29a8 None admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 10:42:32.753867 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a593096b-aaa1-4b44-bfef-814e974f29a8 None admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 10:42:32.792007 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3049ee6a-d151-4b54-b09d-cabb18eb8a87 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 10:42:32.792909 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3049ee6a-d151-4b54-b09d-cabb18eb8a87 None admin] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:42:32.793432 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3049ee6a-d151-4b54-b09d-cabb18eb8a87 None admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:42:32.793873 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3049ee6a-d151-4b54-b09d-cabb18eb8a87 None admin] PATH_INFO: `/v3/roles` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:42:32.794715 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3049ee6a-d151-4b54-b09d-cabb18eb8a87 None admin] RBAC: Adding query filter params (name=service) {{(pid=80144) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 10:42:32.795214 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3049ee6a-d151-4b54-b09d-cabb18eb8a87 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 10:42:32.796712 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3049ee6a-d151-4b54-b09d-cabb18eb8a87 None admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 10:42:32.805564 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 438/860] 10.222.0.25 () {56 vars in 1103 bytes} [Sun Aug 7 10:42:32 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 10:42:32.811892 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f19e5e1b-6005-43a5-84e0-c26b9b03a884 None admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 10:42:32.920898 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f19e5e1b-6005-43a5-84e0-c26b9b03a884 None admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 10:42:32.958146 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d32244ae-2894-4d87-a67f-a41a57c7288f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 10:42:32.958842 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d32244ae-2894-4d87-a67f-a41a57c7288f None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:42:32.959168 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d32244ae-2894-4d87-a67f-a41a57c7288f None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:42:32.959440 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d32244ae-2894-4d87-a67f-a41a57c7288f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:42:32.970602 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d32244ae-2894-4d87-a67f-a41a57c7288f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 10:42:32.972087 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d32244ae-2894-4d87-a67f-a41a57c7288f None admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 10:42:32.973785 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=100 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 10:42:32.988265 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application [None req-d32244ae-2894-4d87-a67f-a41a57c7288f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 07 10:42:32.988265 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 07 10:42:32.988265 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 07 10:42:32.988265 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 07 10:42:32.988265 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 07 10:42:32.988265 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 07 10:42:32.988265 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 07 10:42:32.988265 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 07 10:42:32.988265 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 07 10:42:32.988265 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 07 10:42:32.988265 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 07 10:42:32.988265 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 07 10:42:32.988265 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 07 10:42:32.988265 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 07 10:42:32.988265 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 07 10:42:32.988265 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 07 10:42:32.989258 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 07 10:42:32.989258 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 07 10:42:32.989258 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 07 10:42:32.989258 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 07 10:42:32.989258 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 07 10:42:32.989258 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.get_or_create( Aug 07 10:42:32.989258 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 07 10:42:32.989258 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application with Lock( Aug 07 10:42:32.989258 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 07 10:42:32.989258 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self._enter() Aug 07 10:42:32.989258 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 07 10:42:32.989258 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 07 10:42:32.989258 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 07 10:42:32.989258 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.creator() Aug 07 10:42:32.989258 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 07 10:42:32.989258 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application created_value = creator( Aug 07 10:42:32.989258 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 07 10:42:32.990138 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 07 10:42:32.990138 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 07 10:42:32.990138 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application Aug 07 10:42:32.994520 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 423/861] 10.222.0.25 () {56 vars in 1108 bytes} [Sun Aug 7 10:42:32 2022] GET /identity/v3/domains/Default => generated 87 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 07 10:42:32.995916 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3049ee6a-d151-4b54-b09d-cabb18eb8a87 None admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 10:42:33.035527 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=100 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 10:42:33.109155 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3049ee6a-d151-4b54-b09d-cabb18eb8a87 None admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 10:42:33.143470 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9821607a-1186-4ead-a78e-8da5494b7752 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 10:42:33.144082 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9821607a-1186-4ead-a78e-8da5494b7752 None admin] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:42:33.144361 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9821607a-1186-4ead-a78e-8da5494b7752 None admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:42:33.144634 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9821607a-1186-4ead-a78e-8da5494b7752 None admin] PATH_INFO: `/v3/domains` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:42:33.145276 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9821607a-1186-4ead-a78e-8da5494b7752 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80144) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 10:42:33.145580 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9821607a-1186-4ead-a78e-8da5494b7752 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 10:42:33.146998 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9821607a-1186-4ead-a78e-8da5494b7752 None admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 10:42:33.164207 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 439/862] 10.222.0.25 () {56 vars in 1109 bytes} [Sun Aug 7 10:42:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 10:42:33.165567 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d32244ae-2894-4d87-a67f-a41a57c7288f None admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 10:42:33.208173 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c542d878-f6fa-4254-a2f0-4ba82ab9161b None None] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:42:33.208902 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c542d878-f6fa-4254-a2f0-4ba82ab9161b None None] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:42:33.209253 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c542d878-f6fa-4254-a2f0-4ba82ab9161b None None] PATH_INFO: `/` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:42:33.211306 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 440/863] 10.222.0.25 () {54 vars in 927 bytes} [Sun Aug 7 10:42:33 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 07 10:42:33.217422 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24070fcf-3f9d-4a41-987c-67ed4fdba293 None None] REQUEST_METHOD: `POST` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:42:33.217711 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24070fcf-3f9d-4a41-987c-67ed4fdba293 None None] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:42:33.217978 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24070fcf-3f9d-4a41-987c-67ed4fdba293 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:42:33.241639 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.auth.core [None req-24070fcf-3f9d-4a41-987c-67ed4fdba293 None None] MFA Rules not processed for user `bac4e6b76fe64217ac8fa44e71cffe6a`. Rule list: `[]` (Enabled: `True`). {{(pid=80144) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 10:42:33.243478 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.fernet_utils [None req-24070fcf-3f9d-4a41-987c-67ed4fdba293 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80144) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 10:42:33.281674 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d32244ae-2894-4d87-a67f-a41a57c7288f None admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 10:42:33.314637 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 441/864] 10.222.0.25 () {58 vars in 1024 bytes} [Sun Aug 7 10:42:33 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 07 10:42:33.322874 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acbaa103-091d-4b98-8fbc-29793ac97bd0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 10:42:33.323661 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acbaa103-091d-4b98-8fbc-29793ac97bd0 None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:42:33.323943 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acbaa103-091d-4b98-8fbc-29793ac97bd0 None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:42:33.324218 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acbaa103-091d-4b98-8fbc-29793ac97bd0 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:42:33.333032 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acbaa103-091d-4b98-8fbc-29793ac97bd0 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 10:42:33.334540 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acbaa103-091d-4b98-8fbc-29793ac97bd0 None admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 10:42:33.339736 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24070fcf-3f9d-4a41-987c-67ed4fdba293 None None] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 10:42:33.347622 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application [None req-acbaa103-091d-4b98-8fbc-29793ac97bd0 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 07 10:42:33.347622 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 07 10:42:33.347622 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 07 10:42:33.347622 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 07 10:42:33.347622 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 07 10:42:33.347622 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 07 10:42:33.347622 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 07 10:42:33.347622 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 07 10:42:33.347622 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 07 10:42:33.347622 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 07 10:42:33.347622 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 07 10:42:33.347622 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 07 10:42:33.347622 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 07 10:42:33.347622 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 07 10:42:33.347622 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 07 10:42:33.347622 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 07 10:42:33.348647 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 07 10:42:33.348647 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 07 10:42:33.348647 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 07 10:42:33.348647 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 07 10:42:33.348647 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 07 10:42:33.348647 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 07 10:42:33.348647 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 07 10:42:33.348647 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 07 10:42:33.348647 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 07 10:42:33.348647 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.get_or_create( Aug 07 10:42:33.348647 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 07 10:42:33.348647 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application with Lock( Aug 07 10:42:33.348647 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 07 10:42:33.348647 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self._enter() Aug 07 10:42:33.348647 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 07 10:42:33.348647 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 07 10:42:33.348647 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 07 10:42:33.348647 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.creator() Aug 07 10:42:33.349574 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 07 10:42:33.349574 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application created_value = creator( Aug 07 10:42:33.349574 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 07 10:42:33.349574 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 07 10:42:33.349574 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 07 10:42:33.349574 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 07 10:42:33.349574 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 07 10:42:33.349574 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 07 10:42:33.349574 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 07 10:42:33.349574 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application Aug 07 10:42:33.354090 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 424/865] 10.222.0.25 () {56 vars in 1117 bytes} [Sun Aug 7 10:42:33 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 07 10:42:33.355579 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acbaa103-091d-4b98-8fbc-29793ac97bd0 None admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 10:42:33.373342 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=100 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 10:42:33.432265 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=100 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 10:42:33.469396 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24070fcf-3f9d-4a41-987c-67ed4fdba293 None None] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 10:42:33.491215 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acbaa103-091d-4b98-8fbc-29793ac97bd0 None admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 10:42:33.505654 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1eb85f28-8354-4b08-831d-b434271aaa1e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 10:42:33.506271 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eb85f28-8354-4b08-831d-b434271aaa1e None admin] REQUEST_METHOD: `POST` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:42:33.506551 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eb85f28-8354-4b08-831d-b434271aaa1e None admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:42:33.506825 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eb85f28-8354-4b08-831d-b434271aaa1e None admin] PATH_INFO: `/v3/services` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:42:33.507499 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1eb85f28-8354-4b08-831d-b434271aaa1e None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 10:42:33.508847 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1eb85f28-8354-4b08-831d-b434271aaa1e None admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 10:42:33.526372 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 442/866] 10.222.0.25 () {60 vars in 1141 bytes} [Sun Aug 7 10:42:33 2022] POST /identity/v3/services => generated 251 bytes in 187 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 07 10:42:33.539863 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6536ca33-0e7d-4874-bba2-fb75233e14f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 10:42:33.541176 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6536ca33-0e7d-4874-bba2-fb75233e14f4 None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:42:33.541448 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6536ca33-0e7d-4874-bba2-fb75233e14f4 None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:42:33.541820 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6536ca33-0e7d-4874-bba2-fb75233e14f4 None admin] PATH_INFO: `/v3/users` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:42:33.542455 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6536ca33-0e7d-4874-bba2-fb75233e14f4 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80143) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 10:42:33.542756 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6536ca33-0e7d-4874-bba2-fb75233e14f4 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 10:42:33.544423 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6536ca33-0e7d-4874-bba2-fb75233e14f4 None admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 10:42:33.566507 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 425/867] 10.222.0.25 () {56 vars in 1149 bytes} [Sun Aug 7 10:42:33 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 10:42:33.572703 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1eb85f28-8354-4b08-831d-b434271aaa1e None admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 10:42:33.674887 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 10:42:33.692766 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1eb85f28-8354-4b08-831d-b434271aaa1e None admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 10:42:33.728110 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf520286-da4f-4322-8ecf-62d14ba51288 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 10:42:33.728743 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf520286-da4f-4322-8ecf-62d14ba51288 None admin] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:42:33.729027 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf520286-da4f-4322-8ecf-62d14ba51288 None admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:42:33.729305 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf520286-da4f-4322-8ecf-62d14ba51288 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:42:33.802553 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf520286-da4f-4322-8ecf-62d14ba51288 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 10:42:33.804021 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf520286-da4f-4322-8ecf-62d14ba51288 None admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 10:42:33.890283 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=100 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 10:42:33.907692 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 443/868] 10.222.0.25 () {58 vars in 1282 bytes} [Sun Aug 7 10:42:33 2022] GET /identity/v3/auth/tokens => generated 1262 bytes in 335 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 07 10:42:33.916623 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6536ca33-0e7d-4874-bba2-fb75233e14f4 None admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 10:42:34.021869 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6536ca33-0e7d-4874-bba2-fb75233e14f4 None admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 10:42:34.031876 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=100 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 10:42:34.059237 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53064f6a-b6f4-4e6f-9d61-fffdd8e175fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 10:42:34.059871 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53064f6a-b6f4-4e6f-9d61-fffdd8e175fb None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:42:34.060146 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53064f6a-b6f4-4e6f-9d61-fffdd8e175fb None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:42:34.060415 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53064f6a-b6f4-4e6f-9d61-fffdd8e175fb None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:42:34.071094 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53064f6a-b6f4-4e6f-9d61-fffdd8e175fb None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 10:42:34.072576 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53064f6a-b6f4-4e6f-9d61-fffdd8e175fb None admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 10:42:34.088990 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application [None req-53064f6a-b6f4-4e6f-9d61-fffdd8e175fb None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 07 10:42:34.088990 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 07 10:42:34.088990 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 07 10:42:34.088990 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 07 10:42:34.088990 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 07 10:42:34.088990 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 07 10:42:34.088990 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 07 10:42:34.088990 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 07 10:42:34.088990 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 07 10:42:34.088990 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 07 10:42:34.088990 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 07 10:42:34.088990 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 07 10:42:34.088990 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 07 10:42:34.088990 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 07 10:42:34.088990 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 07 10:42:34.088990 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 07 10:42:34.089973 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 07 10:42:34.089973 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 07 10:42:34.089973 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 07 10:42:34.089973 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 07 10:42:34.089973 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 07 10:42:34.089973 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.get_or_create( Aug 07 10:42:34.089973 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 07 10:42:34.089973 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application with Lock( Aug 07 10:42:34.089973 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 07 10:42:34.089973 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self._enter() Aug 07 10:42:34.089973 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 07 10:42:34.089973 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 07 10:42:34.089973 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 07 10:42:34.089973 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.creator() Aug 07 10:42:34.089973 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 07 10:42:34.089973 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application created_value = creator( Aug 07 10:42:34.089973 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 07 10:42:34.090870 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 07 10:42:34.090870 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 07 10:42:34.090870 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application Aug 07 10:42:34.090870 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 426/869] 10.222.0.25 () {56 vars in 1108 bytes} [Sun Aug 7 10:42:33 2022] GET /identity/v3/domains/Default => generated 87 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 07 10:42:34.096956 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf520286-da4f-4322-8ecf-62d14ba51288 None admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 10:42:34.198833 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf520286-da4f-4322-8ecf-62d14ba51288 None admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 10:42:34.234950 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8343136e-1203-40c2-8b85-adabeba7a4ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 10:42:34.234950 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8343136e-1203-40c2-8b85-adabeba7a4ef None admin] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:42:34.235220 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8343136e-1203-40c2-8b85-adabeba7a4ef None admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:42:34.235480 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8343136e-1203-40c2-8b85-adabeba7a4ef None admin] PATH_INFO: `/v3/domains` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:42:34.238178 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8343136e-1203-40c2-8b85-adabeba7a4ef None admin] RBAC: Adding query filter params (name=Default) {{(pid=80144) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 10:42:34.238178 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8343136e-1203-40c2-8b85-adabeba7a4ef None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 10:42:34.238178 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8343136e-1203-40c2-8b85-adabeba7a4ef None admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 10:42:34.250745 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 444/870] 10.222.0.25 () {56 vars in 1109 bytes} [Sun Aug 7 10:42:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 10:42:34.259452 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53064f6a-b6f4-4e6f-9d61-fffdd8e175fb None admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 10:42:34.338863 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2728547a-4ba1-4b47-91a4-d297bb30a5c1 None None] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:42:34.339264 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2728547a-4ba1-4b47-91a4-d297bb30a5c1 None None] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:42:34.339694 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2728547a-4ba1-4b47-91a4-d297bb30a5c1 None None] PATH_INFO: `/` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:42:34.341446 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 445/871] 10.222.0.25 () {54 vars in 927 bytes} [Sun Aug 7 10:42:34 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 07 10:42:34.346384 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70c06f25-ebbc-4841-a702-5c6c3d27012e None None] REQUEST_METHOD: `POST` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:42:34.346773 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70c06f25-ebbc-4841-a702-5c6c3d27012e None None] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:42:34.347186 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70c06f25-ebbc-4841-a702-5c6c3d27012e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:42:34.366245 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.auth.core [None req-70c06f25-ebbc-4841-a702-5c6c3d27012e None None] MFA Rules not processed for user `bac4e6b76fe64217ac8fa44e71cffe6a`. Rule list: `[]` (Enabled: `True`). {{(pid=80144) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 10:42:34.367693 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.fernet_utils [None req-70c06f25-ebbc-4841-a702-5c6c3d27012e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80144) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 10:42:34.370444 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53064f6a-b6f4-4e6f-9d61-fffdd8e175fb None admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 10:42:34.413479 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7592b05c-d705-479e-9893-e84dbb1dc7f1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 10:42:34.416548 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=100 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 10:42:34.418051 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7592b05c-d705-479e-9893-e84dbb1dc7f1 None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:42:34.418421 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7592b05c-d705-479e-9893-e84dbb1dc7f1 None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:42:34.418759 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7592b05c-d705-479e-9893-e84dbb1dc7f1 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:42:34.429473 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7592b05c-d705-479e-9893-e84dbb1dc7f1 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 10:42:34.431785 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7592b05c-d705-479e-9893-e84dbb1dc7f1 None admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 10:42:34.446746 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application [None req-7592b05c-d705-479e-9893-e84dbb1dc7f1 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 07 10:42:34.446746 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 07 10:42:34.446746 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 07 10:42:34.446746 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 07 10:42:34.446746 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 07 10:42:34.446746 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 07 10:42:34.446746 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 07 10:42:34.446746 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 07 10:42:34.446746 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 07 10:42:34.446746 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 07 10:42:34.446746 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 07 10:42:34.446746 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 07 10:42:34.446746 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 07 10:42:34.446746 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 07 10:42:34.446746 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 07 10:42:34.447902 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 07 10:42:34.447902 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 07 10:42:34.447902 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 07 10:42:34.447902 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 07 10:42:34.447902 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 07 10:42:34.447902 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 07 10:42:34.447902 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.get_or_create( Aug 07 10:42:34.447902 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 07 10:42:34.447902 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application with Lock( Aug 07 10:42:34.447902 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 07 10:42:34.447902 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self._enter() Aug 07 10:42:34.447902 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 07 10:42:34.447902 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 07 10:42:34.447902 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 07 10:42:34.447902 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.creator() Aug 07 10:42:34.447902 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 07 10:42:34.447902 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application created_value = creator( Aug 07 10:42:34.448820 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 07 10:42:34.448820 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 07 10:42:34.448820 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 07 10:42:34.448820 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 07 10:42:34.448820 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 07 10:42:34.448820 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 07 10:42:34.448820 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 07 10:42:34.448820 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application Aug 07 10:42:34.448820 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 427/872] 10.222.0.25 () {56 vars in 1111 bytes} [Sun Aug 7 10:42:34 2022] GET /identity/v3/projects/service => generated 88 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 07 10:42:34.458989 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7592b05c-d705-479e-9893-e84dbb1dc7f1 None admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 10:42:34.459509 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 446/873] 10.222.0.25 () {58 vars in 1024 bytes} [Sun Aug 7 10:42:34 2022] POST /identity/v3/auth/tokens => generated 1262 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 07 10:42:34.474990 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70c06f25-ebbc-4841-a702-5c6c3d27012e None None] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 10:42:34.480702 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=100 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 10:42:34.583430 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7592b05c-d705-479e-9893-e84dbb1dc7f1 None admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 10:42:34.592355 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70c06f25-ebbc-4841-a702-5c6c3d27012e None None] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 10:42:34.621209 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dfb3d10-705e-41b8-92ab-1735335c58a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 10:42:34.622167 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dfb3d10-705e-41b8-92ab-1735335c58a9 None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:42:34.622681 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dfb3d10-705e-41b8-92ab-1735335c58a9 None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:42:34.623189 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dfb3d10-705e-41b8-92ab-1735335c58a9 None admin] PATH_INFO: `/v3/projects` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:42:34.624083 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dfb3d10-705e-41b8-92ab-1735335c58a9 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80143) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 10:42:34.624592 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dfb3d10-705e-41b8-92ab-1735335c58a9 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 10:42:34.626269 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dfb3d10-705e-41b8-92ab-1735335c58a9 None admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 10:42:34.631790 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bb59771-c891-4dd2-88b4-4c034beaa851 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 10:42:34.632403 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bb59771-c891-4dd2-88b4-4c034beaa851 None admin] REQUEST_METHOD: `POST` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:42:34.632681 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bb59771-c891-4dd2-88b4-4c034beaa851 None admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:42:34.632952 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bb59771-c891-4dd2-88b4-4c034beaa851 None admin] PATH_INFO: `/v3/services` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:42:34.633591 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bb59771-c891-4dd2-88b4-4c034beaa851 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 10:42:34.635767 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bb59771-c891-4dd2-88b4-4c034beaa851 None admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 10:42:34.641496 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 428/874] 10.222.0.25 () {56 vars in 1148 bytes} [Sun Aug 7 10:42:34 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 07 10:42:34.650090 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dfb3d10-705e-41b8-92ab-1735335c58a9 None admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 10:42:34.662959 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 447/875] 10.222.0.25 () {60 vars in 1141 bytes} [Sun Aug 7 10:42:34 2022] POST /identity/v3/services => generated 269 bytes in 181 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 07 10:42:34.716070 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75c84abc-24ae-4321-9f56-3cb2b2295fe5 None None] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:42:34.716423 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75c84abc-24ae-4321-9f56-3cb2b2295fe5 None None] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:42:34.716631 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75c84abc-24ae-4321-9f56-3cb2b2295fe5 None None] PATH_INFO: `/` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:42:34.718271 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 448/876] 10.222.0.25 () {54 vars in 927 bytes} [Sun Aug 7 10:42:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 07 10:42:34.723891 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5260423-eff1-4f94-8967-63a16d49739b None None] REQUEST_METHOD: `POST` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:42:34.724190 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5260423-eff1-4f94-8967-63a16d49739b None None] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:42:34.724463 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5260423-eff1-4f94-8967-63a16d49739b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:42:34.745139 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.auth.core [None req-d5260423-eff1-4f94-8967-63a16d49739b None None] MFA Rules not processed for user `bac4e6b76fe64217ac8fa44e71cffe6a`. Rule list: `[]` (Enabled: `True`). {{(pid=80144) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 10:42:34.746462 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.fernet_utils [None req-d5260423-eff1-4f94-8967-63a16d49739b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80144) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 10:42:34.752330 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=100 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 10:42:34.776151 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dfb3d10-705e-41b8-92ab-1735335c58a9 None admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 10:42:34.783401 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 10:42:34.823230 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea07d487-ffeb-4613-8092-3766babf9c3c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 10:42:34.823856 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea07d487-ffeb-4613-8092-3766babf9c3c None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:42:34.824136 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea07d487-ffeb-4613-8092-3766babf9c3c None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:42:34.824406 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea07d487-ffeb-4613-8092-3766babf9c3c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:42:34.825128 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea07d487-ffeb-4613-8092-3766babf9c3c None admin] RBAC: Adding query filter params (role.id=fbf375e9760340c9aa9529333eeb25f9, scope.project.id=6de1f3cac02c4bcdb05dd2337176033d, user.id=de1d6c24b1a14d10a20d82a4c65793d8) {{(pid=80143) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 10:42:34.825433 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea07d487-ffeb-4613-8092-3766babf9c3c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 10:42:34.826917 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea07d487-ffeb-4613-8092-3766babf9c3c None admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 10:42:34.833616 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 429/877] 10.222.0.25 () {56 vars in 1374 bytes} [Sun Aug 7 10:42:34 2022] GET /identity/v3/role_assignments?user.id=de1d6c24b1a14d10a20d82a4c65793d8&scope.project.id=6de1f3cac02c4bcdb05dd2337176033d&role.id=fbf375e9760340c9aa9529333eeb25f9 => generated 608 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 07 10:42:34.847191 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 449/878] 10.222.0.25 () {58 vars in 1024 bytes} [Sun Aug 7 10:42:34 2022] POST /identity/v3/auth/tokens => generated 1372 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 07 10:42:34.865184 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea07d487-ffeb-4613-8092-3766babf9c3c None admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 10:42:34.983910 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea07d487-ffeb-4613-8092-3766babf9c3c None admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 10:42:35.022286 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c6a6517-7559-424b-a511-263316908f49 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 10:42:35.023191 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c6a6517-7559-424b-a511-263316908f49 None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:42:35.023636 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c6a6517-7559-424b-a511-263316908f49 None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:42:35.024053 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c6a6517-7559-424b-a511-263316908f49 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:42:35.025046 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c6a6517-7559-424b-a511-263316908f49 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 10:42:35.026688 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c6a6517-7559-424b-a511-263316908f49 None admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 10:42:35.035862 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application [None req-2c6a6517-7559-424b-a511-263316908f49 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 07 10:42:35.035862 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 07 10:42:35.035862 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 07 10:42:35.035862 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 07 10:42:35.035862 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 07 10:42:35.035862 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 07 10:42:35.035862 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 07 10:42:35.035862 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 07 10:42:35.035862 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 07 10:42:35.035862 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application Aug 07 10:42:35.035862 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 07 10:42:35.035862 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application Aug 07 10:42:35.035862 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 07 10:42:35.035862 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 07 10:42:35.035862 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 07 10:42:35.035862 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 07 10:42:35.035862 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 07 10:42:35.037078 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 07 10:42:35.037078 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 07 10:42:35.037078 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 07 10:42:35.037078 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 07 10:42:35.037078 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 07 10:42:35.037078 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 07 10:42:35.037078 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 07 10:42:35.037078 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 07 10:42:35.037078 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 07 10:42:35.037078 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 07 10:42:35.037078 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 07 10:42:35.037078 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 07 10:42:35.037078 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 07 10:42:35.037078 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 07 10:42:35.037078 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 07 10:42:35.037078 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.get_or_create( Aug 07 10:42:35.037944 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 07 10:42:35.037944 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application with Lock( Aug 07 10:42:35.037944 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 07 10:42:35.037944 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self._enter() Aug 07 10:42:35.037944 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 07 10:42:35.037944 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 07 10:42:35.037944 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 07 10:42:35.037944 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.creator() Aug 07 10:42:35.037944 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 07 10:42:35.037944 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application created_value = creator( Aug 07 10:42:35.037944 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 07 10:42:35.037944 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 07 10:42:35.037944 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 07 10:42:35.037944 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application Aug 07 10:42:35.039783 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 430/879] 10.222.0.25 () {56 vars in 1126 bytes} [Sun Aug 7 10:42:34 2022] GET /identity/v3/services/object-store => generated 93 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 07 10:42:35.045467 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5260423-eff1-4f94-8967-63a16d49739b None None] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 10:42:35.157617 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5260423-eff1-4f94-8967-63a16d49739b None None] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 10:42:35.194143 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d07bad33-db29-4065-b1fb-cefe32adee11 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 10:42:35.194783 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d07bad33-db29-4065-b1fb-cefe32adee11 None admin] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:42:35.195095 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d07bad33-db29-4065-b1fb-cefe32adee11 None admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:42:35.195426 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d07bad33-db29-4065-b1fb-cefe32adee11 None admin] PATH_INFO: `/v3/services` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:42:35.196371 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d07bad33-db29-4065-b1fb-cefe32adee11 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=80144) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 10:42:35.197325 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d07bad33-db29-4065-b1fb-cefe32adee11 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 10:42:35.199125 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d07bad33-db29-4065-b1fb-cefe32adee11 None admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 10:42:35.204386 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 450/880] 10.222.0.25 () {56 vars in 1122 bytes} [Sun Aug 7 10:42:35 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 10:42:35.209864 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c6a6517-7559-424b-a511-263316908f49 None admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 10:42:35.214689 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=100 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 10:42:35.320328 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c6a6517-7559-424b-a511-263316908f49 None admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 10:42:35.354887 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1afd14c1-3f93-4393-9811-8c3b16303da0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 10:42:35.355525 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1afd14c1-3f93-4393-9811-8c3b16303da0 None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:42:35.355856 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1afd14c1-3f93-4393-9811-8c3b16303da0 None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:42:35.356126 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1afd14c1-3f93-4393-9811-8c3b16303da0 None admin] PATH_INFO: `/v3/services` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:42:35.356753 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1afd14c1-3f93-4393-9811-8c3b16303da0 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=80143) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 10:42:35.357052 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1afd14c1-3f93-4393-9811-8c3b16303da0 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 10:42:35.358401 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1afd14c1-3f93-4393-9811-8c3b16303da0 None admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 10:42:35.363751 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 431/881] 10.222.0.25 () {56 vars in 1122 bytes} [Sun Aug 7 10:42:35 2022] GET /identity/v3/services?type=object-store => generated 129 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 10:42:35.621330 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfb59f75-679b-4f12-b8df-91d2c409363e None None] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:42:35.621703 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfb59f75-679b-4f12-b8df-91d2c409363e None None] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:42:35.621902 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfb59f75-679b-4f12-b8df-91d2c409363e None None] PATH_INFO: `/` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:42:35.623534 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 451/882] 10.222.0.25 () {54 vars in 927 bytes} [Sun Aug 7 10:42:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 07 10:42:35.628698 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e150851c-b473-4472-9265-498c8231863b None None] REQUEST_METHOD: `POST` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:42:35.628991 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e150851c-b473-4472-9265-498c8231863b None None] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:42:35.629270 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e150851c-b473-4472-9265-498c8231863b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:42:35.647900 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.auth.core [None req-e150851c-b473-4472-9265-498c8231863b None None] MFA Rules not processed for user `bac4e6b76fe64217ac8fa44e71cffe6a`. Rule list: `[]` (Enabled: `True`). {{(pid=80143) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 10:42:35.649084 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-e150851c-b473-4472-9265-498c8231863b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 10:42:35.713193 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=100 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 10:42:35.723035 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 432/883] 10.222.0.25 () {58 vars in 1024 bytes} [Sun Aug 7 10:42:35 2022] POST /identity/v3/auth/tokens => generated 1372 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 07 10:42:35.741923 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfb59f75-679b-4f12-b8df-91d2c409363e None None] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 10:42:35.774501 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=100 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 10:42:35.850106 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfb59f75-679b-4f12-b8df-91d2c409363e None None] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 10:42:35.886093 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3921a02-59bc-464b-bb00-409e3dc5c132 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 10:42:35.886778 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3921a02-59bc-464b-bb00-409e3dc5c132 None admin] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:42:35.887086 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3921a02-59bc-464b-bb00-409e3dc5c132 None admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:42:35.887356 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3921a02-59bc-464b-bb00-409e3dc5c132 None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:42:35.888137 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3921a02-59bc-464b-bb00-409e3dc5c132 None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 10:42:35.889483 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3921a02-59bc-464b-bb00-409e3dc5c132 None admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 10:42:35.898452 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application [None req-b3921a02-59bc-464b-bb00-409e3dc5c132 None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Aug 07 10:42:35.898452 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 07 10:42:35.898452 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 07 10:42:35.898452 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 07 10:42:35.898452 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 07 10:42:35.898452 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 07 10:42:35.898452 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 07 10:42:35.898452 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 07 10:42:35.898452 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Aug 07 10:42:35.898452 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application Aug 07 10:42:35.898452 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 07 10:42:35.898452 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application Aug 07 10:42:35.898452 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 07 10:42:35.898452 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 07 10:42:35.898452 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 07 10:42:35.898452 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 07 10:42:35.898452 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 07 10:42:35.899463 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 07 10:42:35.899463 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 07 10:42:35.899463 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 07 10:42:35.899463 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 07 10:42:35.899463 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 07 10:42:35.899463 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 07 10:42:35.899463 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 07 10:42:35.899463 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 07 10:42:35.899463 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 07 10:42:35.899463 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 07 10:42:35.899463 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 07 10:42:35.899463 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 07 10:42:35.899463 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 07 10:42:35.899463 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 07 10:42:35.899463 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 07 10:42:35.899463 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self.get_or_create( Aug 07 10:42:35.901086 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 07 10:42:35.901086 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application with Lock( Aug 07 10:42:35.901086 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 07 10:42:35.901086 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self._enter() Aug 07 10:42:35.901086 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 07 10:42:35.901086 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 07 10:42:35.901086 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 07 10:42:35.901086 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self.creator() Aug 07 10:42:35.901086 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 07 10:42:35.901086 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application created_value = creator( Aug 07 10:42:35.901086 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 07 10:42:35.901086 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 07 10:42:35.901086 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Aug 07 10:42:35.901086 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application Aug 07 10:42:35.905165 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 452/884] 10.222.0.25 () {56 vars in 1114 bytes} [Sun Aug 7 10:42:35 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 07 10:42:35.906547 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e150851c-b473-4472-9265-498c8231863b None None] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 10:42:36.010280 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e150851c-b473-4472-9265-498c8231863b None None] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 10:42:36.044716 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-423f16c1-da35-4c36-87b6-05b6a65c7be7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 10:42:36.045307 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-423f16c1-da35-4c36-87b6-05b6a65c7be7 None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:42:36.045584 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-423f16c1-da35-4c36-87b6-05b6a65c7be7 None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:42:36.045850 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-423f16c1-da35-4c36-87b6-05b6a65c7be7 None admin] PATH_INFO: `/v3/services` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:42:36.046476 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-423f16c1-da35-4c36-87b6-05b6a65c7be7 None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=80143) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 10:42:36.046780 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-423f16c1-da35-4c36-87b6-05b6a65c7be7 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 10:42:36.048156 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-423f16c1-da35-4c36-87b6-05b6a65c7be7 None admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 10:42:36.053111 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 433/885] 10.222.0.25 () {56 vars in 1114 bytes} [Sun Aug 7 10:42:35 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 10:42:36.059422 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3921a02-59bc-464b-bb00-409e3dc5c132 None admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 10:42:36.164154 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3921a02-59bc-464b-bb00-409e3dc5c132 None admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 10:42:36.175302 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=100 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 10:42:36.203964 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf0691d4-814f-452a-bb74-aefd88e3e233 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 10:42:36.205379 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf0691d4-814f-452a-bb74-aefd88e3e233 None admin] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:42:36.205858 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf0691d4-814f-452a-bb74-aefd88e3e233 None admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:42:36.206327 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf0691d4-814f-452a-bb74-aefd88e3e233 None admin] PATH_INFO: `/v3/services` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:42:36.207278 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf0691d4-814f-452a-bb74-aefd88e3e233 None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=80144) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 10:42:36.207797 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf0691d4-814f-452a-bb74-aefd88e3e233 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 10:42:36.209350 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf0691d4-814f-452a-bb74-aefd88e3e233 None admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 10:42:36.214805 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 453/886] 10.222.0.25 () {56 vars in 1114 bytes} [Sun Aug 7 10:42:36 2022] GET /identity/v3/services?type=volumev3 => generated 362 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 10:42:36.224003 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-423f16c1-da35-4c36-87b6-05b6a65c7be7 None admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 10:42:36.299575 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=100 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 10:42:36.334356 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-423f16c1-da35-4c36-87b6-05b6a65c7be7 None admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 10:42:36.368869 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8ab4df8-4584-4aeb-a34b-e5d38543f0c4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 10:42:36.369474 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8ab4df8-4584-4aeb-a34b-e5d38543f0c4 None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:42:36.369748 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8ab4df8-4584-4aeb-a34b-e5d38543f0c4 None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:42:36.370016 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8ab4df8-4584-4aeb-a34b-e5d38543f0c4 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:42:36.370711 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8ab4df8-4584-4aeb-a34b-e5d38543f0c4 None admin] RBAC: Adding query filter params (interface=public, service_id=da4adb40f4c145d696e37e672d974848, region_id=RegionOne) {{(pid=80143) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 10:42:36.371043 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8ab4df8-4584-4aeb-a34b-e5d38543f0c4 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 10:42:36.372390 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8ab4df8-4584-4aeb-a34b-e5d38543f0c4 None admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 10:42:36.377821 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 434/887] 10.222.0.25 () {56 vars in 1251 bytes} [Sun Aug 7 10:42:36 2022] GET /identity/v3/endpoints?service_id=da4adb40f4c145d696e37e672d974848&interface=public®ion_id=RegionOne => generated 194 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 10:42:36.383373 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf0691d4-814f-452a-bb74-aefd88e3e233 None admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 10:42:36.486273 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf0691d4-814f-452a-bb74-aefd88e3e233 None admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 10:42:36.520985 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6803de6-98e6-430c-96fd-028de7717210 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 10:42:36.521606 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6803de6-98e6-430c-96fd-028de7717210 None admin] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:42:36.521885 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6803de6-98e6-430c-96fd-028de7717210 None admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:42:36.522155 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6803de6-98e6-430c-96fd-028de7717210 None admin] PATH_INFO: `/v3/services` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:42:36.522757 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6803de6-98e6-430c-96fd-028de7717210 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 10:42:36.524135 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6803de6-98e6-430c-96fd-028de7717210 None admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 10:42:36.535094 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 454/888] 10.222.0.25 () {56 vars in 1087 bytes} [Sun Aug 7 10:42:36 2022] GET /identity/v3/services => generated 1268 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 07 10:42:36.862735 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e08a09b-0a74-473d-afee-8cd103948978 None None] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:42:36.863141 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e08a09b-0a74-473d-afee-8cd103948978 None None] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:42:36.863423 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e08a09b-0a74-473d-afee-8cd103948978 None None] PATH_INFO: `/` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:42:36.864884 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 435/889] 10.222.0.25 () {54 vars in 927 bytes} [Sun Aug 7 10:42:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 07 10:42:36.870741 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90004c35-9fc4-44eb-9886-43da3bce6925 None None] REQUEST_METHOD: `POST` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:42:36.871308 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90004c35-9fc4-44eb-9886-43da3bce6925 None None] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:42:36.871784 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90004c35-9fc4-44eb-9886-43da3bce6925 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:42:36.889856 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.auth.core [None req-90004c35-9fc4-44eb-9886-43da3bce6925 None None] MFA Rules not processed for user `bac4e6b76fe64217ac8fa44e71cffe6a`. Rule list: `[]` (Enabled: `True`). {{(pid=80144) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 10:42:36.891382 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.fernet_utils [None req-90004c35-9fc4-44eb-9886-43da3bce6925 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80144) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 10:42:36.906074 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=100 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 10:42:36.963079 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 455/890] 10.222.0.25 () {58 vars in 1024 bytes} [Sun Aug 7 10:42:36 2022] POST /identity/v3/auth/tokens => generated 1372 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 07 10:42:36.985865 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e08a09b-0a74-473d-afee-8cd103948978 None None] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 10:42:37.052121 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8062892c-f473-4627-a612-8801506396e5 None None] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:42:37.052435 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8062892c-f473-4627-a612-8801506396e5 None None] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:42:37.052678 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8062892c-f473-4627-a612-8801506396e5 None None] PATH_INFO: `/` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:42:37.054257 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 456/891] 10.222.0.25 () {54 vars in 927 bytes} [Sun Aug 7 10:42:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 07 10:42:37.059280 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c94826d2-41d5-420a-a73f-f39283090a65 None None] REQUEST_METHOD: `POST` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:42:37.059562 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c94826d2-41d5-420a-a73f-f39283090a65 None None] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:42:37.059837 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c94826d2-41d5-420a-a73f-f39283090a65 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:42:37.078635 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.auth.core [None req-c94826d2-41d5-420a-a73f-f39283090a65 None None] MFA Rules not processed for user `bac4e6b76fe64217ac8fa44e71cffe6a`. Rule list: `[]` (Enabled: `True`). {{(pid=80144) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 10:42:37.079964 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.fernet_utils [None req-c94826d2-41d5-420a-a73f-f39283090a65 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80144) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 10:42:37.097295 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e08a09b-0a74-473d-afee-8cd103948978 None None] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 10:42:37.125661 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=100 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 10:42:37.138974 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8677bcb7-5343-435a-be07-074cf625478d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 10:42:37.138974 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8677bcb7-5343-435a-be07-074cf625478d None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:42:37.138974 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8677bcb7-5343-435a-be07-074cf625478d None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:42:37.138974 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8677bcb7-5343-435a-be07-074cf625478d None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:42:37.139412 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8677bcb7-5343-435a-be07-074cf625478d None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 10:42:37.140740 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8677bcb7-5343-435a-be07-074cf625478d None admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 10:42:37.152513 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application [None req-8677bcb7-5343-435a-be07-074cf625478d None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Aug 07 10:42:37.152513 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 07 10:42:37.152513 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 07 10:42:37.152513 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 07 10:42:37.152513 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 07 10:42:37.152513 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 07 10:42:37.152513 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 07 10:42:37.152513 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 07 10:42:37.152513 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Aug 07 10:42:37.152513 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application Aug 07 10:42:37.152513 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 07 10:42:37.152513 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application Aug 07 10:42:37.152513 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 07 10:42:37.152513 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 07 10:42:37.152513 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 07 10:42:37.152513 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 07 10:42:37.153553 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 07 10:42:37.153553 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 07 10:42:37.153553 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 07 10:42:37.153553 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 07 10:42:37.153553 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 07 10:42:37.153553 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 07 10:42:37.153553 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 07 10:42:37.153553 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 07 10:42:37.153553 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 07 10:42:37.153553 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 07 10:42:37.153553 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 07 10:42:37.153553 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 07 10:42:37.153553 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 07 10:42:37.153553 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 07 10:42:37.153553 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 07 10:42:37.153553 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 07 10:42:37.153553 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.get_or_create( Aug 07 10:42:37.154708 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 07 10:42:37.154708 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application with Lock( Aug 07 10:42:37.154708 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 07 10:42:37.154708 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self._enter() Aug 07 10:42:37.154708 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 07 10:42:37.154708 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 07 10:42:37.154708 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 07 10:42:37.154708 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.creator() Aug 07 10:42:37.154708 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 07 10:42:37.154708 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application created_value = creator( Aug 07 10:42:37.154708 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 07 10:42:37.154708 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 07 10:42:37.154708 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Aug 07 10:42:37.154708 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application Aug 07 10:42:37.154708 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 436/892] 10.222.0.25 () {56 vars in 1132 bytes} [Sun Aug 7 10:42:36 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 07 10:42:37.160465 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8677bcb7-5343-435a-be07-074cf625478d None admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 10:42:37.164140 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 457/893] 10.222.0.25 () {58 vars in 1024 bytes} [Sun Aug 7 10:42:37 2022] POST /identity/v3/auth/tokens => generated 1372 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 07 10:42:37.187719 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c94826d2-41d5-420a-a73f-f39283090a65 None None] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 10:42:37.285978 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8677bcb7-5343-435a-be07-074cf625478d None admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 10:42:37.286545 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=100 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 10:42:37.309240 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c94826d2-41d5-420a-a73f-f39283090a65 None None] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 10:42:37.329129 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-239f09fa-3e54-4c81-a674-e3bfc6f39d5b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 10:42:37.329740 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-239f09fa-3e54-4c81-a674-e3bfc6f39d5b None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:42:37.330022 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-239f09fa-3e54-4c81-a674-e3bfc6f39d5b None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:42:37.330299 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-239f09fa-3e54-4c81-a674-e3bfc6f39d5b None admin] PATH_INFO: `/v3/services` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:42:37.330953 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-239f09fa-3e54-4c81-a674-e3bfc6f39d5b None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=80143) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 10:42:37.331259 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-239f09fa-3e54-4c81-a674-e3bfc6f39d5b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 10:42:37.333051 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-239f09fa-3e54-4c81-a674-e3bfc6f39d5b None admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 10:42:37.339813 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 437/894] 10.222.0.25 () {56 vars in 1126 bytes} [Sun Aug 7 10:42:37 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 10:42:37.344753 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-239f09fa-3e54-4c81-a674-e3bfc6f39d5b None admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 10:42:37.353907 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6349a26-60b2-4c4c-aaab-bf0565231ef5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 10:42:37.354812 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6349a26-60b2-4c4c-aaab-bf0565231ef5 None admin] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:42:37.355365 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6349a26-60b2-4c4c-aaab-bf0565231ef5 None admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:42:37.355905 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6349a26-60b2-4c4c-aaab-bf0565231ef5 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:42:37.367656 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6349a26-60b2-4c4c-aaab-bf0565231ef5 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 07 10:42:37.373555 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application [None req-c6349a26-60b2-4c4c-aaab-bf0565231ef5 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 07 10:42:37.373555 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 07 10:42:37.373555 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 07 10:42:37.373555 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 07 10:42:37.373555 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 07 10:42:37.373555 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 07 10:42:37.373555 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 07 10:42:37.373555 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 07 10:42:37.373555 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 07 10:42:37.373555 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 07 10:42:37.373555 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 07 10:42:37.373555 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 07 10:42:37.373555 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 07 10:42:37.373555 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 07 10:42:37.373555 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 07 10:42:37.373555 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 07 10:42:37.374461 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 07 10:42:37.374461 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 07 10:42:37.374461 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 07 10:42:37.374461 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 07 10:42:37.374461 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 07 10:42:37.374461 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 07 10:42:37.374461 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 07 10:42:37.374461 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 07 10:42:37.374461 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 07 10:42:37.374461 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self.get_or_create( Aug 07 10:42:37.374461 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 07 10:42:37.374461 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application with Lock( Aug 07 10:42:37.374461 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 07 10:42:37.374461 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self._enter() Aug 07 10:42:37.374461 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 07 10:42:37.374461 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 07 10:42:37.375640 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 07 10:42:37.375640 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self.creator() Aug 07 10:42:37.375640 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 07 10:42:37.375640 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application created_value = creator( Aug 07 10:42:37.375640 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 07 10:42:37.375640 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 07 10:42:37.375640 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 07 10:42:37.375640 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 07 10:42:37.375640 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 07 10:42:37.375640 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 07 10:42:37.375640 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 07 10:42:37.375640 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application Aug 07 10:42:37.382970 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 458/895] 10.222.0.25 () {56 vars in 1120 bytes} [Sun Aug 7 10:42:37 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 07 10:42:37.384726 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6349a26-60b2-4c4c-aaab-bf0565231ef5 None admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 10:42:37.413750 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=100 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 10:42:37.470733 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-239f09fa-3e54-4c81-a674-e3bfc6f39d5b None admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 10:42:37.507299 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7895cb3-36b4-4676-b835-fe355e49088a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 10:42:37.508172 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7895cb3-36b4-4676-b835-fe355e49088a None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:42:37.508594 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7895cb3-36b4-4676-b835-fe355e49088a None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:42:37.509432 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7895cb3-36b4-4676-b835-fe355e49088a None admin] PATH_INFO: `/v3/services` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:42:37.510216 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7895cb3-36b4-4676-b835-fe355e49088a None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=80143) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 10:42:37.510676 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7895cb3-36b4-4676-b835-fe355e49088a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 10:42:37.512209 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7895cb3-36b4-4676-b835-fe355e49088a None admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 10:42:37.518071 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 438/896] 10.222.0.25 () {56 vars in 1126 bytes} [Sun Aug 7 10:42:37 2022] GET /identity/v3/services?type=compute_legacy => generated 386 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 10:42:37.524379 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7895cb3-36b4-4676-b835-fe355e49088a None admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 10:42:37.533883 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6349a26-60b2-4c4c-aaab-bf0565231ef5 None admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 10:42:37.570628 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4184929-f128-40fa-a463-14e96a81fe91 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 10:42:37.571539 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4184929-f128-40fa-a463-14e96a81fe91 None admin] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:42:37.571958 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4184929-f128-40fa-a463-14e96a81fe91 None admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:42:37.572369 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4184929-f128-40fa-a463-14e96a81fe91 None admin] PATH_INFO: `/v3/roles` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:42:37.573167 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4184929-f128-40fa-a463-14e96a81fe91 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80144) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 10:42:37.573633 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4184929-f128-40fa-a463-14e96a81fe91 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 10:42:37.575145 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4184929-f128-40fa-a463-14e96a81fe91 None admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 10:42:37.587604 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 459/897] 10.222.0.25 () {56 vars in 1115 bytes} [Sun Aug 7 10:42:37 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 10:42:37.589078 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-358ed540-791a-4998-bd2c-3ca3db0347b5 None None] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:42:37.589506 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-358ed540-791a-4998-bd2c-3ca3db0347b5 None None] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:42:37.589968 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-358ed540-791a-4998-bd2c-3ca3db0347b5 None None] PATH_INFO: `/` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:42:37.591346 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 460/898] 10.222.0.25 () {54 vars in 927 bytes} [Sun Aug 7 10:42:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 07 10:42:37.593297 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-358ed540-791a-4998-bd2c-3ca3db0347b5 None None] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 10:42:37.596405 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=100 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 10:42:37.647702 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7895cb3-36b4-4676-b835-fe355e49088a None admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 10:42:37.667348 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=100 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 10:42:37.687670 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86f661f6-ed36-4570-bc32-ca6805140457 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 10:42:37.688289 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86f661f6-ed36-4570-bc32-ca6805140457 None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:42:37.688571 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86f661f6-ed36-4570-bc32-ca6805140457 None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:42:37.688844 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86f661f6-ed36-4570-bc32-ca6805140457 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:42:37.689553 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86f661f6-ed36-4570-bc32-ca6805140457 None admin] RBAC: Adding query filter params (interface=public, service_id=5102c62b7e154aa187a68c85af91b0e6, region_id=RegionOne) {{(pid=80143) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 10:42:37.689864 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86f661f6-ed36-4570-bc32-ca6805140457 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 10:42:37.691261 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86f661f6-ed36-4570-bc32-ca6805140457 None admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 10:42:37.697040 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 439/899] 10.222.0.25 () {56 vars in 1251 bytes} [Sun Aug 7 10:42:37 2022] GET /identity/v3/endpoints?service_id=5102c62b7e154aa187a68c85af91b0e6&interface=public®ion_id=RegionOne => generated 194 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 10:42:37.698197 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2593f526-bf4a-4664-8058-26e2c90bf6a5 None None] REQUEST_METHOD: `POST` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:42:37.698477 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2593f526-bf4a-4664-8058-26e2c90bf6a5 None None] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:42:37.698749 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2593f526-bf4a-4664-8058-26e2c90bf6a5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:42:37.708208 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-358ed540-791a-4998-bd2c-3ca3db0347b5 None None] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 10:42:37.717254 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.auth.core [None req-2593f526-bf4a-4664-8058-26e2c90bf6a5 None None] MFA Rules not processed for user `bac4e6b76fe64217ac8fa44e71cffe6a`. Rule list: `[]` (Enabled: `True`). {{(pid=80143) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 10:42:37.718446 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-2593f526-bf4a-4664-8058-26e2c90bf6a5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 10:42:37.746530 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7737ed66-8d63-409f-beeb-2c176ab37a44 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 10:42:37.747449 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7737ed66-8d63-409f-beeb-2c176ab37a44 None admin] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:42:37.747728 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7737ed66-8d63-409f-beeb-2c176ab37a44 None admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:42:37.747996 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7737ed66-8d63-409f-beeb-2c176ab37a44 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:42:37.758371 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7737ed66-8d63-409f-beeb-2c176ab37a44 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 10:42:37.760544 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7737ed66-8d63-409f-beeb-2c176ab37a44 None admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 10:42:37.775084 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application [None req-7737ed66-8d63-409f-beeb-2c176ab37a44 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 07 10:42:37.775084 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 07 10:42:37.775084 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 07 10:42:37.775084 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 07 10:42:37.775084 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 07 10:42:37.775084 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 07 10:42:37.775084 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 07 10:42:37.775084 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 07 10:42:37.775084 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 07 10:42:37.775084 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 07 10:42:37.775084 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 07 10:42:37.775084 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 07 10:42:37.775084 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 07 10:42:37.775084 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 07 10:42:37.775084 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 07 10:42:37.775084 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 07 10:42:37.776029 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 07 10:42:37.776029 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 07 10:42:37.776029 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 07 10:42:37.776029 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 07 10:42:37.776029 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 07 10:42:37.776029 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self.get_or_create( Aug 07 10:42:37.776029 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 07 10:42:37.776029 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application with Lock( Aug 07 10:42:37.776029 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 07 10:42:37.776029 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self._enter() Aug 07 10:42:37.776029 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 07 10:42:37.776029 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 07 10:42:37.776029 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 07 10:42:37.776029 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self.creator() Aug 07 10:42:37.776029 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 07 10:42:37.776029 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application created_value = creator( Aug 07 10:42:37.776029 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 07 10:42:37.777016 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 07 10:42:37.777016 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 07 10:42:37.777016 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application Aug 07 10:42:37.780886 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 461/900] 10.222.0.25 () {56 vars in 1108 bytes} [Sun Aug 7 10:42:37 2022] GET /identity/v3/domains/Default => generated 87 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 07 10:42:37.781737 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7737ed66-8d63-409f-beeb-2c176ab37a44 None admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 10:42:37.792579 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 440/901] 10.222.0.25 () {58 vars in 1024 bytes} [Sun Aug 7 10:42:37 2022] POST /identity/v3/auth/tokens => generated 1372 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 07 10:42:37.795776 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2593f526-bf4a-4664-8058-26e2c90bf6a5 None None] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 10:42:37.850497 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=100 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 10:42:37.897330 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7737ed66-8d63-409f-beeb-2c176ab37a44 None admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 10:42:37.910644 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2593f526-bf4a-4664-8058-26e2c90bf6a5 None None] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 10:42:37.925801 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=100 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 10:42:37.932812 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-872b9400-e97b-4c6e-b48b-f5797741a9eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 10:42:37.933425 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-872b9400-e97b-4c6e-b48b-f5797741a9eb None admin] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:42:37.933707 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-872b9400-e97b-4c6e-b48b-f5797741a9eb None admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:42:37.933986 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-872b9400-e97b-4c6e-b48b-f5797741a9eb None admin] PATH_INFO: `/v3/services` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:42:37.934595 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-872b9400-e97b-4c6e-b48b-f5797741a9eb None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 10:42:37.935990 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-872b9400-e97b-4c6e-b48b-f5797741a9eb None admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 10:42:37.941660 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 462/902] 10.222.0.25 () {56 vars in 1087 bytes} [Sun Aug 7 10:42:37 2022] GET /identity/v3/services => generated 1268 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 07 10:42:37.942418 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-872b9400-e97b-4c6e-b48b-f5797741a9eb None admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 10:42:37.961070 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a89fbc1-1a6c-4269-b6e4-b582209e1e50 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 10:42:37.962223 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a89fbc1-1a6c-4269-b6e4-b582209e1e50 None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:42:37.962733 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a89fbc1-1a6c-4269-b6e4-b582209e1e50 None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:42:37.963226 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a89fbc1-1a6c-4269-b6e4-b582209e1e50 None admin] PATH_INFO: `/v3/domains` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:42:37.964223 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a89fbc1-1a6c-4269-b6e4-b582209e1e50 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80143) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 10:42:37.964704 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a89fbc1-1a6c-4269-b6e4-b582209e1e50 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 10:42:37.966555 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a89fbc1-1a6c-4269-b6e4-b582209e1e50 None admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 10:42:37.980977 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 441/903] 10.222.0.25 () {56 vars in 1109 bytes} [Sun Aug 7 10:42:37 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 10:42:37.986419 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a89fbc1-1a6c-4269-b6e4-b582209e1e50 None admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 10:42:38.069239 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-872b9400-e97b-4c6e-b48b-f5797741a9eb None admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 10:42:38.089623 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=100 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 10:42:38.108393 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a89fbc1-1a6c-4269-b6e4-b582209e1e50 None admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 10:42:38.109458 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2cfb9bc-d696-419e-a8f8-e83e55b64293 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 10:42:38.110083 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2cfb9bc-d696-419e-a8f8-e83e55b64293 None admin] REQUEST_METHOD: `POST` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:42:38.110361 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2cfb9bc-d696-419e-a8f8-e83e55b64293 None admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:42:38.110633 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2cfb9bc-d696-419e-a8f8-e83e55b64293 None admin] PATH_INFO: `/v3/services` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:42:38.111462 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2cfb9bc-d696-419e-a8f8-e83e55b64293 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 10:42:38.113625 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2cfb9bc-d696-419e-a8f8-e83e55b64293 None admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 10:42:38.130094 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 463/904] 10.222.0.25 () {60 vars in 1141 bytes} [Sun Aug 7 10:42:37 2022] POST /identity/v3/services => generated 241 bytes in 188 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 07 10:42:38.160880 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acdf00aa-1071-4d4a-aa0b-4b87ab7d7b0b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 10:42:38.161878 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acdf00aa-1071-4d4a-aa0b-4b87ab7d7b0b None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:42:38.162366 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acdf00aa-1071-4d4a-aa0b-4b87ab7d7b0b None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:42:38.162862 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acdf00aa-1071-4d4a-aa0b-4b87ab7d7b0b None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:42:38.175103 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acdf00aa-1071-4d4a-aa0b-4b87ab7d7b0b None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 10:42:38.176559 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acdf00aa-1071-4d4a-aa0b-4b87ab7d7b0b None admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 10:42:38.190375 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application [None req-acdf00aa-1071-4d4a-aa0b-4b87ab7d7b0b None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 07 10:42:38.190375 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 07 10:42:38.190375 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 07 10:42:38.190375 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 07 10:42:38.190375 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 07 10:42:38.190375 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 07 10:42:38.190375 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 07 10:42:38.190375 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 07 10:42:38.190375 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 07 10:42:38.190375 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 07 10:42:38.190375 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 07 10:42:38.190375 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 07 10:42:38.190375 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 07 10:42:38.190375 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 07 10:42:38.190375 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 07 10:42:38.190375 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 07 10:42:38.191755 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 07 10:42:38.191755 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 07 10:42:38.191755 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 07 10:42:38.191755 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 07 10:42:38.191755 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 07 10:42:38.191755 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 07 10:42:38.191755 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 07 10:42:38.191755 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 07 10:42:38.191755 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 07 10:42:38.191755 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.get_or_create( Aug 07 10:42:38.191755 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 07 10:42:38.191755 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application with Lock( Aug 07 10:42:38.191755 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 07 10:42:38.191755 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self._enter() Aug 07 10:42:38.191755 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 07 10:42:38.191755 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 07 10:42:38.191755 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 07 10:42:38.191755 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.creator() Aug 07 10:42:38.192751 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 07 10:42:38.192751 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application created_value = creator( Aug 07 10:42:38.192751 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 07 10:42:38.192751 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 07 10:42:38.192751 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 07 10:42:38.192751 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 07 10:42:38.192751 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 07 10:42:38.192751 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 07 10:42:38.192751 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 07 10:42:38.192751 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application Aug 07 10:42:38.192751 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 442/905] 10.222.0.25 () {56 vars in 1117 bytes} [Sun Aug 7 10:42:37 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 206 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 07 10:42:38.199920 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2cfb9bc-d696-419e-a8f8-e83e55b64293 None admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 10:42:38.325053 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2cfb9bc-d696-419e-a8f8-e83e55b64293 None admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 10:42:38.361153 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9208b031-3f1f-4710-a188-7350b7157af4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 10:42:38.361772 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9208b031-3f1f-4710-a188-7350b7157af4 None admin] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:42:38.362051 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9208b031-3f1f-4710-a188-7350b7157af4 None admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:42:38.362323 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9208b031-3f1f-4710-a188-7350b7157af4 None admin] PATH_INFO: `/v3/users` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:42:38.363355 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9208b031-3f1f-4710-a188-7350b7157af4 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80144) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 10:42:38.363661 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9208b031-3f1f-4710-a188-7350b7157af4 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 10:42:38.365078 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9208b031-3f1f-4710-a188-7350b7157af4 None admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 10:42:38.384251 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 464/906] 10.222.0.25 () {56 vars in 1149 bytes} [Sun Aug 7 10:42:38 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 10:42:38.391578 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acdf00aa-1071-4d4a-aa0b-4b87ab7d7b0b None admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 10:42:38.424372 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=100 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 10:42:38.502637 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acdf00aa-1071-4d4a-aa0b-4b87ab7d7b0b None admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 10:42:38.544356 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65786d01-1551-419a-aa72-daf5e42b4a55 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 10:42:38.545357 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65786d01-1551-419a-aa72-daf5e42b4a55 None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:42:38.545846 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65786d01-1551-419a-aa72-daf5e42b4a55 None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:42:38.546325 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65786d01-1551-419a-aa72-daf5e42b4a55 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:42:38.598955 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-987f49f0-814e-4516-b288-c51f62a2d3d4 None None] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:42:38.599305 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-987f49f0-814e-4516-b288-c51f62a2d3d4 None None] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:42:38.599535 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-987f49f0-814e-4516-b288-c51f62a2d3d4 None None] PATH_INFO: `/` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:42:38.601131 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 465/907] 10.222.0.25 () {54 vars in 927 bytes} [Sun Aug 7 10:42:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 07 10:42:38.606317 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fdddbe1-bd45-40df-aa91-ab729831cce8 None None] REQUEST_METHOD: `POST` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:42:38.606607 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fdddbe1-bd45-40df-aa91-ab729831cce8 None None] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:42:38.606888 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fdddbe1-bd45-40df-aa91-ab729831cce8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:42:38.622188 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65786d01-1551-419a-aa72-daf5e42b4a55 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 10:42:38.623962 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65786d01-1551-419a-aa72-daf5e42b4a55 None admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 10:42:38.628984 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.auth.core [None req-1fdddbe1-bd45-40df-aa91-ab729831cce8 None None] MFA Rules not processed for user `bac4e6b76fe64217ac8fa44e71cffe6a`. Rule list: `[]` (Enabled: `True`). {{(pid=80144) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 10:42:38.630299 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.fernet_utils [None req-1fdddbe1-bd45-40df-aa91-ab729831cce8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80144) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 10:42:38.660105 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=100 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 10:42:38.671946 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 10:42:38.724574 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 466/908] 10.222.0.25 () {58 vars in 1024 bytes} [Sun Aug 7 10:42:38 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 07 10:42:38.747999 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fdddbe1-bd45-40df-aa91-ab729831cce8 None None] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 10:42:38.752239 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 443/909] 10.222.0.25 () {58 vars in 1282 bytes} [Sun Aug 7 10:42:38 2022] GET /identity/v3/auth/tokens => generated 1474 bytes in 361 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 07 10:42:38.758013 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65786d01-1551-419a-aa72-daf5e42b4a55 None admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 10:42:38.854035 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fdddbe1-bd45-40df-aa91-ab729831cce8 None None] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 10:42:38.886170 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65786d01-1551-419a-aa72-daf5e42b4a55 None admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 10:42:38.892473 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6c21981-54ba-45b0-bed2-a055f8dfaeee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 10:42:38.893389 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6c21981-54ba-45b0-bed2-a055f8dfaeee None admin] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:42:38.893881 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6c21981-54ba-45b0-bed2-a055f8dfaeee None admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:42:38.894359 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6c21981-54ba-45b0-bed2-a055f8dfaeee None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:42:38.895200 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6c21981-54ba-45b0-bed2-a055f8dfaeee None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 10:42:38.896769 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6c21981-54ba-45b0-bed2-a055f8dfaeee None admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 10:42:38.906595 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application [None req-a6c21981-54ba-45b0-bed2-a055f8dfaeee None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Aug 07 10:42:38.906595 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 07 10:42:38.906595 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 07 10:42:38.906595 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 07 10:42:38.906595 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 07 10:42:38.906595 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 07 10:42:38.906595 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 07 10:42:38.906595 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 07 10:42:38.906595 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Aug 07 10:42:38.906595 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application Aug 07 10:42:38.906595 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 07 10:42:38.906595 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application Aug 07 10:42:38.906595 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 07 10:42:38.906595 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 07 10:42:38.906595 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 07 10:42:38.906595 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 07 10:42:38.906595 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 07 10:42:38.907787 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 07 10:42:38.907787 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 07 10:42:38.907787 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 07 10:42:38.907787 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 07 10:42:38.907787 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 07 10:42:38.907787 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 07 10:42:38.907787 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 07 10:42:38.907787 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 07 10:42:38.907787 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 07 10:42:38.907787 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 07 10:42:38.907787 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 07 10:42:38.907787 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 07 10:42:38.907787 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 07 10:42:38.907787 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 07 10:42:38.907787 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 07 10:42:38.907787 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self.get_or_create( Aug 07 10:42:38.908616 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 07 10:42:38.908616 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application with Lock( Aug 07 10:42:38.908616 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 07 10:42:38.908616 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self._enter() Aug 07 10:42:38.908616 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 07 10:42:38.908616 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 07 10:42:38.908616 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 07 10:42:38.908616 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self.creator() Aug 07 10:42:38.908616 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 07 10:42:38.908616 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application created_value = creator( Aug 07 10:42:38.908616 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 07 10:42:38.908616 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 07 10:42:38.908616 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Aug 07 10:42:38.908616 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application Aug 07 10:42:38.911289 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 467/910] 10.222.0.25 () {56 vars in 1114 bytes} [Sun Aug 7 10:42:38 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 07 10:42:38.917585 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6c21981-54ba-45b0-bed2-a055f8dfaeee None admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 10:42:38.921273 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=100 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 10:42:38.932664 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33446ff6-3bda-40d6-b34c-3b13e5e41284 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 10:42:38.933620 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33446ff6-3bda-40d6-b34c-3b13e5e41284 None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:42:38.933904 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33446ff6-3bda-40d6-b34c-3b13e5e41284 None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:42:38.934179 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33446ff6-3bda-40d6-b34c-3b13e5e41284 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:42:38.948869 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33446ff6-3bda-40d6-b34c-3b13e5e41284 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 10:42:38.951001 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33446ff6-3bda-40d6-b34c-3b13e5e41284 None admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 10:42:38.953213 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=100 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 10:42:38.969818 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application [None req-33446ff6-3bda-40d6-b34c-3b13e5e41284 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 07 10:42:38.969818 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 07 10:42:38.969818 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 07 10:42:38.969818 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 07 10:42:38.969818 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 07 10:42:38.969818 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 07 10:42:38.969818 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 07 10:42:38.969818 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 07 10:42:38.969818 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 07 10:42:38.969818 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 07 10:42:38.969818 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 07 10:42:38.969818 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 07 10:42:38.969818 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 07 10:42:38.969818 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 07 10:42:38.969818 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 07 10:42:38.969818 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 07 10:42:38.970808 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 07 10:42:38.970808 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 07 10:42:38.970808 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 07 10:42:38.970808 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 07 10:42:38.970808 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 07 10:42:38.970808 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.get_or_create( Aug 07 10:42:38.970808 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 07 10:42:38.970808 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application with Lock( Aug 07 10:42:38.970808 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 07 10:42:38.970808 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self._enter() Aug 07 10:42:38.970808 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 07 10:42:38.970808 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 07 10:42:38.970808 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 07 10:42:38.970808 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.creator() Aug 07 10:42:38.970808 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 07 10:42:38.970808 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application created_value = creator( Aug 07 10:42:38.970808 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 07 10:42:38.972140 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 07 10:42:38.972140 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 07 10:42:38.972140 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application Aug 07 10:42:38.978579 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 444/911] 10.222.0.25 () {56 vars in 1108 bytes} [Sun Aug 7 10:42:38 2022] GET /identity/v3/domains/Default => generated 87 bytes in 216 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 07 10:42:38.979686 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33446ff6-3bda-40d6-b34c-3b13e5e41284 None admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 10:42:39.053273 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6c21981-54ba-45b0-bed2-a055f8dfaeee None admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 10:42:39.093182 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33446ff6-3bda-40d6-b34c-3b13e5e41284 None admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 10:42:39.094110 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17df5e38-868c-4fa9-b867-713bc4a6c081 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 10:42:39.094999 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17df5e38-868c-4fa9-b867-713bc4a6c081 None admin] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:42:39.095485 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17df5e38-868c-4fa9-b867-713bc4a6c081 None admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:42:39.095982 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17df5e38-868c-4fa9-b867-713bc4a6c081 None admin] PATH_INFO: `/v3/services` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:42:39.096811 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17df5e38-868c-4fa9-b867-713bc4a6c081 None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=80144) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 10:42:39.097318 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17df5e38-868c-4fa9-b867-713bc4a6c081 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 10:42:39.098874 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17df5e38-868c-4fa9-b867-713bc4a6c081 None admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 10:42:39.109605 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 468/912] 10.222.0.25 () {56 vars in 1114 bytes} [Sun Aug 7 10:42:38 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 07 10:42:39.110782 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17df5e38-868c-4fa9-b867-713bc4a6c081 None admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 10:42:39.129869 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd6bd190-4660-460b-9f18-fd95598b4f2b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 10:42:39.130579 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd6bd190-4660-460b-9f18-fd95598b4f2b None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:42:39.130866 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd6bd190-4660-460b-9f18-fd95598b4f2b None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:42:39.131187 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd6bd190-4660-460b-9f18-fd95598b4f2b None admin] PATH_INFO: `/v3/domains` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:42:39.131838 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd6bd190-4660-460b-9f18-fd95598b4f2b None admin] RBAC: Adding query filter params (name=Default) {{(pid=80143) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 10:42:39.132442 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd6bd190-4660-460b-9f18-fd95598b4f2b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 10:42:39.133604 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd6bd190-4660-460b-9f18-fd95598b4f2b None admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 10:42:39.151908 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 445/913] 10.222.0.25 () {56 vars in 1109 bytes} [Sun Aug 7 10:42:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 10:42:39.153673 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd6bd190-4660-460b-9f18-fd95598b4f2b None admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 10:42:39.209215 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=100 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 10:42:39.246183 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=100 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 10:42:39.257841 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17df5e38-868c-4fa9-b867-713bc4a6c081 None admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 10:42:39.276420 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd6bd190-4660-460b-9f18-fd95598b4f2b None admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 10:42:39.297184 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f886264-2742-48e3-944f-68fde413a134 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 10:42:39.298126 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f886264-2742-48e3-944f-68fde413a134 None admin] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:42:39.298624 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f886264-2742-48e3-944f-68fde413a134 None admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:42:39.299078 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f886264-2742-48e3-944f-68fde413a134 None admin] PATH_INFO: `/v3/services` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:42:39.299873 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f886264-2742-48e3-944f-68fde413a134 None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=80144) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 10:42:39.300337 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f886264-2742-48e3-944f-68fde413a134 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 10:42:39.302071 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f886264-2742-48e3-944f-68fde413a134 None admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 10:42:39.307826 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 469/914] 10.222.0.25 () {56 vars in 1114 bytes} [Sun Aug 7 10:42:39 2022] GET /identity/v3/services?type=volumev3 => generated 362 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 10:42:39.313137 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f886264-2742-48e3-944f-68fde413a134 None admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 10:42:39.328098 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bcd6f22-33c2-4b58-9796-7556063617b4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 10:42:39.329552 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bcd6f22-33c2-4b58-9796-7556063617b4 None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:42:39.330112 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bcd6f22-33c2-4b58-9796-7556063617b4 None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:42:39.330588 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bcd6f22-33c2-4b58-9796-7556063617b4 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:42:39.340742 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bcd6f22-33c2-4b58-9796-7556063617b4 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 10:42:39.343093 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bcd6f22-33c2-4b58-9796-7556063617b4 None admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 10:42:39.358914 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application [None req-1bcd6f22-33c2-4b58-9796-7556063617b4 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 07 10:42:39.358914 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 07 10:42:39.358914 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 07 10:42:39.358914 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 07 10:42:39.358914 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 07 10:42:39.358914 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 07 10:42:39.358914 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 07 10:42:39.358914 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 07 10:42:39.358914 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 07 10:42:39.358914 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 07 10:42:39.358914 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 07 10:42:39.358914 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 07 10:42:39.358914 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 07 10:42:39.358914 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 07 10:42:39.358914 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 07 10:42:39.359852 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 07 10:42:39.359852 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 07 10:42:39.359852 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 07 10:42:39.359852 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 07 10:42:39.359852 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 07 10:42:39.359852 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 07 10:42:39.359852 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.get_or_create( Aug 07 10:42:39.359852 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 07 10:42:39.359852 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application with Lock( Aug 07 10:42:39.359852 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 07 10:42:39.359852 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self._enter() Aug 07 10:42:39.359852 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 07 10:42:39.359852 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 07 10:42:39.359852 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 07 10:42:39.359852 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.creator() Aug 07 10:42:39.359852 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 07 10:42:39.359852 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application created_value = creator( Aug 07 10:42:39.360707 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 07 10:42:39.360707 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 07 10:42:39.360707 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 07 10:42:39.360707 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 07 10:42:39.360707 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 07 10:42:39.360707 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 07 10:42:39.360707 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 07 10:42:39.360707 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application Aug 07 10:42:39.363610 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 446/915] 10.222.0.25 () {56 vars in 1111 bytes} [Sun Aug 7 10:42:39 2022] GET /identity/v3/projects/service => generated 88 bytes in 210 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 07 10:42:39.370288 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bcd6f22-33c2-4b58-9796-7556063617b4 None admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 10:42:39.452189 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f886264-2742-48e3-944f-68fde413a134 None admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 10:42:39.508535 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=100 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 10:42:39.509856 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cff90086-9263-4fcd-b927-dda648c29593 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 10:42:39.510397 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cff90086-9263-4fcd-b927-dda648c29593 None admin] REQUEST_METHOD: `POST` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:42:39.510776 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cff90086-9263-4fcd-b927-dda648c29593 None admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:42:39.511175 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cff90086-9263-4fcd-b927-dda648c29593 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:42:39.512010 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cff90086-9263-4fcd-b927-dda648c29593 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 10:42:39.513437 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cff90086-9263-4fcd-b927-dda648c29593 None admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 10:42:39.515174 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bcd6f22-33c2-4b58-9796-7556063617b4 None admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 10:42:39.539079 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 470/916] 10.222.0.25 () {60 vars in 1144 bytes} [Sun Aug 7 10:42:39 2022] POST /identity/v3/endpoints => generated 346 bytes in 226 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 07 10:42:39.563576 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dc92d33-3bbe-42ae-8579-e2ff0b255930 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 10:42:39.564499 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dc92d33-3bbe-42ae-8579-e2ff0b255930 None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:42:39.564920 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dc92d33-3bbe-42ae-8579-e2ff0b255930 None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:42:39.565437 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dc92d33-3bbe-42ae-8579-e2ff0b255930 None admin] PATH_INFO: `/v3/projects` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:42:39.566325 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dc92d33-3bbe-42ae-8579-e2ff0b255930 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80143) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 10:42:39.566777 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dc92d33-3bbe-42ae-8579-e2ff0b255930 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 10:42:39.568468 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dc92d33-3bbe-42ae-8579-e2ff0b255930 None admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 10:42:39.584834 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 447/917] 10.222.0.25 () {56 vars in 1148 bytes} [Sun Aug 7 10:42:39 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 10:42:39.591486 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cff90086-9263-4fcd-b927-dda648c29593 None admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 10:42:39.614304 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 10:42:39.715973 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cff90086-9263-4fcd-b927-dda648c29593 None admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 10:42:39.753486 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb74efb8-7f18-4cd9-9e7b-36e49aeee123 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 10:42:39.754345 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb74efb8-7f18-4cd9-9e7b-36e49aeee123 None admin] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:42:39.754769 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb74efb8-7f18-4cd9-9e7b-36e49aeee123 None admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:42:39.755207 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb74efb8-7f18-4cd9-9e7b-36e49aeee123 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:42:39.756162 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb74efb8-7f18-4cd9-9e7b-36e49aeee123 None admin] RBAC: Adding query filter params (role.id=3f1b8463c86840fb848a4cdf7c3d98be, scope.project.id=6de1f3cac02c4bcdb05dd2337176033d, user.id=de1d6c24b1a14d10a20d82a4c65793d8) {{(pid=80144) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 10:42:39.756613 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb74efb8-7f18-4cd9-9e7b-36e49aeee123 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 10:42:39.758163 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb74efb8-7f18-4cd9-9e7b-36e49aeee123 None admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 10:42:39.764708 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 471/918] 10.222.0.25 () {56 vars in 1374 bytes} [Sun Aug 7 10:42:39 2022] GET /identity/v3/role_assignments?user.id=de1d6c24b1a14d10a20d82a4c65793d8&scope.project.id=6de1f3cac02c4bcdb05dd2337176033d&role.id=3f1b8463c86840fb848a4cdf7c3d98be => generated 259 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 10:42:40.155324 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e01480be-3df5-4073-b72d-7505161ce280 None None] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:42:40.155519 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e01480be-3df5-4073-b72d-7505161ce280 None None] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:42:40.155787 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e01480be-3df5-4073-b72d-7505161ce280 None None] PATH_INFO: `/` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:42:40.157382 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 448/919] 10.222.0.25 () {54 vars in 927 bytes} [Sun Aug 7 10:42:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 07 10:42:40.162618 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c30a5303-f64b-47cd-8af7-855df85749df None None] REQUEST_METHOD: `POST` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:42:40.162908 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c30a5303-f64b-47cd-8af7-855df85749df None None] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:42:40.163209 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c30a5303-f64b-47cd-8af7-855df85749df None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:42:40.182082 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.auth.core [None req-c30a5303-f64b-47cd-8af7-855df85749df None None] MFA Rules not processed for user `bac4e6b76fe64217ac8fa44e71cffe6a`. Rule list: `[]` (Enabled: `True`). {{(pid=80144) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 10:42:40.183312 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.fernet_utils [None req-c30a5303-f64b-47cd-8af7-855df85749df None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80144) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 10:42:40.241510 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1baf65d8-3c2d-4612-9638-9c5658892607 None None] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:42:40.242130 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1baf65d8-3c2d-4612-9638-9c5658892607 None None] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:42:40.242677 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1baf65d8-3c2d-4612-9638-9c5658892607 None None] PATH_INFO: `/` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:42:40.244542 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 449/920] 10.222.0.25 () {54 vars in 927 bytes} [Sun Aug 7 10:42:40 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 07 10:42:40.249797 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73c57a3e-a0e2-4cde-b8dd-02487c0ed731 None None] REQUEST_METHOD: `POST` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:42:40.250312 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73c57a3e-a0e2-4cde-b8dd-02487c0ed731 None None] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:42:40.250794 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73c57a3e-a0e2-4cde-b8dd-02487c0ed731 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:42:40.254960 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 472/921] 10.222.0.25 () {58 vars in 1024 bytes} [Sun Aug 7 10:42:40 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 07 10:42:40.271161 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.auth.core [None req-73c57a3e-a0e2-4cde-b8dd-02487c0ed731 None None] MFA Rules not processed for user `bac4e6b76fe64217ac8fa44e71cffe6a`. Rule list: `[]` (Enabled: `True`). {{(pid=80143) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 10:42:40.272606 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-73c57a3e-a0e2-4cde-b8dd-02487c0ed731 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 10:42:40.277843 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c30a5303-f64b-47cd-8af7-855df85749df None None] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 10:42:40.288507 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=100 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 10:42:40.349534 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 450/922] 10.222.0.25 () {58 vars in 1024 bytes} [Sun Aug 7 10:42:40 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 07 10:42:40.372796 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73c57a3e-a0e2-4cde-b8dd-02487c0ed731 None None] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 10:42:40.395273 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c30a5303-f64b-47cd-8af7-855df85749df None None] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 10:42:40.435773 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27205338-caaa-4ec6-8282-8ea76f5f3991 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 10:42:40.436420 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27205338-caaa-4ec6-8282-8ea76f5f3991 None admin] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:42:40.436702 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27205338-caaa-4ec6-8282-8ea76f5f3991 None admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:42:40.436981 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27205338-caaa-4ec6-8282-8ea76f5f3991 None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:42:40.437557 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27205338-caaa-4ec6-8282-8ea76f5f3991 None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 10:42:40.439012 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27205338-caaa-4ec6-8282-8ea76f5f3991 None admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 10:42:40.448551 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application [None req-27205338-caaa-4ec6-8282-8ea76f5f3991 None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Aug 07 10:42:40.448551 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 07 10:42:40.448551 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 07 10:42:40.448551 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 07 10:42:40.448551 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 07 10:42:40.448551 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 07 10:42:40.448551 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 07 10:42:40.448551 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 07 10:42:40.448551 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Aug 07 10:42:40.448551 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application Aug 07 10:42:40.448551 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 07 10:42:40.448551 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application Aug 07 10:42:40.448551 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 07 10:42:40.448551 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 07 10:42:40.448551 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 07 10:42:40.448551 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 07 10:42:40.449621 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 07 10:42:40.449621 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 07 10:42:40.449621 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 07 10:42:40.449621 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 07 10:42:40.449621 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 07 10:42:40.449621 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 07 10:42:40.449621 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 07 10:42:40.449621 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 07 10:42:40.449621 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 07 10:42:40.449621 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 07 10:42:40.449621 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 07 10:42:40.449621 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 07 10:42:40.449621 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 07 10:42:40.449621 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 07 10:42:40.449621 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 07 10:42:40.449621 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 07 10:42:40.449621 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self.get_or_create( Aug 07 10:42:40.450532 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 07 10:42:40.450532 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application with Lock( Aug 07 10:42:40.450532 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 07 10:42:40.450532 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self._enter() Aug 07 10:42:40.450532 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 07 10:42:40.450532 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 07 10:42:40.450532 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 07 10:42:40.450532 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self.creator() Aug 07 10:42:40.450532 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 07 10:42:40.450532 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application created_value = creator( Aug 07 10:42:40.450532 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 07 10:42:40.450532 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 07 10:42:40.450532 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Aug 07 10:42:40.450532 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application Aug 07 10:42:40.450532 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 473/923] 10.222.0.25 () {56 vars in 1132 bytes} [Sun Aug 7 10:42:40 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 07 10:42:40.455864 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27205338-caaa-4ec6-8282-8ea76f5f3991 None admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 10:42:40.495831 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73c57a3e-a0e2-4cde-b8dd-02487c0ed731 None None] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 10:42:40.531375 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=100 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 10:42:40.532222 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09fa2da2-3543-4616-a1d7-e1a5fcbc7619 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 10:42:40.533329 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09fa2da2-3543-4616-a1d7-e1a5fcbc7619 None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:42:40.533609 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09fa2da2-3543-4616-a1d7-e1a5fcbc7619 None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:42:40.533885 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09fa2da2-3543-4616-a1d7-e1a5fcbc7619 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:42:40.534736 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09fa2da2-3543-4616-a1d7-e1a5fcbc7619 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 10:42:40.536440 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09fa2da2-3543-4616-a1d7-e1a5fcbc7619 None admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 10:42:40.547157 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=100 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 10:42:40.549071 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application [None req-09fa2da2-3543-4616-a1d7-e1a5fcbc7619 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 07 10:42:40.549071 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 07 10:42:40.549071 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 07 10:42:40.549071 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 07 10:42:40.549071 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 07 10:42:40.549071 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 07 10:42:40.549071 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 07 10:42:40.549071 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 07 10:42:40.549071 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 07 10:42:40.549071 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application Aug 07 10:42:40.549071 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 07 10:42:40.549071 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application Aug 07 10:42:40.549071 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 07 10:42:40.549071 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 07 10:42:40.549071 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 07 10:42:40.549071 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 07 10:42:40.549071 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 07 10:42:40.550271 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 07 10:42:40.550271 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 07 10:42:40.550271 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 07 10:42:40.550271 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 07 10:42:40.550271 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 07 10:42:40.550271 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 07 10:42:40.550271 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 07 10:42:40.550271 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 07 10:42:40.550271 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 07 10:42:40.550271 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 07 10:42:40.550271 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 07 10:42:40.550271 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 07 10:42:40.550271 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 07 10:42:40.550271 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 07 10:42:40.550271 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 07 10:42:40.550271 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.get_or_create( Aug 07 10:42:40.551741 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 07 10:42:40.551741 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application with Lock( Aug 07 10:42:40.551741 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 07 10:42:40.551741 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self._enter() Aug 07 10:42:40.551741 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 07 10:42:40.551741 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 07 10:42:40.551741 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 07 10:42:40.551741 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.creator() Aug 07 10:42:40.551741 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 07 10:42:40.551741 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application created_value = creator( Aug 07 10:42:40.551741 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 07 10:42:40.551741 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 07 10:42:40.551741 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 07 10:42:40.551741 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application Aug 07 10:42:40.551741 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 451/924] 10.222.0.25 () {56 vars in 1126 bytes} [Sun Aug 7 10:42:40 2022] GET /identity/v3/services/object-store => generated 93 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 07 10:42:40.560189 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09fa2da2-3543-4616-a1d7-e1a5fcbc7619 None admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 10:42:40.590398 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27205338-caaa-4ec6-8282-8ea76f5f3991 None admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 10:42:40.631870 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc6bc37a-4b45-4a72-aa4a-7bec7a63f883 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 10:42:40.632529 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc6bc37a-4b45-4a72-aa4a-7bec7a63f883 None admin] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:42:40.632811 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc6bc37a-4b45-4a72-aa4a-7bec7a63f883 None admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:42:40.633223 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc6bc37a-4b45-4a72-aa4a-7bec7a63f883 None admin] PATH_INFO: `/v3/services` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:42:40.633973 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc6bc37a-4b45-4a72-aa4a-7bec7a63f883 None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=80144) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 10:42:40.634279 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc6bc37a-4b45-4a72-aa4a-7bec7a63f883 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 10:42:40.635894 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc6bc37a-4b45-4a72-aa4a-7bec7a63f883 None admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 10:42:40.648541 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 474/925] 10.222.0.25 () {56 vars in 1126 bytes} [Sun Aug 7 10:42:40 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 07 10:42:40.650765 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc6bc37a-4b45-4a72-aa4a-7bec7a63f883 None admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 10:42:40.695029 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09fa2da2-3543-4616-a1d7-e1a5fcbc7619 None admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 10:42:40.726888 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51f3f3e6-f6b9-4215-b578-5c8b1ba44713 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 10:42:40.728220 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51f3f3e6-f6b9-4215-b578-5c8b1ba44713 None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:42:40.728543 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51f3f3e6-f6b9-4215-b578-5c8b1ba44713 None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:42:40.728816 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51f3f3e6-f6b9-4215-b578-5c8b1ba44713 None admin] PATH_INFO: `/v3/services` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:42:40.729456 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51f3f3e6-f6b9-4215-b578-5c8b1ba44713 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=80143) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 10:42:40.729755 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51f3f3e6-f6b9-4215-b578-5c8b1ba44713 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 10:42:40.731193 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51f3f3e6-f6b9-4215-b578-5c8b1ba44713 None admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 10:42:40.737362 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 452/926] 10.222.0.25 () {56 vars in 1122 bytes} [Sun Aug 7 10:42:40 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 10:42:40.743513 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51f3f3e6-f6b9-4215-b578-5c8b1ba44713 None admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 10:42:40.769224 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=100 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 10:42:40.773920 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc6bc37a-4b45-4a72-aa4a-7bec7a63f883 None admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 10:42:40.817405 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe4ac749-71ba-4143-8e8b-6aa861c52ff9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 10:42:40.818015 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe4ac749-71ba-4143-8e8b-6aa861c52ff9 None admin] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:42:40.818293 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe4ac749-71ba-4143-8e8b-6aa861c52ff9 None admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:42:40.818566 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe4ac749-71ba-4143-8e8b-6aa861c52ff9 None admin] PATH_INFO: `/v3/services` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:42:40.819213 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe4ac749-71ba-4143-8e8b-6aa861c52ff9 None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=80144) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 10:42:40.819518 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe4ac749-71ba-4143-8e8b-6aa861c52ff9 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 10:42:40.820941 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe4ac749-71ba-4143-8e8b-6aa861c52ff9 None admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 10:42:40.823821 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=100 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 10:42:40.831971 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 475/927] 10.222.0.25 () {56 vars in 1126 bytes} [Sun Aug 7 10:42:40 2022] GET /identity/v3/services?type=compute_legacy => generated 386 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 10:42:40.833199 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe4ac749-71ba-4143-8e8b-6aa861c52ff9 None admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 10:42:40.889212 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51f3f3e6-f6b9-4215-b578-5c8b1ba44713 None admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 10:42:40.933585 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b076da40-d11f-4abb-98d5-68464621832a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 10:42:40.934746 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b076da40-d11f-4abb-98d5-68464621832a None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:42:40.935208 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b076da40-d11f-4abb-98d5-68464621832a None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:42:40.935654 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b076da40-d11f-4abb-98d5-68464621832a None admin] PATH_INFO: `/v3/services` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:42:40.936606 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b076da40-d11f-4abb-98d5-68464621832a None admin] RBAC: Adding query filter params (type=object-store) {{(pid=80143) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 10:42:40.937506 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b076da40-d11f-4abb-98d5-68464621832a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 10:42:40.939489 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b076da40-d11f-4abb-98d5-68464621832a None admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 10:42:40.951452 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 453/928] 10.222.0.25 () {56 vars in 1122 bytes} [Sun Aug 7 10:42:40 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 10:42:40.952823 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b076da40-d11f-4abb-98d5-68464621832a None admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 10:42:40.963467 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe4ac749-71ba-4143-8e8b-6aa861c52ff9 None admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 10:42:41.006344 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9203227b-36d8-4115-8116-b9c26eee770e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 10:42:41.007288 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9203227b-36d8-4115-8116-b9c26eee770e None admin] REQUEST_METHOD: `POST` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:42:41.007800 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9203227b-36d8-4115-8116-b9c26eee770e None admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:42:41.012429 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9203227b-36d8-4115-8116-b9c26eee770e None admin] PATH_INFO: `/v3/endpoints` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:42:41.013004 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9203227b-36d8-4115-8116-b9c26eee770e None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 10:42:41.014706 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9203227b-36d8-4115-8116-b9c26eee770e None admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 10:42:41.048283 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 476/929] 10.222.0.25 () {60 vars in 1144 bytes} [Sun Aug 7 10:42:40 2022] POST /identity/v3/endpoints => generated 347 bytes in 215 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 07 10:42:41.051412 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=100 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 10:42:41.079374 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b076da40-d11f-4abb-98d5-68464621832a None admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 10:42:41.125922 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10d17897-7941-4e41-b13a-3fb9ed5aee23 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 10:42:41.126754 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10d17897-7941-4e41-b13a-3fb9ed5aee23 None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:42:41.127126 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10d17897-7941-4e41-b13a-3fb9ed5aee23 None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:42:41.127643 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10d17897-7941-4e41-b13a-3fb9ed5aee23 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:42:41.128978 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10d17897-7941-4e41-b13a-3fb9ed5aee23 None admin] RBAC: Adding query filter params (interface=public, service_id=8e77fc87597446b18b3955ba5f279642, region_id=RegionOne) {{(pid=80143) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 10:42:41.129614 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10d17897-7941-4e41-b13a-3fb9ed5aee23 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 10:42:41.131666 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10d17897-7941-4e41-b13a-3fb9ed5aee23 None admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 10:42:41.138656 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 454/930] 10.222.0.25 () {56 vars in 1251 bytes} [Sun Aug 7 10:42:40 2022] GET /identity/v3/endpoints?service_id=8e77fc87597446b18b3955ba5f279642&interface=public®ion_id=RegionOne => generated 194 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 10:42:41.144511 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9203227b-36d8-4115-8116-b9c26eee770e None admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 10:42:41.209392 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 10:42:41.263248 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9203227b-36d8-4115-8116-b9c26eee770e None admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 10:42:41.299386 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12882c6f-b487-48d5-bf61-4ecb8eec3e85 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 10:42:41.300352 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12882c6f-b487-48d5-bf61-4ecb8eec3e85 None admin] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:42:41.300864 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12882c6f-b487-48d5-bf61-4ecb8eec3e85 None admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:42:41.301342 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12882c6f-b487-48d5-bf61-4ecb8eec3e85 None admin] PATH_INFO: `/v3/services` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:42:41.302170 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12882c6f-b487-48d5-bf61-4ecb8eec3e85 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 10:42:41.303759 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12882c6f-b487-48d5-bf61-4ecb8eec3e85 None admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 10:42:41.311071 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 477/931] 10.222.0.25 () {56 vars in 1087 bytes} [Sun Aug 7 10:42:41 2022] GET /identity/v3/services => generated 1497 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 07 10:42:41.654718 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d16f558-ec17-4a5f-944f-fb6a0dca2fdc None None] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:42:41.655285 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d16f558-ec17-4a5f-944f-fb6a0dca2fdc None None] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:42:41.655285 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d16f558-ec17-4a5f-944f-fb6a0dca2fdc None None] PATH_INFO: `/` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:42:41.656863 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 455/932] 10.222.0.25 () {54 vars in 927 bytes} [Sun Aug 7 10:42:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 07 10:42:41.662024 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f59f3b60-fa83-406b-b128-2ef2dda23e68 None None] REQUEST_METHOD: `POST` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:42:41.662483 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f59f3b60-fa83-406b-b128-2ef2dda23e68 None None] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:42:41.662767 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f59f3b60-fa83-406b-b128-2ef2dda23e68 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:42:41.682259 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.auth.core [None req-f59f3b60-fa83-406b-b128-2ef2dda23e68 None None] MFA Rules not processed for user `bac4e6b76fe64217ac8fa44e71cffe6a`. Rule list: `[]` (Enabled: `True`). {{(pid=80144) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 10:42:41.683641 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.fernet_utils [None req-f59f3b60-fa83-406b-b128-2ef2dda23e68 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80144) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 10:42:41.754607 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 478/933] 10.222.0.25 () {58 vars in 1024 bytes} [Sun Aug 7 10:42:41 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 07 10:42:41.778131 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d16f558-ec17-4a5f-944f-fb6a0dca2fdc None None] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 10:42:41.845297 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98d75f19-4a3a-4b96-b9e2-303ea3c53d8b None None] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:42:41.845580 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98d75f19-4a3a-4b96-b9e2-303ea3c53d8b None None] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:42:41.845858 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98d75f19-4a3a-4b96-b9e2-303ea3c53d8b None None] PATH_INFO: `/` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:42:41.848442 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 479/934] 10.222.0.25 () {54 vars in 927 bytes} [Sun Aug 7 10:42:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 07 10:42:41.852398 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31c40cd0-02b2-417e-8899-d4a7db356fe8 None None] REQUEST_METHOD: `POST` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:42:41.852888 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31c40cd0-02b2-417e-8899-d4a7db356fe8 None None] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:42:41.853179 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31c40cd0-02b2-417e-8899-d4a7db356fe8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:42:41.872044 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.auth.core [None req-31c40cd0-02b2-417e-8899-d4a7db356fe8 None None] MFA Rules not processed for user `bac4e6b76fe64217ac8fa44e71cffe6a`. Rule list: `[]` (Enabled: `True`). {{(pid=80144) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 10:42:41.873307 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.fernet_utils [None req-31c40cd0-02b2-417e-8899-d4a7db356fe8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80144) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 10:42:41.888039 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d16f558-ec17-4a5f-944f-fb6a0dca2fdc None None] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 10:42:41.928240 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b648cc1-8fbc-487f-a677-af3b2b24a3ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 10:42:41.928907 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b648cc1-8fbc-487f-a677-af3b2b24a3ed None admin] REQUEST_METHOD: `POST` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:42:41.929363 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b648cc1-8fbc-487f-a677-af3b2b24a3ed None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:42:41.929735 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b648cc1-8fbc-487f-a677-af3b2b24a3ed None admin] PATH_INFO: `/v3/projects` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:42:41.930467 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b648cc1-8fbc-487f-a677-af3b2b24a3ed None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 10:42:41.932291 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b648cc1-8fbc-487f-a677-af3b2b24a3ed None admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 10:42:41.933598 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=100 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 10:42:41.936569 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: WARNING keystone.server.flask.common [None req-0b648cc1-8fbc-487f-a677-af3b2b24a3ed None admin] Deprecated: Not specifying a domain during a create user, group or project call, and relying on falling back to the default domain, is deprecated as of Liberty. There is no plan to remove this compatibility, however, future API versions may remove this, so please specify the domain explicitly or use a domain-scoped token. Aug 07 10:42:41.953397 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=100 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 10:42:41.967075 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 480/935] 10.222.0.25 () {58 vars in 1024 bytes} [Sun Aug 7 10:42:41 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 07 10:42:41.973275 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.sql.core [None req-0b648cc1-8fbc-487f-a677-af3b2b24a3ed None admin] Conflict project: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Aug 07 10:42:41.973275 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] Aug 07 10:42:41.973275 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: [parameters: {'id': '9e3b9a476aa04c128d97c038fbef27b7', 'name': 'service', 'domain_id': 'default', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': 'default', 'is_domain': 0}] Aug 07 10:42:41.973275 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=80143) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Aug 07 10:42:41.982062 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application [None req-0b648cc1-8fbc-487f-a677-af3b2b24a3ed None admin] Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id 9e3b9a476aa04c128d97c038fbef27b7.: keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id 9e3b9a476aa04c128d97c038fbef27b7. Aug 07 10:42:41.982062 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 07 10:42:41.982062 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Aug 07 10:42:41.982062 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application self.dialect.do_execute( Aug 07 10:42:41.982062 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Aug 07 10:42:41.982062 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Aug 07 10:42:41.982062 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Aug 07 10:42:41.982062 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application result = self._query(query) Aug 07 10:42:41.982062 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Aug 07 10:42:41.982062 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application conn.query(q) Aug 07 10:42:41.982062 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Aug 07 10:42:41.982062 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Aug 07 10:42:41.982062 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Aug 07 10:42:41.982907 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application result.read() Aug 07 10:42:41.982907 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Aug 07 10:42:41.982907 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Aug 07 10:42:41.982907 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Aug 07 10:42:41.982907 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application packet.raise_for_error() Aug 07 10:42:41.982907 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Aug 07 10:42:41.982907 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Aug 07 10:42:41.982907 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Aug 07 10:42:41.982907 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Aug 07 10:42:41.982907 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Aug 07 10:42:41.982907 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application Aug 07 10:42:41.982907 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Aug 07 10:42:41.982907 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application Aug 07 10:42:41.982907 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 07 10:42:41.982907 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Aug 07 10:42:41.982907 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return method(*args, **kwargs) Aug 07 10:42:41.982907 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 241, in create_project Aug 07 10:42:41.982907 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return project_ref.to_dict() Aug 07 10:42:41.984151 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Aug 07 10:42:41.984151 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application next(self.gen) Aug 07 10:42:41.984151 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Aug 07 10:42:41.984151 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application yield resource Aug 07 10:42:41.984151 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Aug 07 10:42:41.984151 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application next(self.gen) Aug 07 10:42:41.984151 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Aug 07 10:42:41.984151 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application self.session.rollback() Aug 07 10:42:41.984151 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Aug 07 10:42:41.984151 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application self.force_reraise() Aug 07 10:42:41.984151 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Aug 07 10:42:41.984151 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application raise self.value Aug 07 10:42:41.984151 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Aug 07 10:42:41.984151 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Aug 07 10:42:41.984151 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Aug 07 10:42:41.984151 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application session.commit() Aug 07 10:42:41.984151 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Aug 07 10:42:41.985051 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Aug 07 10:42:41.985051 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Aug 07 10:42:41.985051 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application self._prepare_impl() Aug 07 10:42:41.985051 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Aug 07 10:42:41.985051 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application self.session.flush() Aug 07 10:42:41.985051 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Aug 07 10:42:41.985051 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application self._flush(objects) Aug 07 10:42:41.985051 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Aug 07 10:42:41.985051 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Aug 07 10:42:41.985051 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Aug 07 10:42:41.985051 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application compat.raise_( Aug 07 10:42:41.985051 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Aug 07 10:42:41.985051 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application raise exception Aug 07 10:42:41.985051 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Aug 07 10:42:41.985051 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application flush_context.execute() Aug 07 10:42:41.985051 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Aug 07 10:42:41.985051 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application rec.execute(self) Aug 07 10:42:41.986773 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Aug 07 10:42:41.986773 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Aug 07 10:42:41.986773 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Aug 07 10:42:41.986773 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application _emit_insert_statements( Aug 07 10:42:41.986773 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1097, in _emit_insert_statements Aug 07 10:42:41.986773 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application c = connection._execute_20( Aug 07 10:42:41.986773 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1631, in _execute_20 Aug 07 10:42:41.986773 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Aug 07 10:42:41.986773 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 332, in _execute_on_connection Aug 07 10:42:41.986773 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Aug 07 10:42:41.986773 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1498, in _execute_clauseelement Aug 07 10:42:41.986773 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application ret = self._execute_context( Aug 07 10:42:41.986773 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1862, in _execute_context Aug 07 10:42:41.986773 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Aug 07 10:42:41.986773 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2041, in _handle_dbapi_exception Aug 07 10:42:41.989045 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Aug 07 10:42:41.989045 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Aug 07 10:42:41.989045 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application raise exception Aug 07 10:42:41.989045 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Aug 07 10:42:41.989045 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application self.dialect.do_execute( Aug 07 10:42:41.989045 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Aug 07 10:42:41.989045 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Aug 07 10:42:41.989045 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Aug 07 10:42:41.989045 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application result = self._query(query) Aug 07 10:42:41.989045 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Aug 07 10:42:41.989045 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application conn.query(q) Aug 07 10:42:41.989045 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Aug 07 10:42:41.989045 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Aug 07 10:42:41.989045 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Aug 07 10:42:41.989045 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application result.read() Aug 07 10:42:41.989045 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Aug 07 10:42:41.989045 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Aug 07 10:42:41.991392 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Aug 07 10:42:41.991392 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application packet.raise_for_error() Aug 07 10:42:41.991392 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Aug 07 10:42:41.991392 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Aug 07 10:42:41.991392 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Aug 07 10:42:41.991392 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Aug 07 10:42:41.991392 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Aug 07 10:42:41.991392 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] Aug 07 10:42:41.991392 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application [parameters: {'id': '9e3b9a476aa04c128d97c038fbef27b7', 'name': 'service', 'domain_id': 'default', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': 'default', 'is_domain': 0}] Aug 07 10:42:41.991392 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Aug 07 10:42:41.991392 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application Aug 07 10:42:41.991392 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 07 10:42:41.991392 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application Aug 07 10:42:41.991392 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 07 10:42:41.991392 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 216, in create_project Aug 07 10:42:41.993463 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application ret = self.driver.create_project(project_id, project) Aug 07 10:42:41.993463 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Aug 07 10:42:41.993463 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Aug 07 10:42:41.993463 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - Duplicate entry found with name service at domain ID default. Aug 07 10:42:41.993463 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application Aug 07 10:42:41.993463 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 07 10:42:41.993463 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application Aug 07 10:42:41.993463 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 07 10:42:41.993463 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 07 10:42:41.993463 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 07 10:42:41.993463 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 07 10:42:41.993463 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 07 10:42:41.993463 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 07 10:42:41.993463 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 07 10:42:41.993463 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 07 10:42:41.993463 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 07 10:42:41.993463 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 07 10:42:41.994730 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 07 10:42:41.994730 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 176, in post Aug 07 10:42:41.994730 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application ref = PROVIDERS.resource_api.create_project( Aug 07 10:42:41.994730 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 07 10:42:41.994730 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 07 10:42:41.994730 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 218, in create_project Aug 07 10:42:41.994730 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application raise exception.Conflict( Aug 07 10:42:41.994730 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id 9e3b9a476aa04c128d97c038fbef27b7. Aug 07 10:42:41.994730 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application Aug 07 10:42:41.994730 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 456/936] 10.222.0.25 () {60 vars in 1140 bytes} [Sun Aug 7 10:42:41 2022] POST /identity/v3/projects => generated 262 bytes in 211 msecs (HTTP/1.1 409) 5 headers in 183 bytes (1 switches on core 0) Aug 07 10:42:41.997118 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31c40cd0-02b2-417e-8899-d4a7db356fe8 None None] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 10:42:41.997221 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b648cc1-8fbc-487f-a677-af3b2b24a3ed None admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 10:42:42.115353 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b648cc1-8fbc-487f-a677-af3b2b24a3ed None admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 10:42:42.137665 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31c40cd0-02b2-417e-8899-d4a7db356fe8 None None] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 10:42:42.157467 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05bd0f7f-8c2f-472e-b7f1-75a21eb80e2d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 10:42:42.158110 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05bd0f7f-8c2f-472e-b7f1-75a21eb80e2d None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:42:42.158392 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05bd0f7f-8c2f-472e-b7f1-75a21eb80e2d None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:42:42.158820 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05bd0f7f-8c2f-472e-b7f1-75a21eb80e2d None admin] PATH_INFO: `/v3/projects/service` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:42:42.168800 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05bd0f7f-8c2f-472e-b7f1-75a21eb80e2d None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 10:42:42.171226 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05bd0f7f-8c2f-472e-b7f1-75a21eb80e2d None admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 10:42:42.179648 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ce88ff8-4815-40d8-a012-637cae6d0489 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 10:42:42.180346 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ce88ff8-4815-40d8-a012-637cae6d0489 None admin] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:42:42.180785 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ce88ff8-4815-40d8-a012-637cae6d0489 None admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:42:42.181130 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ce88ff8-4815-40d8-a012-637cae6d0489 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:42:42.185138 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application [None req-05bd0f7f-8c2f-472e-b7f1-75a21eb80e2d None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 07 10:42:42.185138 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 07 10:42:42.185138 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 07 10:42:42.185138 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 07 10:42:42.185138 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 07 10:42:42.185138 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 07 10:42:42.185138 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 07 10:42:42.185138 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 07 10:42:42.185138 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 07 10:42:42.185138 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 07 10:42:42.185138 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 07 10:42:42.185138 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 07 10:42:42.185138 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 07 10:42:42.185138 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 07 10:42:42.185138 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 07 10:42:42.185981 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 07 10:42:42.185981 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 07 10:42:42.185981 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 07 10:42:42.185981 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 07 10:42:42.185981 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 07 10:42:42.185981 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 07 10:42:42.185981 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.get_or_create( Aug 07 10:42:42.185981 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 07 10:42:42.185981 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application with Lock( Aug 07 10:42:42.185981 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 07 10:42:42.185981 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self._enter() Aug 07 10:42:42.185981 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 07 10:42:42.185981 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 07 10:42:42.185981 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 07 10:42:42.185981 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.creator() Aug 07 10:42:42.185981 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 07 10:42:42.185981 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application created_value = creator( Aug 07 10:42:42.186856 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 07 10:42:42.186856 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 07 10:42:42.186856 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 07 10:42:42.186856 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 07 10:42:42.186856 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 07 10:42:42.186856 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 07 10:42:42.186856 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 07 10:42:42.186856 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application Aug 07 10:42:42.186856 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 457/937] 10.222.0.25 () {56 vars in 1111 bytes} [Sun Aug 7 10:42:41 2022] GET /identity/v3/projects/service => generated 88 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 07 10:42:42.192586 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05bd0f7f-8c2f-472e-b7f1-75a21eb80e2d None admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 10:42:42.198652 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ce88ff8-4815-40d8-a012-637cae6d0489 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 07 10:42:42.204290 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application [None req-6ce88ff8-4815-40d8-a012-637cae6d0489 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 07 10:42:42.204290 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 07 10:42:42.204290 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 07 10:42:42.204290 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 07 10:42:42.204290 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 07 10:42:42.204290 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 07 10:42:42.204290 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 07 10:42:42.204290 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 07 10:42:42.204290 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 07 10:42:42.204290 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 07 10:42:42.204290 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 07 10:42:42.204290 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 07 10:42:42.204290 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 07 10:42:42.204290 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 07 10:42:42.204290 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 07 10:42:42.204290 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 07 10:42:42.205398 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 07 10:42:42.205398 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 07 10:42:42.205398 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 07 10:42:42.205398 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 07 10:42:42.205398 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 07 10:42:42.205398 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 07 10:42:42.205398 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 07 10:42:42.205398 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 07 10:42:42.205398 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 07 10:42:42.205398 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self.get_or_create( Aug 07 10:42:42.205398 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 07 10:42:42.205398 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application with Lock( Aug 07 10:42:42.205398 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 07 10:42:42.205398 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self._enter() Aug 07 10:42:42.205398 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 07 10:42:42.205398 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 07 10:42:42.206349 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 07 10:42:42.206349 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self.creator() Aug 07 10:42:42.206349 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 07 10:42:42.206349 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application created_value = creator( Aug 07 10:42:42.206349 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 07 10:42:42.206349 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 07 10:42:42.206349 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 07 10:42:42.206349 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 07 10:42:42.206349 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 07 10:42:42.206349 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 07 10:42:42.206349 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 07 10:42:42.206349 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application Aug 07 10:42:42.211005 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 481/938] 10.222.0.25 () {56 vars in 1120 bytes} [Sun Aug 7 10:42:41 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 212 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 07 10:42:42.214605 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ce88ff8-4815-40d8-a012-637cae6d0489 None admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 10:42:42.264624 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=100 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 10:42:42.302467 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 10:42:42.329119 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ce88ff8-4815-40d8-a012-637cae6d0489 None admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 10:42:42.345827 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05bd0f7f-8c2f-472e-b7f1-75a21eb80e2d None admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 10:42:42.366281 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf84e99f-2a65-4680-8cae-64f5c799bd02 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 10:42:42.366894 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf84e99f-2a65-4680-8cae-64f5c799bd02 None admin] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:42:42.367206 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf84e99f-2a65-4680-8cae-64f5c799bd02 None admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:42:42.367484 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf84e99f-2a65-4680-8cae-64f5c799bd02 None admin] PATH_INFO: `/v3/roles` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:42:42.368438 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf84e99f-2a65-4680-8cae-64f5c799bd02 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80144) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 10:42:42.368885 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf84e99f-2a65-4680-8cae-64f5c799bd02 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 10:42:42.370299 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf84e99f-2a65-4680-8cae-64f5c799bd02 None admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 10:42:42.379528 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 482/939] 10.222.0.25 () {56 vars in 1115 bytes} [Sun Aug 7 10:42:42 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 07 10:42:42.388354 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf84e99f-2a65-4680-8cae-64f5c799bd02 None admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 10:42:42.388785 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6817b512-1a5b-4e95-84b4-c336578b50a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 10:42:42.390339 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6817b512-1a5b-4e95-84b4-c336578b50a6 None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:42:42.390709 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6817b512-1a5b-4e95-84b4-c336578b50a6 None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:42:42.391089 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6817b512-1a5b-4e95-84b4-c336578b50a6 None admin] PATH_INFO: `/v3/projects` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:42:42.391841 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6817b512-1a5b-4e95-84b4-c336578b50a6 None admin] RBAC: Adding query filter params (name=service) {{(pid=80143) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 10:42:42.392220 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6817b512-1a5b-4e95-84b4-c336578b50a6 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 10:42:42.393793 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6817b512-1a5b-4e95-84b4-c336578b50a6 None admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 10:42:42.421077 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 458/940] 10.222.0.25 () {56 vars in 1112 bytes} [Sun Aug 7 10:42:42 2022] GET /identity/v3/projects?name=service => generated 411 bytes in 229 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 10:42:42.509884 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf84e99f-2a65-4680-8cae-64f5c799bd02 None admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 10:42:42.511683 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=100 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 10:42:42.548208 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64d4dbcd-47d2-46d8-89a8-2b36c804b1f7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 10:42:42.548861 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64d4dbcd-47d2-46d8-89a8-2b36c804b1f7 None admin] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:42:42.549142 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64d4dbcd-47d2-46d8-89a8-2b36c804b1f7 None admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:42:42.549414 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64d4dbcd-47d2-46d8-89a8-2b36c804b1f7 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:42:42.561269 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64d4dbcd-47d2-46d8-89a8-2b36c804b1f7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 10:42:42.562732 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64d4dbcd-47d2-46d8-89a8-2b36c804b1f7 None admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 10:42:42.578582 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application [None req-64d4dbcd-47d2-46d8-89a8-2b36c804b1f7 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 07 10:42:42.578582 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 07 10:42:42.578582 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 07 10:42:42.578582 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 07 10:42:42.578582 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 07 10:42:42.578582 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 07 10:42:42.578582 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 07 10:42:42.578582 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 07 10:42:42.578582 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 07 10:42:42.578582 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 07 10:42:42.578582 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 07 10:42:42.578582 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 07 10:42:42.578582 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 07 10:42:42.578582 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 07 10:42:42.578582 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 07 10:42:42.578582 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 07 10:42:42.579811 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 07 10:42:42.579811 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 07 10:42:42.579811 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 07 10:42:42.579811 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 07 10:42:42.579811 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 07 10:42:42.579811 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self.get_or_create( Aug 07 10:42:42.579811 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 07 10:42:42.579811 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application with Lock( Aug 07 10:42:42.579811 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 07 10:42:42.579811 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self._enter() Aug 07 10:42:42.579811 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 07 10:42:42.579811 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 07 10:42:42.579811 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 07 10:42:42.579811 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self.creator() Aug 07 10:42:42.579811 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 07 10:42:42.579811 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application created_value = creator( Aug 07 10:42:42.579811 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 07 10:42:42.580880 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 07 10:42:42.580880 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 07 10:42:42.580880 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application Aug 07 10:42:42.583291 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 483/941] 10.222.0.25 () {56 vars in 1108 bytes} [Sun Aug 7 10:42:42 2022] GET /identity/v3/domains/Default => generated 87 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 07 10:42:42.585867 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6817b512-1a5b-4e95-84b4-c336578b50a6 None admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 10:42:42.695023 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6817b512-1a5b-4e95-84b4-c336578b50a6 None admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 10:42:42.709938 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=100 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 10:42:42.732969 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af7ab3a0-3b84-4faa-81dd-dcfa75c3bafe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 10:42:42.733588 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af7ab3a0-3b84-4faa-81dd-dcfa75c3bafe None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:42:42.733868 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af7ab3a0-3b84-4faa-81dd-dcfa75c3bafe None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:42:42.734141 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af7ab3a0-3b84-4faa-81dd-dcfa75c3bafe None admin] PATH_INFO: `/v3/domains` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:42:42.734783 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af7ab3a0-3b84-4faa-81dd-dcfa75c3bafe None admin] RBAC: Adding query filter params (name=Default) {{(pid=80143) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 10:42:42.735167 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af7ab3a0-3b84-4faa-81dd-dcfa75c3bafe None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 10:42:42.736633 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af7ab3a0-3b84-4faa-81dd-dcfa75c3bafe None admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 10:42:42.749264 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 459/942] 10.222.0.25 () {56 vars in 1109 bytes} [Sun Aug 7 10:42:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 10:42:42.754616 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64d4dbcd-47d2-46d8-89a8-2b36c804b1f7 None admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 10:42:42.861476 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64d4dbcd-47d2-46d8-89a8-2b36c804b1f7 None admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 10:42:42.897117 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f50a104a-8f30-475f-a244-2aed9d66b430 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 10:42:42.897755 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f50a104a-8f30-475f-a244-2aed9d66b430 None admin] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:42:42.898036 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f50a104a-8f30-475f-a244-2aed9d66b430 None admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:42:42.898308 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f50a104a-8f30-475f-a244-2aed9d66b430 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:42:42.906495 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f50a104a-8f30-475f-a244-2aed9d66b430 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 10:42:42.907964 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f50a104a-8f30-475f-a244-2aed9d66b430 None admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 10:42:42.919931 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application [None req-f50a104a-8f30-475f-a244-2aed9d66b430 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 07 10:42:42.919931 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 07 10:42:42.919931 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 07 10:42:42.919931 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 07 10:42:42.919931 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 07 10:42:42.919931 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 07 10:42:42.919931 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 07 10:42:42.919931 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 07 10:42:42.919931 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 07 10:42:42.919931 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 07 10:42:42.919931 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 07 10:42:42.919931 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 07 10:42:42.919931 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 07 10:42:42.919931 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 07 10:42:42.919931 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 07 10:42:42.919931 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 07 10:42:42.921087 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 07 10:42:42.921087 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 07 10:42:42.921087 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 07 10:42:42.921087 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 07 10:42:42.921087 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 07 10:42:42.921087 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 07 10:42:42.921087 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 07 10:42:42.921087 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 07 10:42:42.921087 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 07 10:42:42.921087 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self.get_or_create( Aug 07 10:42:42.921087 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 07 10:42:42.921087 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application with Lock( Aug 07 10:42:42.921087 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 07 10:42:42.921087 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self._enter() Aug 07 10:42:42.921087 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 07 10:42:42.921087 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 07 10:42:42.921087 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 07 10:42:42.921087 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self.creator() Aug 07 10:42:42.922218 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 07 10:42:42.922218 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application created_value = creator( Aug 07 10:42:42.922218 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 07 10:42:42.922218 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 07 10:42:42.922218 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 07 10:42:42.922218 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 07 10:42:42.922218 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 07 10:42:42.922218 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 07 10:42:42.922218 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 07 10:42:42.922218 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application Aug 07 10:42:42.922218 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 484/943] 10.222.0.25 () {56 vars in 1117 bytes} [Sun Aug 7 10:42:42 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 07 10:42:42.928046 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af7ab3a0-3b84-4faa-81dd-dcfa75c3bafe None admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 10:42:43.031671 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af7ab3a0-3b84-4faa-81dd-dcfa75c3bafe None admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 10:42:43.066562 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2db7c307-5660-4175-8cb3-c270e9261112 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 10:42:43.067257 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2db7c307-5660-4175-8cb3-c270e9261112 None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:42:43.067541 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2db7c307-5660-4175-8cb3-c270e9261112 None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:42:43.067813 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2db7c307-5660-4175-8cb3-c270e9261112 None admin] PATH_INFO: `/v3/users` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:42:43.068540 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2db7c307-5660-4175-8cb3-c270e9261112 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80143) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 10:42:43.068899 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2db7c307-5660-4175-8cb3-c270e9261112 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 10:42:43.070419 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2db7c307-5660-4175-8cb3-c270e9261112 None admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 10:42:43.092382 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 460/944] 10.222.0.25 () {56 vars in 1149 bytes} [Sun Aug 7 10:42:42 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 10:42:43.097853 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f50a104a-8f30-475f-a244-2aed9d66b430 None admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 10:42:43.115138 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=100 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 10:42:43.206142 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f50a104a-8f30-475f-a244-2aed9d66b430 None admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 10:42:43.242497 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f3f27b8-2c31-498f-8640-bd3a94e36ca8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 10:42:43.243231 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f3f27b8-2c31-498f-8640-bd3a94e36ca8 None admin] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:42:43.243511 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f3f27b8-2c31-498f-8640-bd3a94e36ca8 None admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:42:43.243783 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f3f27b8-2c31-498f-8640-bd3a94e36ca8 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:42:43.254840 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f3f27b8-2c31-498f-8640-bd3a94e36ca8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 10:42:43.256303 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f3f27b8-2c31-498f-8640-bd3a94e36ca8 None admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 10:42:43.264910 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74327ae5-1efd-4c3d-85c7-7ba38e77c387 None None] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:42:43.265205 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74327ae5-1efd-4c3d-85c7-7ba38e77c387 None None] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:42:43.265507 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74327ae5-1efd-4c3d-85c7-7ba38e77c387 None None] PATH_INFO: `/` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:42:43.266972 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 461/945] 10.222.0.25 () {54 vars in 927 bytes} [Sun Aug 7 10:42:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 07 10:42:43.273126 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd10d48b-e74a-463c-94a5-5fe1dea104c3 None None] REQUEST_METHOD: `POST` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:42:43.273432 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd10d48b-e74a-463c-94a5-5fe1dea104c3 None None] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:42:43.273711 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd10d48b-e74a-463c-94a5-5fe1dea104c3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:42:43.276990 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application [None req-7f3f27b8-2c31-498f-8640-bd3a94e36ca8 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 07 10:42:43.276990 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 07 10:42:43.276990 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 07 10:42:43.276990 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 07 10:42:43.276990 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 07 10:42:43.276990 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 07 10:42:43.276990 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 07 10:42:43.276990 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 07 10:42:43.276990 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 07 10:42:43.276990 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 07 10:42:43.276990 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 07 10:42:43.276990 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 07 10:42:43.276990 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 07 10:42:43.276990 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 07 10:42:43.276990 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 07 10:42:43.276990 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 07 10:42:43.277903 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 07 10:42:43.277903 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 07 10:42:43.277903 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 07 10:42:43.277903 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 07 10:42:43.277903 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 07 10:42:43.277903 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self.get_or_create( Aug 07 10:42:43.277903 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 07 10:42:43.277903 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application with Lock( Aug 07 10:42:43.277903 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 07 10:42:43.277903 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self._enter() Aug 07 10:42:43.277903 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 07 10:42:43.277903 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 07 10:42:43.277903 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 07 10:42:43.277903 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self.creator() Aug 07 10:42:43.277903 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 07 10:42:43.277903 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application created_value = creator( Aug 07 10:42:43.277903 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 07 10:42:43.278793 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 07 10:42:43.278793 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 07 10:42:43.278793 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application Aug 07 10:42:43.280276 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 485/946] 10.222.0.25 () {56 vars in 1108 bytes} [Sun Aug 7 10:42:43 2022] GET /identity/v3/domains/Default => generated 87 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 07 10:42:43.285861 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f3f27b8-2c31-498f-8640-bd3a94e36ca8 None admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 10:42:43.300486 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.auth.core [None req-dd10d48b-e74a-463c-94a5-5fe1dea104c3 None None] MFA Rules not processed for user `bac4e6b76fe64217ac8fa44e71cffe6a`. Rule list: `[]` (Enabled: `True`). {{(pid=80143) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 10:42:43.302349 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-dd10d48b-e74a-463c-94a5-5fe1dea104c3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 10:42:43.335882 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=100 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 10:42:43.371824 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=100 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 10:42:43.395068 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 462/947] 10.222.0.25 () {58 vars in 1024 bytes} [Sun Aug 7 10:42:43 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 07 10:42:43.396359 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ee7d435-4d93-4c77-9d8e-95ec83b642c5 None None] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:42:43.396641 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ee7d435-4d93-4c77-9d8e-95ec83b642c5 None None] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:42:43.396913 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ee7d435-4d93-4c77-9d8e-95ec83b642c5 None None] PATH_INFO: `/` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:42:43.398223 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 463/948] 10.222.0.25 () {54 vars in 927 bytes} [Sun Aug 7 10:42:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 07 10:42:43.403825 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1c1b40f-dbee-4ac8-9ca8-ed9f0bfb11b1 None None] REQUEST_METHOD: `POST` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:42:43.404102 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1c1b40f-dbee-4ac8-9ca8-ed9f0bfb11b1 None None] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:42:43.404372 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1c1b40f-dbee-4ac8-9ca8-ed9f0bfb11b1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:42:43.425794 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f3f27b8-2c31-498f-8640-bd3a94e36ca8 None admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 10:42:43.431325 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.auth.core [None req-a1c1b40f-dbee-4ac8-9ca8-ed9f0bfb11b1 None None] MFA Rules not processed for user `bac4e6b76fe64217ac8fa44e71cffe6a`. Rule list: `[]` (Enabled: `True`). {{(pid=80143) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 10:42:43.432660 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-a1c1b40f-dbee-4ac8-9ca8-ed9f0bfb11b1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 10:42:43.461565 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c325cf75-fa59-4636-8266-595bda765238 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 10:42:43.462232 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c325cf75-fa59-4636-8266-595bda765238 None admin] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:42:43.462565 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c325cf75-fa59-4636-8266-595bda765238 None admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:42:43.462787 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c325cf75-fa59-4636-8266-595bda765238 None admin] PATH_INFO: `/v3/domains` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:42:43.463477 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c325cf75-fa59-4636-8266-595bda765238 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80144) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 10:42:43.463794 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c325cf75-fa59-4636-8266-595bda765238 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 10:42:43.465170 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c325cf75-fa59-4636-8266-595bda765238 None admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 10:42:43.477398 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 486/949] 10.222.0.25 () {56 vars in 1109 bytes} [Sun Aug 7 10:42:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 07 10:42:43.478338 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c325cf75-fa59-4636-8266-595bda765238 None admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 10:42:43.509711 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 464/950] 10.222.0.25 () {58 vars in 1024 bytes} [Sun Aug 7 10:42:43 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 07 10:42:43.519242 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1c1b40f-dbee-4ac8-9ca8-ed9f0bfb11b1 None None] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 10:42:43.584395 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c325cf75-fa59-4636-8266-595bda765238 None admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 10:42:43.610876 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=100 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 10:42:43.621516 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9283e152-d69d-48aa-8002-5576ecef577c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 10:42:43.622208 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9283e152-d69d-48aa-8002-5576ecef577c None admin] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:42:43.622425 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9283e152-d69d-48aa-8002-5576ecef577c None admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:42:43.622550 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=100 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 10:42:43.622718 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9283e152-d69d-48aa-8002-5576ecef577c None admin] PATH_INFO: `/v3/services/compute` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:42:43.623312 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9283e152-d69d-48aa-8002-5576ecef577c None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 10:42:43.624710 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9283e152-d69d-48aa-8002-5576ecef577c None admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 10:42:43.632881 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1c1b40f-dbee-4ac8-9ca8-ed9f0bfb11b1 None None] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 10:42:43.634095 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application [None req-9283e152-d69d-48aa-8002-5576ecef577c None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Aug 07 10:42:43.634095 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 07 10:42:43.634095 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 07 10:42:43.634095 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 07 10:42:43.634095 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 07 10:42:43.634095 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 07 10:42:43.634095 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 07 10:42:43.634095 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 07 10:42:43.634095 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Aug 07 10:42:43.634095 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application Aug 07 10:42:43.634095 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 07 10:42:43.634095 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application Aug 07 10:42:43.634095 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 07 10:42:43.634095 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 07 10:42:43.634095 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 07 10:42:43.634095 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 07 10:42:43.634095 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 07 10:42:43.635101 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 07 10:42:43.635101 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 07 10:42:43.635101 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 07 10:42:43.635101 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 07 10:42:43.635101 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 07 10:42:43.635101 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 07 10:42:43.635101 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 07 10:42:43.635101 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 07 10:42:43.635101 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 07 10:42:43.635101 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 07 10:42:43.635101 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 07 10:42:43.635101 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 07 10:42:43.635101 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 07 10:42:43.635101 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 07 10:42:43.635101 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 07 10:42:43.635101 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self.get_or_create( Aug 07 10:42:43.635948 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 07 10:42:43.635948 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application with Lock( Aug 07 10:42:43.635948 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 07 10:42:43.635948 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self._enter() Aug 07 10:42:43.635948 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 07 10:42:43.635948 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 07 10:42:43.635948 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 07 10:42:43.635948 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self.creator() Aug 07 10:42:43.635948 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 07 10:42:43.635948 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application created_value = creator( Aug 07 10:42:43.635948 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 07 10:42:43.635948 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 07 10:42:43.635948 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Aug 07 10:42:43.635948 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application Aug 07 10:42:43.635948 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 487/951] 10.222.0.25 () {56 vars in 1111 bytes} [Sun Aug 7 10:42:43 2022] GET /identity/v3/services/compute => generated 88 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 07 10:42:43.636753 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9283e152-d69d-48aa-8002-5576ecef577c None admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 10:42:43.676559 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7897ff52-b101-4ba1-af33-a66ff1014f19 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 10:42:43.676853 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7897ff52-b101-4ba1-af33-a66ff1014f19 None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:42:43.677120 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7897ff52-b101-4ba1-af33-a66ff1014f19 None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:42:43.677390 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7897ff52-b101-4ba1-af33-a66ff1014f19 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:42:43.684445 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7897ff52-b101-4ba1-af33-a66ff1014f19 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 10:42:43.685898 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7897ff52-b101-4ba1-af33-a66ff1014f19 None admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 10:42:43.697815 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application [None req-7897ff52-b101-4ba1-af33-a66ff1014f19 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 07 10:42:43.697815 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 07 10:42:43.697815 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 07 10:42:43.697815 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 07 10:42:43.697815 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 07 10:42:43.697815 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 07 10:42:43.697815 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 07 10:42:43.697815 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 07 10:42:43.697815 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 07 10:42:43.697815 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 07 10:42:43.697815 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 07 10:42:43.697815 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 07 10:42:43.697815 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 07 10:42:43.697815 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 07 10:42:43.697815 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 07 10:42:43.699178 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 07 10:42:43.699178 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 07 10:42:43.699178 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 07 10:42:43.699178 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 07 10:42:43.699178 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 07 10:42:43.699178 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 07 10:42:43.699178 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.get_or_create( Aug 07 10:42:43.699178 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 07 10:42:43.699178 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application with Lock( Aug 07 10:42:43.699178 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 07 10:42:43.699178 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self._enter() Aug 07 10:42:43.699178 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 07 10:42:43.699178 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 07 10:42:43.699178 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 07 10:42:43.699178 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.creator() Aug 07 10:42:43.699178 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 07 10:42:43.699178 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application created_value = creator( Aug 07 10:42:43.700048 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 07 10:42:43.700048 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 07 10:42:43.700048 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 07 10:42:43.700048 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 07 10:42:43.700048 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 07 10:42:43.700048 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 07 10:42:43.700048 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 07 10:42:43.700048 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application Aug 07 10:42:43.700048 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 465/952] 10.222.0.25 () {56 vars in 1111 bytes} [Sun Aug 7 10:42:43 2022] GET /identity/v3/projects/service => generated 88 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 07 10:42:43.700048 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7897ff52-b101-4ba1-af33-a66ff1014f19 None admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 10:42:43.749058 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9283e152-d69d-48aa-8002-5576ecef577c None admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 10:42:43.783306 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bd3886e-d17a-495d-873c-ca15498587a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 10:42:43.783919 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bd3886e-d17a-495d-873c-ca15498587a1 None admin] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:42:43.784216 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bd3886e-d17a-495d-873c-ca15498587a1 None admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:42:43.784484 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bd3886e-d17a-495d-873c-ca15498587a1 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:42:43.785056 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bd3886e-d17a-495d-873c-ca15498587a1 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 10:42:43.786477 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bd3886e-d17a-495d-873c-ca15498587a1 None admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 10:42:43.796310 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application [None req-8bd3886e-d17a-495d-873c-ca15498587a1 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 07 10:42:43.796310 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 07 10:42:43.796310 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 07 10:42:43.796310 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 07 10:42:43.796310 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 07 10:42:43.796310 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 07 10:42:43.796310 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 07 10:42:43.796310 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 07 10:42:43.796310 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 07 10:42:43.796310 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application Aug 07 10:42:43.796310 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 07 10:42:43.796310 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application Aug 07 10:42:43.796310 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 07 10:42:43.796310 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 07 10:42:43.796310 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 07 10:42:43.796310 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 07 10:42:43.796310 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 07 10:42:43.797514 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 07 10:42:43.797514 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 07 10:42:43.797514 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 07 10:42:43.797514 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 07 10:42:43.797514 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 07 10:42:43.797514 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 07 10:42:43.797514 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 07 10:42:43.797514 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 07 10:42:43.797514 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 07 10:42:43.797514 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 07 10:42:43.797514 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 07 10:42:43.797514 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 07 10:42:43.797514 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 07 10:42:43.797514 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 07 10:42:43.797514 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 07 10:42:43.797514 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self.get_or_create( Aug 07 10:42:43.798383 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 07 10:42:43.798383 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application with Lock( Aug 07 10:42:43.798383 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 07 10:42:43.798383 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self._enter() Aug 07 10:42:43.798383 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 07 10:42:43.798383 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 07 10:42:43.798383 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 07 10:42:43.798383 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self.creator() Aug 07 10:42:43.798383 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 07 10:42:43.798383 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application created_value = creator( Aug 07 10:42:43.798383 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 07 10:42:43.798383 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 07 10:42:43.798383 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 07 10:42:43.798383 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application Aug 07 10:42:43.798383 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 488/953] 10.222.0.25 () {56 vars in 1126 bytes} [Sun Aug 7 10:42:43 2022] GET /identity/v3/services/object-store => generated 93 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 07 10:42:43.799761 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bd3886e-d17a-495d-873c-ca15498587a1 None admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 10:42:43.817477 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7897ff52-b101-4ba1-af33-a66ff1014f19 None admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 10:42:43.851812 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-141a60bd-55f7-4a1d-8fe4-c89741536808 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 10:42:43.852926 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-141a60bd-55f7-4a1d-8fe4-c89741536808 None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:42:43.853210 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-141a60bd-55f7-4a1d-8fe4-c89741536808 None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:42:43.853481 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-141a60bd-55f7-4a1d-8fe4-c89741536808 None admin] PATH_INFO: `/v3/services` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:42:43.854103 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-141a60bd-55f7-4a1d-8fe4-c89741536808 None admin] RBAC: Adding query filter params (name=compute) {{(pid=80143) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 10:42:43.854403 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-141a60bd-55f7-4a1d-8fe4-c89741536808 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 10:42:43.855781 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-141a60bd-55f7-4a1d-8fe4-c89741536808 None admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 10:42:43.862979 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 466/954] 10.222.0.25 () {56 vars in 1112 bytes} [Sun Aug 7 10:42:43 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 10:42:43.862979 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-141a60bd-55f7-4a1d-8fe4-c89741536808 None admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 10:42:43.865739 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=100 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 10:42:43.880837 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=100 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 10:42:43.925409 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bd3886e-d17a-495d-873c-ca15498587a1 None admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 10:42:43.964303 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-141a60bd-55f7-4a1d-8fe4-c89741536808 None admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 10:42:43.968033 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f84c63c2-6be3-4b70-89c5-45d8dd922ce5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 10:42:43.968649 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f84c63c2-6be3-4b70-89c5-45d8dd922ce5 None admin] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:42:43.968931 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f84c63c2-6be3-4b70-89c5-45d8dd922ce5 None admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:42:43.969206 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f84c63c2-6be3-4b70-89c5-45d8dd922ce5 None admin] PATH_INFO: `/v3/projects` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:42:43.969901 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f84c63c2-6be3-4b70-89c5-45d8dd922ce5 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80144) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 10:42:43.970211 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f84c63c2-6be3-4b70-89c5-45d8dd922ce5 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 10:42:43.971691 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f84c63c2-6be3-4b70-89c5-45d8dd922ce5 None admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 10:42:43.984692 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 489/955] 10.222.0.25 () {56 vars in 1148 bytes} [Sun Aug 7 10:42:43 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 10:42:43.985357 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f84c63c2-6be3-4b70-89c5-45d8dd922ce5 None admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 10:42:44.005329 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c90b39e3-e8a0-4065-b269-16b81120b316 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 10:42:44.005938 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c90b39e3-e8a0-4065-b269-16b81120b316 None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:42:44.006216 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c90b39e3-e8a0-4065-b269-16b81120b316 None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:42:44.006484 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c90b39e3-e8a0-4065-b269-16b81120b316 None admin] PATH_INFO: `/v3/services` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:42:44.007141 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c90b39e3-e8a0-4065-b269-16b81120b316 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=80143) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 10:42:44.007444 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c90b39e3-e8a0-4065-b269-16b81120b316 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 10:42:44.008868 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c90b39e3-e8a0-4065-b269-16b81120b316 None admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 10:42:44.013987 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 467/956] 10.222.0.25 () {56 vars in 1122 bytes} [Sun Aug 7 10:42:43 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 10:42:44.019107 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c90b39e3-e8a0-4065-b269-16b81120b316 None admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 10:42:44.089594 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f84c63c2-6be3-4b70-89c5-45d8dd922ce5 None admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 10:42:44.104341 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=100 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 10:42:44.107139 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=100 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 10:42:44.126929 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c68f2e4-552d-4ae5-8f69-c4c9b4bac572 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 10:42:44.127566 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c68f2e4-552d-4ae5-8f69-c4c9b4bac572 None admin] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:42:44.127845 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c68f2e4-552d-4ae5-8f69-c4c9b4bac572 None admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:42:44.128118 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c68f2e4-552d-4ae5-8f69-c4c9b4bac572 None admin] PATH_INFO: `/v3/services` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:42:44.128595 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c90b39e3-e8a0-4065-b269-16b81120b316 None admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 10:42:44.128777 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c68f2e4-552d-4ae5-8f69-c4c9b4bac572 None admin] RBAC: Adding query filter params (type=compute) {{(pid=80144) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 10:42:44.129040 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c68f2e4-552d-4ae5-8f69-c4c9b4bac572 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 10:42:44.130385 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c68f2e4-552d-4ae5-8f69-c4c9b4bac572 None admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 10:42:44.135130 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 490/957] 10.222.0.25 () {56 vars in 1112 bytes} [Sun Aug 7 10:42:43 2022] GET /identity/v3/services?type=compute => generated 124 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 10:42:44.140375 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c68f2e4-552d-4ae5-8f69-c4c9b4bac572 None admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 10:42:44.176229 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-734a53c1-8c95-46ba-8bbb-8b67fe1a8751 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 10:42:44.177561 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-734a53c1-8c95-46ba-8bbb-8b67fe1a8751 None admin] REQUEST_METHOD: `PUT` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:42:44.178308 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-734a53c1-8c95-46ba-8bbb-8b67fe1a8751 None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:42:44.179298 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-734a53c1-8c95-46ba-8bbb-8b67fe1a8751 None admin] PATH_INFO: `/v3/projects/6de1f3cac02c4bcdb05dd2337176033d/users/de1d6c24b1a14d10a20d82a4c65793d8/roles/3f1b8463c86840fb848a4cdf7c3d98be` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:42:44.183440 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-734a53c1-8c95-46ba-8bbb-8b67fe1a8751 None admin] RBAC: Authorizing `identity:create_grant(project_id=6de1f3cac02c4bcdb05dd2337176033d, user_id=de1d6c24b1a14d10a20d82a4c65793d8, role_id=3f1b8463c86840fb848a4cdf7c3d98be)` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 10:42:44.186347 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-734a53c1-8c95-46ba-8bbb-8b67fe1a8751 None admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 10:42:44.207996 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 468/958] 10.222.0.25 () {58 vars in 1439 bytes} [Sun Aug 7 10:42:44 2022] PUT /identity/v3/projects/6de1f3cac02c4bcdb05dd2337176033d/users/de1d6c24b1a14d10a20d82a4c65793d8/roles/3f1b8463c86840fb848a4cdf7c3d98be => generated 0 bytes in 189 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 07 10:42:44.270540 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c68f2e4-552d-4ae5-8f69-c4c9b4bac572 None admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 10:42:44.316304 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22c4ff36-c7cf-45b3-b4a1-48103428681a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 10:42:44.316951 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22c4ff36-c7cf-45b3-b4a1-48103428681a None admin] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:42:44.317242 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22c4ff36-c7cf-45b3-b4a1-48103428681a None admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:42:44.317515 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22c4ff36-c7cf-45b3-b4a1-48103428681a None admin] PATH_INFO: `/v3/services` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:42:44.318241 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22c4ff36-c7cf-45b3-b4a1-48103428681a None admin] RBAC: Adding query filter params (type=object-store) {{(pid=80144) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 10:42:44.318600 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22c4ff36-c7cf-45b3-b4a1-48103428681a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 10:42:44.322101 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22c4ff36-c7cf-45b3-b4a1-48103428681a None admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 10:42:44.327370 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 491/959] 10.222.0.25 () {56 vars in 1122 bytes} [Sun Aug 7 10:42:44 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 10:42:44.333033 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-734a53c1-8c95-46ba-8bbb-8b67fe1a8751 None admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 10:42:44.452382 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-734a53c1-8c95-46ba-8bbb-8b67fe1a8751 None admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 10:42:44.492330 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a597d7c-238d-43c1-869c-32a915496e58 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 10:42:44.493410 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a597d7c-238d-43c1-869c-32a915496e58 None admin] REQUEST_METHOD: `POST` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:42:44.493899 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a597d7c-238d-43c1-869c-32a915496e58 None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:42:44.494385 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a597d7c-238d-43c1-869c-32a915496e58 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:42:44.495290 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a597d7c-238d-43c1-869c-32a915496e58 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 10:42:44.496864 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a597d7c-238d-43c1-869c-32a915496e58 None admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 10:42:44.507125 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 10:42:44.524921 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 469/960] 10.222.0.25 () {60 vars in 1144 bytes} [Sun Aug 7 10:42:44 2022] POST /identity/v3/endpoints => generated 349 bytes in 192 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 07 10:42:44.574872 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e166aa0-1d29-49db-badd-948a5d9ac10e None None] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:42:44.575188 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e166aa0-1d29-49db-badd-948a5d9ac10e None None] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:42:44.575468 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e166aa0-1d29-49db-badd-948a5d9ac10e None None] PATH_INFO: `/` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:42:44.577162 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 492/961] 10.222.0.25 () {54 vars in 927 bytes} [Sun Aug 7 10:42:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 07 10:42:44.583630 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-581fc4e0-5688-47ee-b98c-2be83f25c8b0 None None] REQUEST_METHOD: `POST` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:42:44.584225 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-581fc4e0-5688-47ee-b98c-2be83f25c8b0 None None] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:42:44.584725 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-581fc4e0-5688-47ee-b98c-2be83f25c8b0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:42:44.606335 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.auth.core [None req-581fc4e0-5688-47ee-b98c-2be83f25c8b0 None None] MFA Rules not processed for user `bac4e6b76fe64217ac8fa44e71cffe6a`. Rule list: `[]` (Enabled: `True`). {{(pid=80143) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 10:42:44.607872 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-581fc4e0-5688-47ee-b98c-2be83f25c8b0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 10:42:44.695146 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 470/962] 10.222.0.25 () {58 vars in 1024 bytes} [Sun Aug 7 10:42:44 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 07 10:42:44.717008 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e166aa0-1d29-49db-badd-948a5d9ac10e None None] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 10:42:44.765874 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=100 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 10:42:44.832626 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e166aa0-1d29-49db-badd-948a5d9ac10e None None] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 10:42:44.871792 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4846400-75f4-4423-82cc-06e5cfe2c978 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 10:42:44.872697 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4846400-75f4-4423-82cc-06e5cfe2c978 None admin] REQUEST_METHOD: `POST` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:42:44.873135 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4846400-75f4-4423-82cc-06e5cfe2c978 None admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:42:44.873564 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4846400-75f4-4423-82cc-06e5cfe2c978 None admin] PATH_INFO: `/v3/users` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:42:44.874366 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4846400-75f4-4423-82cc-06e5cfe2c978 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 10:42:44.875968 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4846400-75f4-4423-82cc-06e5cfe2c978 None admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 10:42:44.878747 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: WARNING keystone.server.flask.common [None req-a4846400-75f4-4423-82cc-06e5cfe2c978 None admin] Deprecated: Not specifying a domain during a create user, group or project call, and relying on falling back to the default domain, is deprecated as of Liberty. There is no plan to remove this compatibility, however, future API versions may remove this, so please specify the domain explicitly or use a domain-scoped token. Aug 07 10:42:44.892024 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.sql.core [None req-a4846400-75f4-4423-82cc-06e5cfe2c978 None admin] Conflict user: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Aug 07 10:42:44.892024 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: [SQL: INSERT INTO local_user (user_id, domain_id, name, failed_auth_count, failed_auth_at) VALUES (%(user_id)s, %(domain_id)s, %(name)s, %(failed_auth_count)s, %(failed_auth_at)s)] Aug 07 10:42:44.892024 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: [parameters: {'user_id': 'ce48e9fa1e3d45ec965fcf5072dcdcce', 'domain_id': 'default', 'name': 'cinder', 'failed_auth_count': 0, 'failed_auth_at': None}] Aug 07 10:42:44.892024 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=80144) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Aug 07 10:42:44.909483 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application [None req-a4846400-75f4-4423-82cc-06e5cfe2c978 None admin] Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default.: keystone.exception.Conflict: Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default. Aug 07 10:42:44.909483 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 07 10:42:44.909483 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Aug 07 10:42:44.909483 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application self.dialect.do_execute( Aug 07 10:42:44.909483 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Aug 07 10:42:44.909483 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Aug 07 10:42:44.909483 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Aug 07 10:42:44.909483 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application result = self._query(query) Aug 07 10:42:44.909483 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Aug 07 10:42:44.909483 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application conn.query(q) Aug 07 10:42:44.909483 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Aug 07 10:42:44.909483 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Aug 07 10:42:44.909483 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Aug 07 10:42:44.909483 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application result.read() Aug 07 10:42:44.909483 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Aug 07 10:42:44.910665 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Aug 07 10:42:44.910665 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Aug 07 10:42:44.910665 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application packet.raise_for_error() Aug 07 10:42:44.910665 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Aug 07 10:42:44.910665 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Aug 07 10:42:44.910665 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Aug 07 10:42:44.910665 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Aug 07 10:42:44.910665 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Aug 07 10:42:44.910665 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application Aug 07 10:42:44.910665 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Aug 07 10:42:44.910665 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application Aug 07 10:42:44.910665 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 07 10:42:44.910665 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Aug 07 10:42:44.910665 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return method(*args, **kwargs) Aug 07 10:42:44.910665 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 141, in create_user Aug 07 10:42:44.910665 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return base.filter_user(user_ref.to_dict()) Aug 07 10:42:44.910665 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Aug 07 10:42:44.910665 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application next(self.gen) Aug 07 10:42:44.911920 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Aug 07 10:42:44.911920 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application yield resource Aug 07 10:42:44.911920 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Aug 07 10:42:44.911920 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application next(self.gen) Aug 07 10:42:44.911920 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Aug 07 10:42:44.911920 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application self.session.rollback() Aug 07 10:42:44.911920 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Aug 07 10:42:44.911920 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application self.force_reraise() Aug 07 10:42:44.911920 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Aug 07 10:42:44.911920 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application raise self.value Aug 07 10:42:44.911920 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Aug 07 10:42:44.911920 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Aug 07 10:42:44.911920 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Aug 07 10:42:44.911920 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application session.commit() Aug 07 10:42:44.911920 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Aug 07 10:42:44.911920 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Aug 07 10:42:44.911920 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Aug 07 10:42:44.913016 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application self._prepare_impl() Aug 07 10:42:44.913016 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Aug 07 10:42:44.913016 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application self.session.flush() Aug 07 10:42:44.913016 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Aug 07 10:42:44.913016 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application self._flush(objects) Aug 07 10:42:44.913016 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Aug 07 10:42:44.913016 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Aug 07 10:42:44.913016 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Aug 07 10:42:44.913016 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application compat.raise_( Aug 07 10:42:44.913016 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Aug 07 10:42:44.913016 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application raise exception Aug 07 10:42:44.913016 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Aug 07 10:42:44.913016 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application flush_context.execute() Aug 07 10:42:44.913016 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Aug 07 10:42:44.913016 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application rec.execute(self) Aug 07 10:42:44.913016 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Aug 07 10:42:44.913016 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Aug 07 10:42:44.913910 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Aug 07 10:42:44.913910 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application _emit_insert_statements( Aug 07 10:42:44.913910 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1238, in _emit_insert_statements Aug 07 10:42:44.913910 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application result = connection._execute_20( Aug 07 10:42:44.913910 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1631, in _execute_20 Aug 07 10:42:44.913910 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Aug 07 10:42:44.913910 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 332, in _execute_on_connection Aug 07 10:42:44.913910 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Aug 07 10:42:44.913910 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1498, in _execute_clauseelement Aug 07 10:42:44.913910 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application ret = self._execute_context( Aug 07 10:42:44.913910 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1862, in _execute_context Aug 07 10:42:44.913910 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Aug 07 10:42:44.913910 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2041, in _handle_dbapi_exception Aug 07 10:42:44.913910 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Aug 07 10:42:44.913910 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Aug 07 10:42:44.913910 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application raise exception Aug 07 10:42:44.914778 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Aug 07 10:42:44.914778 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application self.dialect.do_execute( Aug 07 10:42:44.914778 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Aug 07 10:42:44.914778 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Aug 07 10:42:44.914778 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Aug 07 10:42:44.914778 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application result = self._query(query) Aug 07 10:42:44.914778 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Aug 07 10:42:44.914778 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application conn.query(q) Aug 07 10:42:44.914778 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Aug 07 10:42:44.914778 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Aug 07 10:42:44.914778 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Aug 07 10:42:44.914778 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application result.read() Aug 07 10:42:44.914778 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Aug 07 10:42:44.914778 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Aug 07 10:42:44.914778 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Aug 07 10:42:44.914778 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application packet.raise_for_error() Aug 07 10:42:44.915720 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Aug 07 10:42:44.915720 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Aug 07 10:42:44.915720 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Aug 07 10:42:44.915720 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Aug 07 10:42:44.915720 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Aug 07 10:42:44.915720 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application [SQL: INSERT INTO local_user (user_id, domain_id, name, failed_auth_count, failed_auth_at) VALUES (%(user_id)s, %(domain_id)s, %(name)s, %(failed_auth_count)s, %(failed_auth_at)s)] Aug 07 10:42:44.915720 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application [parameters: {'user_id': 'ce48e9fa1e3d45ec965fcf5072dcdcce', 'domain_id': 'default', 'name': 'cinder', 'failed_auth_count': 0, 'failed_auth_at': None}] Aug 07 10:42:44.915720 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Aug 07 10:42:44.915720 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application Aug 07 10:42:44.915720 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 07 10:42:44.915720 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application Aug 07 10:42:44.915720 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 07 10:42:44.915720 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 07 10:42:44.915720 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 07 10:42:44.915720 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 07 10:42:44.916741 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 07 10:42:44.916741 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 07 10:42:44.916741 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 07 10:42:44.916741 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 07 10:42:44.916741 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 07 10:42:44.916741 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 07 10:42:44.916741 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 07 10:42:44.916741 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 244, in post Aug 07 10:42:44.916741 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.create_user( Aug 07 10:42:44.916741 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 07 10:42:44.916741 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 07 10:42:44.916741 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 07 10:42:44.916741 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 07 10:42:44.916741 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 07 10:42:44.916741 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 07 10:42:44.916741 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1000, in create_user Aug 07 10:42:44.916741 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application ref = self._create_user_with_federated_objects(user, driver) Aug 07 10:42:44.918001 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 966, in _create_user_with_federated_objects Aug 07 10:42:44.918001 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application user = driver.create_user(user['id'], user) Aug 07 10:42:44.918001 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Aug 07 10:42:44.918001 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Aug 07 10:42:44.918001 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default. Aug 07 10:42:44.918001 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application Aug 07 10:42:44.920337 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 493/963] 10.222.0.25 () {60 vars in 1131 bytes} [Sun Aug 7 10:42:44 2022] POST /identity/v3/users => generated 160 bytes in 204 msecs (HTTP/1.1 409) 5 headers in 183 bytes (1 switches on core 0) Aug 07 10:42:44.926975 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-581fc4e0-5688-47ee-b98c-2be83f25c8b0 None None] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 10:42:45.039244 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-581fc4e0-5688-47ee-b98c-2be83f25c8b0 None None] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 10:42:45.064267 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=100 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 10:42:45.077059 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f2afff0-6bbd-4b93-8297-18d0a06b71ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 10:42:45.077831 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f2afff0-6bbd-4b93-8297-18d0a06b71ee None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:42:45.078203 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f2afff0-6bbd-4b93-8297-18d0a06b71ee None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:42:45.078621 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f2afff0-6bbd-4b93-8297-18d0a06b71ee None admin] PATH_INFO: `/v3/users/cinder` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:42:45.086692 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f2afff0-6bbd-4b93-8297-18d0a06b71ee None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 10:42:45.088841 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f2afff0-6bbd-4b93-8297-18d0a06b71ee None admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 10:42:45.101296 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application [None req-8f2afff0-6bbd-4b93-8297-18d0a06b71ee None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Aug 07 10:42:45.101296 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 07 10:42:45.101296 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 07 10:42:45.101296 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 07 10:42:45.101296 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 07 10:42:45.101296 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 07 10:42:45.101296 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 07 10:42:45.101296 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 07 10:42:45.101296 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 07 10:42:45.101296 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 07 10:42:45.101296 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 07 10:42:45.101296 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 07 10:42:45.101296 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 07 10:42:45.101296 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 07 10:42:45.101296 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 07 10:42:45.101296 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 07 10:42:45.102551 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 07 10:42:45.102551 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 07 10:42:45.102551 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 07 10:42:45.102551 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 07 10:42:45.102551 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 07 10:42:45.102551 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 07 10:42:45.102551 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 07 10:42:45.102551 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 07 10:42:45.102551 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 07 10:42:45.102551 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.get_or_create( Aug 07 10:42:45.102551 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 07 10:42:45.102551 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application with Lock( Aug 07 10:42:45.102551 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 07 10:42:45.102551 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self._enter() Aug 07 10:42:45.102551 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 07 10:42:45.102551 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 07 10:42:45.102551 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 07 10:42:45.102551 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.creator() Aug 07 10:42:45.103821 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 07 10:42:45.103821 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application created_value = creator( Aug 07 10:42:45.103821 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 07 10:42:45.103821 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 07 10:42:45.103821 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 07 10:42:45.103821 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 07 10:42:45.103821 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 07 10:42:45.103821 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 07 10:42:45.103821 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Aug 07 10:42:45.103821 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application Aug 07 10:42:45.105440 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 471/964] 10.222.0.25 () {56 vars in 1099 bytes} [Sun Aug 7 10:42:44 2022] GET /identity/v3/users/cinder => generated 84 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 07 10:42:45.111461 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4846400-75f4-4423-82cc-06e5cfe2c978 None admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 10:42:45.218587 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4846400-75f4-4423-82cc-06e5cfe2c978 None admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 10:42:45.221355 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 10:42:45.257587 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4392e5ba-8b8b-4886-893e-92da483e8252 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 10:42:45.258464 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4392e5ba-8b8b-4886-893e-92da483e8252 None admin] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:42:45.259636 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4392e5ba-8b8b-4886-893e-92da483e8252 None admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:42:45.260075 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4392e5ba-8b8b-4886-893e-92da483e8252 None admin] PATH_INFO: `/v3/users` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:42:45.260968 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4392e5ba-8b8b-4886-893e-92da483e8252 None admin] RBAC: Adding query filter params (name=cinder) {{(pid=80144) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 10:42:45.261426 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4392e5ba-8b8b-4886-893e-92da483e8252 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 10:42:45.263192 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4392e5ba-8b8b-4886-893e-92da483e8252 None admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 10:42:45.302578 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 494/965] 10.222.0.25 () {56 vars in 1101 bytes} [Sun Aug 7 10:42:45 2022] GET /identity/v3/users?name=cinder => generated 354 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 10:42:46.279877 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30e03972-ea6f-4eb5-accd-ab1f69c85b02 None None] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:42:46.280424 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30e03972-ea6f-4eb5-accd-ab1f69c85b02 None None] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:42:46.280424 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30e03972-ea6f-4eb5-accd-ab1f69c85b02 None None] PATH_INFO: `/` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:42:46.281895 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 472/966] 10.222.0.25 () {54 vars in 927 bytes} [Sun Aug 7 10:42:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 07 10:42:46.287113 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73053971-25a8-4609-8955-9ff0385f7a9d None None] REQUEST_METHOD: `POST` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:42:46.287510 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73053971-25a8-4609-8955-9ff0385f7a9d None None] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:42:46.287672 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73053971-25a8-4609-8955-9ff0385f7a9d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:42:46.307124 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.auth.core [None req-73053971-25a8-4609-8955-9ff0385f7a9d None None] MFA Rules not processed for user `bac4e6b76fe64217ac8fa44e71cffe6a`. Rule list: `[]` (Enabled: `True`). {{(pid=80144) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 10:42:46.308633 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.fernet_utils [None req-73053971-25a8-4609-8955-9ff0385f7a9d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80144) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 10:42:46.370133 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6f3c159-7eda-4828-b63a-1621e1223112 None None] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:42:46.370451 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6f3c159-7eda-4828-b63a-1621e1223112 None None] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:42:46.370682 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6f3c159-7eda-4828-b63a-1621e1223112 None None] PATH_INFO: `/` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:42:46.372291 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 473/967] 10.222.0.25 () {54 vars in 927 bytes} [Sun Aug 7 10:42:46 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 07 10:42:46.377784 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69fb6c3c-34ea-49c9-b662-bf897e619bc6 None None] REQUEST_METHOD: `POST` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:42:46.378126 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69fb6c3c-34ea-49c9-b662-bf897e619bc6 None None] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:42:46.378362 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69fb6c3c-34ea-49c9-b662-bf897e619bc6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:42:46.383041 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 495/968] 10.222.0.25 () {58 vars in 1024 bytes} [Sun Aug 7 10:42:46 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 07 10:42:46.398772 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.auth.core [None req-69fb6c3c-34ea-49c9-b662-bf897e619bc6 None None] MFA Rules not processed for user `bac4e6b76fe64217ac8fa44e71cffe6a`. Rule list: `[]` (Enabled: `True`). {{(pid=80143) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 10:42:46.400024 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-69fb6c3c-34ea-49c9-b662-bf897e619bc6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 10:42:46.403860 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73053971-25a8-4609-8955-9ff0385f7a9d None None] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 10:42:46.474827 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 474/969] 10.222.0.25 () {58 vars in 1024 bytes} [Sun Aug 7 10:42:46 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 07 10:42:46.498155 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69fb6c3c-34ea-49c9-b662-bf897e619bc6 None None] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 10:42:46.500954 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=100 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 10:42:46.512636 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73053971-25a8-4609-8955-9ff0385f7a9d None None] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 10:42:46.549711 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09ef7698-1b5d-4423-97c6-a8190d8e9fac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 10:42:46.550258 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09ef7698-1b5d-4423-97c6-a8190d8e9fac None admin] REQUEST_METHOD: `POST` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:42:46.550523 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09ef7698-1b5d-4423-97c6-a8190d8e9fac None admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:42:46.550915 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09ef7698-1b5d-4423-97c6-a8190d8e9fac None admin] PATH_INFO: `/v3/services` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:42:46.551583 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09ef7698-1b5d-4423-97c6-a8190d8e9fac None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 10:42:46.552934 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09ef7698-1b5d-4423-97c6-a8190d8e9fac None admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 10:42:46.578689 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 496/970] 10.222.0.25 () {60 vars in 1141 bytes} [Sun Aug 7 10:42:46 2022] POST /identity/v3/services => generated 242 bytes in 166 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 07 10:42:46.608840 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69fb6c3c-34ea-49c9-b662-bf897e619bc6 None None] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 10:42:46.626002 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=100 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 10:42:46.657783 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1e906f2-7000-4533-bf5a-c61c4f696659 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 10:42:46.658434 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1e906f2-7000-4533-bf5a-c61c4f696659 None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:42:46.658717 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1e906f2-7000-4533-bf5a-c61c4f696659 None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:42:46.659032 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1e906f2-7000-4533-bf5a-c61c4f696659 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:42:46.671397 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1e906f2-7000-4533-bf5a-c61c4f696659 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 07 10:42:46.676216 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application [None req-c1e906f2-7000-4533-bf5a-c61c4f696659 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 07 10:42:46.676216 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 07 10:42:46.676216 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 07 10:42:46.676216 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 07 10:42:46.676216 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 07 10:42:46.676216 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 07 10:42:46.676216 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 07 10:42:46.676216 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 07 10:42:46.676216 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 07 10:42:46.676216 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 07 10:42:46.676216 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 07 10:42:46.676216 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 07 10:42:46.676216 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 07 10:42:46.676216 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 07 10:42:46.676216 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 07 10:42:46.676216 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 07 10:42:46.677318 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 07 10:42:46.677318 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 07 10:42:46.677318 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 07 10:42:46.677318 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 07 10:42:46.677318 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 07 10:42:46.677318 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 07 10:42:46.677318 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 07 10:42:46.677318 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 07 10:42:46.677318 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 07 10:42:46.677318 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.get_or_create( Aug 07 10:42:46.677318 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 07 10:42:46.677318 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application with Lock( Aug 07 10:42:46.677318 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 07 10:42:46.677318 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self._enter() Aug 07 10:42:46.677318 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 07 10:42:46.677318 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 07 10:42:46.678174 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 07 10:42:46.678174 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.creator() Aug 07 10:42:46.678174 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 07 10:42:46.678174 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application created_value = creator( Aug 07 10:42:46.678174 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 07 10:42:46.678174 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 07 10:42:46.678174 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 07 10:42:46.678174 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 07 10:42:46.678174 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 07 10:42:46.678174 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 07 10:42:46.678174 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 07 10:42:46.678174 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application Aug 07 10:42:46.678858 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98bd928c-68ae-44ea-9a58-ef90d7023145 None None] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:42:46.678858 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98bd928c-68ae-44ea-9a58-ef90d7023145 None None] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:42:46.678858 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98bd928c-68ae-44ea-9a58-ef90d7023145 None None] PATH_INFO: `/` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:42:46.679940 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 497/971] 10.222.0.25 () {54 vars in 927 bytes} [Sun Aug 7 10:42:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 07 10:42:46.683334 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 475/972] 10.222.0.25 () {56 vars in 1120 bytes} [Sun Aug 7 10:42:46 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 07 10:42:46.684786 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98bd928c-68ae-44ea-9a58-ef90d7023145 None None] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 10:42:46.685266 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be92ea37-3f53-41f6-8908-8f1a58545710 None None] REQUEST_METHOD: `POST` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:42:46.685553 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be92ea37-3f53-41f6-8908-8f1a58545710 None None] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:42:46.685827 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be92ea37-3f53-41f6-8908-8f1a58545710 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:42:46.704558 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.auth.core [None req-be92ea37-3f53-41f6-8908-8f1a58545710 None None] MFA Rules not processed for user `bac4e6b76fe64217ac8fa44e71cffe6a`. Rule list: `[]` (Enabled: `True`). {{(pid=80143) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 10:42:46.705785 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-be92ea37-3f53-41f6-8908-8f1a58545710 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 10:42:46.786949 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 476/973] 10.222.0.25 () {58 vars in 1024 bytes} [Sun Aug 7 10:42:46 2022] POST /identity/v3/auth/tokens => generated 1570 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 07 10:42:46.803326 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98bd928c-68ae-44ea-9a58-ef90d7023145 None None] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 10:42:46.806597 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be92ea37-3f53-41f6-8908-8f1a58545710 None None] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 10:42:46.842863 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a46ab325-1a2a-468c-affc-04d355e8870c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 10:42:46.843511 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a46ab325-1a2a-468c-affc-04d355e8870c None admin] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:42:46.843791 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a46ab325-1a2a-468c-affc-04d355e8870c None admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:42:46.844066 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a46ab325-1a2a-468c-affc-04d355e8870c None admin] PATH_INFO: `/v3/roles` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:42:46.845088 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a46ab325-1a2a-468c-affc-04d355e8870c None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80144) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 10:42:46.845088 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a46ab325-1a2a-468c-affc-04d355e8870c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 10:42:46.846371 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a46ab325-1a2a-468c-affc-04d355e8870c None admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 10:42:46.857202 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 498/974] 10.222.0.25 () {56 vars in 1115 bytes} [Sun Aug 7 10:42:46 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 10:42:46.861624 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a46ab325-1a2a-468c-affc-04d355e8870c None admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 10:42:46.878241 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 10:42:46.916270 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=100 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 10:42:46.920332 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be92ea37-3f53-41f6-8908-8f1a58545710 None None] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 10:42:46.958215 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-464a458c-17b4-4e7b-ad84-5ebd7be8a626 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 10:42:46.958848 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-464a458c-17b4-4e7b-ad84-5ebd7be8a626 None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:42:46.959153 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-464a458c-17b4-4e7b-ad84-5ebd7be8a626 None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:42:46.959427 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-464a458c-17b4-4e7b-ad84-5ebd7be8a626 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:42:46.960066 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-464a458c-17b4-4e7b-ad84-5ebd7be8a626 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 10:42:46.961455 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-464a458c-17b4-4e7b-ad84-5ebd7be8a626 None admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 10:42:46.970493 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application [None req-464a458c-17b4-4e7b-ad84-5ebd7be8a626 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 07 10:42:46.970493 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 07 10:42:46.970493 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 07 10:42:46.970493 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 07 10:42:46.970493 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 07 10:42:46.970493 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 07 10:42:46.970493 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 07 10:42:46.970493 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 07 10:42:46.970493 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 07 10:42:46.970493 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application Aug 07 10:42:46.970493 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 07 10:42:46.970493 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application Aug 07 10:42:46.970493 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 07 10:42:46.970493 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 07 10:42:46.970493 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 07 10:42:46.970493 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 07 10:42:46.970493 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 07 10:42:46.971546 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 07 10:42:46.971546 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 07 10:42:46.971546 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 07 10:42:46.971546 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 07 10:42:46.971546 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 07 10:42:46.971546 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 07 10:42:46.971546 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 07 10:42:46.971546 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 07 10:42:46.971546 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 07 10:42:46.971546 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 07 10:42:46.971546 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 07 10:42:46.971546 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 07 10:42:46.971546 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 07 10:42:46.971546 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 07 10:42:46.971546 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 07 10:42:46.971546 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.get_or_create( Aug 07 10:42:46.972399 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 07 10:42:46.972399 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application with Lock( Aug 07 10:42:46.972399 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 07 10:42:46.972399 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self._enter() Aug 07 10:42:46.972399 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 07 10:42:46.972399 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 07 10:42:46.972399 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 07 10:42:46.972399 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.creator() Aug 07 10:42:46.972399 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 07 10:42:46.972399 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application created_value = creator( Aug 07 10:42:46.972399 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 07 10:42:46.972399 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 07 10:42:46.972399 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 07 10:42:46.972399 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application Aug 07 10:42:46.973195 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a46ab325-1a2a-468c-affc-04d355e8870c None admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 10:42:46.973279 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 477/975] 10.222.0.25 () {56 vars in 1126 bytes} [Sun Aug 7 10:42:46 2022] GET /identity/v3/services/object-store => generated 93 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 07 10:42:46.977897 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-464a458c-17b4-4e7b-ad84-5ebd7be8a626 None admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 10:42:47.013251 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a70cacbe-d007-4e21-ae36-9f45374163ba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 10:42:47.013893 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a70cacbe-d007-4e21-ae36-9f45374163ba None admin] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:42:47.014180 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a70cacbe-d007-4e21-ae36-9f45374163ba None admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:42:47.014455 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a70cacbe-d007-4e21-ae36-9f45374163ba None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:42:47.025868 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a70cacbe-d007-4e21-ae36-9f45374163ba None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 10:42:47.027353 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a70cacbe-d007-4e21-ae36-9f45374163ba None admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 10:42:47.042250 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application [None req-a70cacbe-d007-4e21-ae36-9f45374163ba None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 07 10:42:47.042250 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 07 10:42:47.042250 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 07 10:42:47.042250 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 07 10:42:47.042250 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 07 10:42:47.042250 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 07 10:42:47.042250 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 07 10:42:47.042250 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 07 10:42:47.042250 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 07 10:42:47.042250 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 07 10:42:47.042250 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 07 10:42:47.042250 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 07 10:42:47.042250 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 07 10:42:47.042250 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 07 10:42:47.042250 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 07 10:42:47.042250 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 07 10:42:47.043310 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 07 10:42:47.043310 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 07 10:42:47.043310 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 07 10:42:47.043310 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 07 10:42:47.043310 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 07 10:42:47.043310 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self.get_or_create( Aug 07 10:42:47.043310 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 07 10:42:47.043310 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application with Lock( Aug 07 10:42:47.043310 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 07 10:42:47.043310 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self._enter() Aug 07 10:42:47.043310 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 07 10:42:47.043310 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 07 10:42:47.043310 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 07 10:42:47.043310 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self.creator() Aug 07 10:42:47.043310 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 07 10:42:47.043310 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application created_value = creator( Aug 07 10:42:47.043310 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 07 10:42:47.044240 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 07 10:42:47.044240 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 07 10:42:47.044240 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application Aug 07 10:42:47.044240 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 499/976] 10.222.0.25 () {56 vars in 1108 bytes} [Sun Aug 7 10:42:46 2022] GET /identity/v3/domains/Default => generated 87 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 07 10:42:47.048895 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a70cacbe-d007-4e21-ae36-9f45374163ba None admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 10:42:47.087860 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-464a458c-17b4-4e7b-ad84-5ebd7be8a626 None admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 10:42:47.122140 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=100 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 10:42:47.123140 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c65db31-00ef-4da0-bda9-fbc0c5a5e657 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 10:42:47.123851 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c65db31-00ef-4da0-bda9-fbc0c5a5e657 None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:42:47.124141 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c65db31-00ef-4da0-bda9-fbc0c5a5e657 None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:42:47.124532 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c65db31-00ef-4da0-bda9-fbc0c5a5e657 None admin] PATH_INFO: `/v3/services` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:42:47.125432 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c65db31-00ef-4da0-bda9-fbc0c5a5e657 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=80143) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 10:42:47.125724 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c65db31-00ef-4da0-bda9-fbc0c5a5e657 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 10:42:47.127153 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c65db31-00ef-4da0-bda9-fbc0c5a5e657 None admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 10:42:47.132164 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 478/977] 10.222.0.25 () {56 vars in 1122 bytes} [Sun Aug 7 10:42:46 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 07 10:42:47.138426 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c65db31-00ef-4da0-bda9-fbc0c5a5e657 None admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 10:42:47.163922 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a70cacbe-d007-4e21-ae36-9f45374163ba None admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 10:42:47.168699 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=100 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 10:42:47.201760 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84923641-5b3c-4e1c-8771-901c3d372046 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 10:42:47.202372 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84923641-5b3c-4e1c-8771-901c3d372046 None admin] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:42:47.202659 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84923641-5b3c-4e1c-8771-901c3d372046 None admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:42:47.202959 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84923641-5b3c-4e1c-8771-901c3d372046 None admin] PATH_INFO: `/v3/domains` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:42:47.203605 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84923641-5b3c-4e1c-8771-901c3d372046 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80144) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 10:42:47.203908 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84923641-5b3c-4e1c-8771-901c3d372046 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 10:42:47.205256 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84923641-5b3c-4e1c-8771-901c3d372046 None admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 10:42:47.218253 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 500/978] 10.222.0.25 () {56 vars in 1109 bytes} [Sun Aug 7 10:42:47 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 07 10:42:47.223928 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84923641-5b3c-4e1c-8771-901c3d372046 None admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 10:42:47.256308 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c65db31-00ef-4da0-bda9-fbc0c5a5e657 None admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 10:42:47.295079 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d3cbc49-352e-450a-8662-4f576fbbb27e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 10:42:47.295739 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d3cbc49-352e-450a-8662-4f576fbbb27e None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:42:47.295966 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d3cbc49-352e-450a-8662-4f576fbbb27e None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:42:47.296246 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d3cbc49-352e-450a-8662-4f576fbbb27e None admin] PATH_INFO: `/v3/services` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:42:47.296882 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d3cbc49-352e-450a-8662-4f576fbbb27e None admin] RBAC: Adding query filter params (type=object-store) {{(pid=80143) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 10:42:47.297186 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d3cbc49-352e-450a-8662-4f576fbbb27e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 10:42:47.298542 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d3cbc49-352e-450a-8662-4f576fbbb27e None admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 10:42:47.303538 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 479/979] 10.222.0.25 () {56 vars in 1122 bytes} [Sun Aug 7 10:42:47 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 07 10:42:47.309865 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d3cbc49-352e-450a-8662-4f576fbbb27e None admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 10:42:47.341120 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84923641-5b3c-4e1c-8771-901c3d372046 None admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 10:42:47.368864 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=100 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 10:42:47.379179 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29b917af-08d9-4d50-8b07-cafc5c5878b9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 10:42:47.379813 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29b917af-08d9-4d50-8b07-cafc5c5878b9 None admin] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:42:47.380169 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29b917af-08d9-4d50-8b07-cafc5c5878b9 None admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:42:47.380445 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29b917af-08d9-4d50-8b07-cafc5c5878b9 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:42:47.387989 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29b917af-08d9-4d50-8b07-cafc5c5878b9 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 10:42:47.389393 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29b917af-08d9-4d50-8b07-cafc5c5878b9 None admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 10:42:47.401191 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application [None req-29b917af-08d9-4d50-8b07-cafc5c5878b9 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 07 10:42:47.401191 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 07 10:42:47.401191 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 07 10:42:47.401191 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 07 10:42:47.401191 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 07 10:42:47.401191 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 07 10:42:47.401191 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 07 10:42:47.401191 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 07 10:42:47.401191 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 07 10:42:47.401191 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 07 10:42:47.401191 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 07 10:42:47.401191 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 07 10:42:47.401191 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 07 10:42:47.401191 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 07 10:42:47.401191 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 07 10:42:47.401191 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 07 10:42:47.402129 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 07 10:42:47.402129 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 07 10:42:47.402129 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 07 10:42:47.402129 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 07 10:42:47.402129 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 07 10:42:47.402129 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 07 10:42:47.402129 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 07 10:42:47.402129 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 07 10:42:47.402129 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 07 10:42:47.402129 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self.get_or_create( Aug 07 10:42:47.402129 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 07 10:42:47.402129 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application with Lock( Aug 07 10:42:47.402129 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 07 10:42:47.402129 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self._enter() Aug 07 10:42:47.402129 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 07 10:42:47.402129 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 07 10:42:47.402129 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 07 10:42:47.402129 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self.creator() Aug 07 10:42:47.403089 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 07 10:42:47.403089 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application created_value = creator( Aug 07 10:42:47.403089 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 07 10:42:47.403089 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 07 10:42:47.403089 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 07 10:42:47.403089 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 07 10:42:47.403089 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 07 10:42:47.403089 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 07 10:42:47.403089 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 07 10:42:47.403089 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application Aug 07 10:42:47.403089 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 501/980] 10.222.0.25 () {56 vars in 1117 bytes} [Sun Aug 7 10:42:47 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 07 10:42:47.407130 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=100 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 10:42:47.408138 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29b917af-08d9-4d50-8b07-cafc5c5878b9 None admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 10:42:47.426117 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d3cbc49-352e-450a-8662-4f576fbbb27e None admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 10:42:47.464626 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-661c7a51-5008-40ab-a2be-ea595d06d7aa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 10:42:47.465325 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-661c7a51-5008-40ab-a2be-ea595d06d7aa None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:42:47.465683 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-661c7a51-5008-40ab-a2be-ea595d06d7aa None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:42:47.466051 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-661c7a51-5008-40ab-a2be-ea595d06d7aa None admin] PATH_INFO: `/v3/endpoints` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:42:47.466830 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-661c7a51-5008-40ab-a2be-ea595d06d7aa None admin] RBAC: Adding query filter params (interface=admin, service_id=8e77fc87597446b18b3955ba5f279642, region_id=RegionOne) {{(pid=80143) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 10:42:47.467385 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-661c7a51-5008-40ab-a2be-ea595d06d7aa None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 10:42:47.468678 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-661c7a51-5008-40ab-a2be-ea595d06d7aa None admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 10:42:47.479148 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 480/981] 10.222.0.25 () {56 vars in 1249 bytes} [Sun Aug 7 10:42:47 2022] GET /identity/v3/endpoints?service_id=8e77fc87597446b18b3955ba5f279642&interface=admin®ion_id=RegionOne => generated 193 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 10:42:47.480138 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-661c7a51-5008-40ab-a2be-ea595d06d7aa None admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 10:42:47.517356 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29b917af-08d9-4d50-8b07-cafc5c5878b9 None admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 10:42:47.552060 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7adaa41f-ea64-4688-bc6e-dc02eeabf3b9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 10:42:47.552926 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7adaa41f-ea64-4688-bc6e-dc02eeabf3b9 None admin] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:42:47.553369 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7adaa41f-ea64-4688-bc6e-dc02eeabf3b9 None admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:42:47.553650 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7adaa41f-ea64-4688-bc6e-dc02eeabf3b9 None admin] PATH_INFO: `/v3/users` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:42:47.554385 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7adaa41f-ea64-4688-bc6e-dc02eeabf3b9 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80144) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 10:42:47.554700 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7adaa41f-ea64-4688-bc6e-dc02eeabf3b9 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 10:42:47.556375 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7adaa41f-ea64-4688-bc6e-dc02eeabf3b9 None admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 10:42:47.579140 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 502/982] 10.222.0.25 () {56 vars in 1149 bytes} [Sun Aug 7 10:42:47 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 07 10:42:47.580184 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7adaa41f-ea64-4688-bc6e-dc02eeabf3b9 None admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 10:42:47.601195 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-661c7a51-5008-40ab-a2be-ea595d06d7aa None admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 10:42:47.630163 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=100 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 10:42:47.638332 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1a7cf83-ab31-4a0f-bf3e-02734304af1a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 10:42:47.639167 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1a7cf83-ab31-4a0f-bf3e-02734304af1a None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:42:47.639280 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1a7cf83-ab31-4a0f-bf3e-02734304af1a None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:42:47.639515 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1a7cf83-ab31-4a0f-bf3e-02734304af1a None admin] PATH_INFO: `/v3/services` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:42:47.640125 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1a7cf83-ab31-4a0f-bf3e-02734304af1a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 10:42:47.641493 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1a7cf83-ab31-4a0f-bf3e-02734304af1a None admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 10:42:47.647915 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 481/983] 10.222.0.25 () {56 vars in 1087 bytes} [Sun Aug 7 10:42:47 2022] GET /identity/v3/services => generated 1727 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 07 10:42:47.693520 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7adaa41f-ea64-4688-bc6e-dc02eeabf3b9 None admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 10:42:47.733776 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44f2697a-2807-4a3b-b93a-a6897616abbf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 10:42:47.734414 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44f2697a-2807-4a3b-b93a-a6897616abbf None admin] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:42:47.734702 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44f2697a-2807-4a3b-b93a-a6897616abbf None admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:42:47.735019 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44f2697a-2807-4a3b-b93a-a6897616abbf None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:42:47.808769 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44f2697a-2807-4a3b-b93a-a6897616abbf None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 10:42:47.810212 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44f2697a-2807-4a3b-b93a-a6897616abbf None admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 10:42:47.857528 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=100 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 10:42:47.920285 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 503/984] 10.222.0.25 () {58 vars in 1282 bytes} [Sun Aug 7 10:42:47 2022] GET /identity/v3/auth/tokens => generated 1570 bytes in 336 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 07 10:42:47.921932 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1a7cf83-ab31-4a0f-bf3e-02734304af1a None admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 10:42:47.941384 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=100 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 10:42:48.025665 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1a7cf83-ab31-4a0f-bf3e-02734304af1a None admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 10:42:48.060079 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c5239c5-6b5b-4840-9c21-d36c5027b972 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 10:42:48.060806 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c5239c5-6b5b-4840-9c21-d36c5027b972 None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:42:48.061092 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c5239c5-6b5b-4840-9c21-d36c5027b972 None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:42:48.061371 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c5239c5-6b5b-4840-9c21-d36c5027b972 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:42:48.071572 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c5239c5-6b5b-4840-9c21-d36c5027b972 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 10:42:48.073015 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c5239c5-6b5b-4840-9c21-d36c5027b972 None admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 10:42:48.090017 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application [None req-5c5239c5-6b5b-4840-9c21-d36c5027b972 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 07 10:42:48.090017 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 07 10:42:48.090017 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 07 10:42:48.090017 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 07 10:42:48.090017 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 07 10:42:48.090017 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 07 10:42:48.090017 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 07 10:42:48.090017 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 07 10:42:48.090017 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 07 10:42:48.090017 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 07 10:42:48.090017 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 07 10:42:48.090017 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 07 10:42:48.090017 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 07 10:42:48.090017 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 07 10:42:48.090017 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 07 10:42:48.090017 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 07 10:42:48.091014 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 07 10:42:48.091014 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 07 10:42:48.091014 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 07 10:42:48.091014 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 07 10:42:48.091014 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 07 10:42:48.091014 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.get_or_create( Aug 07 10:42:48.091014 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 07 10:42:48.091014 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application with Lock( Aug 07 10:42:48.091014 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 07 10:42:48.091014 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self._enter() Aug 07 10:42:48.091014 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 07 10:42:48.091014 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 07 10:42:48.091014 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 07 10:42:48.091014 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.creator() Aug 07 10:42:48.091014 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 07 10:42:48.091014 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application created_value = creator( Aug 07 10:42:48.091014 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 07 10:42:48.096873 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 07 10:42:48.096873 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 07 10:42:48.096873 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application Aug 07 10:42:48.096873 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 482/985] 10.222.0.25 () {56 vars in 1108 bytes} [Sun Aug 7 10:42:47 2022] GET /identity/v3/domains/Default => generated 87 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 07 10:42:48.098594 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44f2697a-2807-4a3b-b93a-a6897616abbf None admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 10:42:48.202352 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44f2697a-2807-4a3b-b93a-a6897616abbf None admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 10:42:48.237035 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da18fc79-f0db-4542-a5e3-a8e644058d89 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 10:42:48.237901 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da18fc79-f0db-4542-a5e3-a8e644058d89 None admin] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:42:48.238183 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da18fc79-f0db-4542-a5e3-a8e644058d89 None admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:42:48.238456 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da18fc79-f0db-4542-a5e3-a8e644058d89 None admin] PATH_INFO: `/v3/domains` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:42:48.239160 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da18fc79-f0db-4542-a5e3-a8e644058d89 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80144) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 10:42:48.239469 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da18fc79-f0db-4542-a5e3-a8e644058d89 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 10:42:48.240817 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da18fc79-f0db-4542-a5e3-a8e644058d89 None admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 10:42:48.252202 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 504/986] 10.222.0.25 () {56 vars in 1109 bytes} [Sun Aug 7 10:42:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 07 10:42:48.257580 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c5239c5-6b5b-4840-9c21-d36c5027b972 None admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 10:42:48.330278 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=100 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 10:42:48.364041 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c5239c5-6b5b-4840-9c21-d36c5027b972 None admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 10:42:48.398059 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6aeb343-f999-42ad-9994-9329d8271bc2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 10:42:48.398698 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6aeb343-f999-42ad-9994-9329d8271bc2 None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:42:48.399006 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6aeb343-f999-42ad-9994-9329d8271bc2 None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:42:48.399317 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6aeb343-f999-42ad-9994-9329d8271bc2 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:42:48.406007 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6aeb343-f999-42ad-9994-9329d8271bc2 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 10:42:48.410380 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6aeb343-f999-42ad-9994-9329d8271bc2 None admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 10:42:48.424323 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application [None req-a6aeb343-f999-42ad-9994-9329d8271bc2 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 07 10:42:48.424323 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 07 10:42:48.424323 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 07 10:42:48.424323 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 07 10:42:48.424323 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 07 10:42:48.424323 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 07 10:42:48.424323 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 07 10:42:48.424323 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 07 10:42:48.424323 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 07 10:42:48.424323 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 07 10:42:48.424323 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 07 10:42:48.424323 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 07 10:42:48.424323 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 07 10:42:48.424323 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 07 10:42:48.424323 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 07 10:42:48.425214 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 07 10:42:48.425214 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 07 10:42:48.425214 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 07 10:42:48.425214 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 07 10:42:48.425214 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 07 10:42:48.425214 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 07 10:42:48.425214 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.get_or_create( Aug 07 10:42:48.425214 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 07 10:42:48.425214 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application with Lock( Aug 07 10:42:48.425214 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 07 10:42:48.425214 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self._enter() Aug 07 10:42:48.425214 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 07 10:42:48.425214 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 07 10:42:48.425214 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 07 10:42:48.425214 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.creator() Aug 07 10:42:48.425214 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 07 10:42:48.425214 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application created_value = creator( Aug 07 10:42:48.426118 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 07 10:42:48.426118 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 07 10:42:48.426118 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 07 10:42:48.426118 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 07 10:42:48.426118 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 07 10:42:48.426118 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 07 10:42:48.426118 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 07 10:42:48.426118 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application Aug 07 10:42:48.426118 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 483/987] 10.222.0.25 () {56 vars in 1111 bytes} [Sun Aug 7 10:42:48 2022] GET /identity/v3/projects/service => generated 88 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 07 10:42:48.433119 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da18fc79-f0db-4542-a5e3-a8e644058d89 None admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 10:42:48.437096 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=100 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 10:42:48.535370 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da18fc79-f0db-4542-a5e3-a8e644058d89 None admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 10:42:48.568520 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcc2b97b-d674-49e1-9989-6969a92b3f05 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 10:42:48.569124 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcc2b97b-d674-49e1-9989-6969a92b3f05 None admin] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:42:48.569402 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcc2b97b-d674-49e1-9989-6969a92b3f05 None admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:42:48.569674 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcc2b97b-d674-49e1-9989-6969a92b3f05 None admin] PATH_INFO: `/v3/projects` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:42:48.570354 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcc2b97b-d674-49e1-9989-6969a92b3f05 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80144) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 10:42:48.570659 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcc2b97b-d674-49e1-9989-6969a92b3f05 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 10:42:48.572816 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcc2b97b-d674-49e1-9989-6969a92b3f05 None admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 10:42:48.585411 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 505/988] 10.222.0.25 () {56 vars in 1148 bytes} [Sun Aug 7 10:42:48 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 10:42:48.591410 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6aeb343-f999-42ad-9994-9329d8271bc2 None admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 10:42:48.600802 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a1042f7-d715-4bcc-8f6e-5417a0456dfd None None] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:42:48.601094 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a1042f7-d715-4bcc-8f6e-5417a0456dfd None None] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:42:48.601405 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a1042f7-d715-4bcc-8f6e-5417a0456dfd None None] PATH_INFO: `/` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:42:48.603389 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 506/989] 10.222.0.25 () {54 vars in 927 bytes} [Sun Aug 7 10:42:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 07 10:42:48.608083 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2231c28e-451f-4a53-8a70-f0e32bd369b4 None None] REQUEST_METHOD: `POST` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:42:48.608367 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2231c28e-451f-4a53-8a70-f0e32bd369b4 None None] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:42:48.608644 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2231c28e-451f-4a53-8a70-f0e32bd369b4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:42:48.627499 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.auth.core [None req-2231c28e-451f-4a53-8a70-f0e32bd369b4 None None] MFA Rules not processed for user `bac4e6b76fe64217ac8fa44e71cffe6a`. Rule list: `[]` (Enabled: `True`). {{(pid=80144) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 10:42:48.628700 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.fernet_utils [None req-2231c28e-451f-4a53-8a70-f0e32bd369b4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80144) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 10:42:48.690892 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=100 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 10:42:48.702099 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6aeb343-f999-42ad-9994-9329d8271bc2 None admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 10:42:48.711593 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 507/990] 10.222.0.25 () {58 vars in 1024 bytes} [Sun Aug 7 10:42:48 2022] POST /identity/v3/auth/tokens => generated 1570 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 07 10:42:48.724465 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2231c28e-451f-4a53-8a70-f0e32bd369b4 None None] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 10:42:48.731606 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=100 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 10:42:48.738800 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1311b293-5f2e-4413-aff9-783a05f0234f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 10:42:48.739458 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1311b293-5f2e-4413-aff9-783a05f0234f None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:42:48.739743 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1311b293-5f2e-4413-aff9-783a05f0234f None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:42:48.740021 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1311b293-5f2e-4413-aff9-783a05f0234f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:42:48.740753 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1311b293-5f2e-4413-aff9-783a05f0234f None admin] RBAC: Adding query filter params (role.id=3f1b8463c86840fb848a4cdf7c3d98be, scope.project.id=6de1f3cac02c4bcdb05dd2337176033d, user.id=de1d6c24b1a14d10a20d82a4c65793d8) {{(pid=80143) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 10:42:48.741065 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1311b293-5f2e-4413-aff9-783a05f0234f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 10:42:48.742486 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1311b293-5f2e-4413-aff9-783a05f0234f None admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 10:42:48.758966 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 484/991] 10.222.0.25 () {56 vars in 1374 bytes} [Sun Aug 7 10:42:48 2022] GET /identity/v3/role_assignments?user.id=de1d6c24b1a14d10a20d82a4c65793d8&scope.project.id=6de1f3cac02c4bcdb05dd2337176033d&role.id=3f1b8463c86840fb848a4cdf7c3d98be => generated 608 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 10:42:48.844417 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2231c28e-451f-4a53-8a70-f0e32bd369b4 None None] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 10:42:48.881211 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9affecb9-cbb3-414c-ad37-8c33574ab547 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 10:42:48.881860 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9affecb9-cbb3-414c-ad37-8c33574ab547 None admin] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:42:48.882144 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9affecb9-cbb3-414c-ad37-8c33574ab547 None admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:42:48.882422 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9affecb9-cbb3-414c-ad37-8c33574ab547 None admin] PATH_INFO: `/v3/services/compute` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:42:48.883042 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9affecb9-cbb3-414c-ad37-8c33574ab547 None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 10:42:48.884491 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9affecb9-cbb3-414c-ad37-8c33574ab547 None admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 10:42:48.893911 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application [None req-9affecb9-cbb3-414c-ad37-8c33574ab547 None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Aug 07 10:42:48.893911 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 07 10:42:48.893911 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 07 10:42:48.893911 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 07 10:42:48.893911 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 07 10:42:48.893911 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 07 10:42:48.893911 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 07 10:42:48.893911 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 07 10:42:48.893911 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Aug 07 10:42:48.893911 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application Aug 07 10:42:48.893911 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 07 10:42:48.893911 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application Aug 07 10:42:48.893911 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 07 10:42:48.893911 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 07 10:42:48.893911 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 07 10:42:48.893911 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 07 10:42:48.893911 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 07 10:42:48.895179 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 07 10:42:48.895179 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 07 10:42:48.895179 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 07 10:42:48.895179 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 07 10:42:48.895179 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 07 10:42:48.895179 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 07 10:42:48.895179 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 07 10:42:48.895179 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 07 10:42:48.895179 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 07 10:42:48.895179 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 07 10:42:48.895179 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 07 10:42:48.895179 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 07 10:42:48.895179 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 07 10:42:48.895179 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 07 10:42:48.895179 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 07 10:42:48.895179 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self.get_or_create( Aug 07 10:42:48.896310 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 07 10:42:48.896310 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application with Lock( Aug 07 10:42:48.896310 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 07 10:42:48.896310 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self._enter() Aug 07 10:42:48.896310 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 07 10:42:48.896310 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 07 10:42:48.896310 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 07 10:42:48.896310 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self.creator() Aug 07 10:42:48.896310 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 07 10:42:48.896310 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application created_value = creator( Aug 07 10:42:48.896310 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 07 10:42:48.896310 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 07 10:42:48.896310 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Aug 07 10:42:48.896310 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application Aug 07 10:42:48.900693 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 508/992] 10.222.0.25 () {56 vars in 1111 bytes} [Sun Aug 7 10:42:48 2022] GET /identity/v3/services/compute => generated 88 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 07 10:42:48.901477 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1311b293-5f2e-4413-aff9-783a05f0234f None admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 10:42:49.010978 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1311b293-5f2e-4413-aff9-783a05f0234f None admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 10:42:49.045587 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d24115a-6771-4a8c-969d-e83efd2d7421 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 10:42:49.046201 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d24115a-6771-4a8c-969d-e83efd2d7421 None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:42:49.046485 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d24115a-6771-4a8c-969d-e83efd2d7421 None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:42:49.046760 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d24115a-6771-4a8c-969d-e83efd2d7421 None admin] PATH_INFO: `/v3/services` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:42:49.047416 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d24115a-6771-4a8c-969d-e83efd2d7421 None admin] RBAC: Adding query filter params (name=compute) {{(pid=80143) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 10:42:49.047724 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d24115a-6771-4a8c-969d-e83efd2d7421 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 10:42:49.049095 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d24115a-6771-4a8c-969d-e83efd2d7421 None admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 10:42:49.053920 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 485/993] 10.222.0.25 () {56 vars in 1112 bytes} [Sun Aug 7 10:42:48 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 10:42:49.059648 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9affecb9-cbb3-414c-ad37-8c33574ab547 None admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 10:42:49.120159 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=100 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 10:42:49.164671 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9affecb9-cbb3-414c-ad37-8c33574ab547 None admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 10:42:49.198916 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b816919e-2e4a-4207-9d82-62085623ce98 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 10:42:49.199551 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b816919e-2e4a-4207-9d82-62085623ce98 None admin] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:42:49.199836 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b816919e-2e4a-4207-9d82-62085623ce98 None admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:42:49.200111 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b816919e-2e4a-4207-9d82-62085623ce98 None admin] PATH_INFO: `/v3/services` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:42:49.200738 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b816919e-2e4a-4207-9d82-62085623ce98 None admin] RBAC: Adding query filter params (type=compute) {{(pid=80144) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 10:42:49.201047 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b816919e-2e4a-4207-9d82-62085623ce98 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 10:42:49.202399 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b816919e-2e4a-4207-9d82-62085623ce98 None admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 10:42:49.212865 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 509/994] 10.222.0.25 () {56 vars in 1112 bytes} [Sun Aug 7 10:42:49 2022] GET /identity/v3/services?type=compute => generated 352 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 10:42:49.214132 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d24115a-6771-4a8c-969d-e83efd2d7421 None admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 10:42:49.275903 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=100 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 10:42:49.319906 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d24115a-6771-4a8c-969d-e83efd2d7421 None admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 10:42:49.353941 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54f272b7-4e3b-4774-9253-8ba52f3991c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 10:42:49.354568 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54f272b7-4e3b-4774-9253-8ba52f3991c2 None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:42:49.354825 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54f272b7-4e3b-4774-9253-8ba52f3991c2 None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:42:49.355131 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54f272b7-4e3b-4774-9253-8ba52f3991c2 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:42:49.355844 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54f272b7-4e3b-4774-9253-8ba52f3991c2 None admin] RBAC: Adding query filter params (interface=public, service_id=91d79125f8fd4ad598f3141cdb345c72, region_id=RegionOne) {{(pid=80143) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 10:42:49.356155 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54f272b7-4e3b-4774-9253-8ba52f3991c2 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 10:42:49.357516 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54f272b7-4e3b-4774-9253-8ba52f3991c2 None admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 10:42:49.367695 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 486/995] 10.222.0.25 () {56 vars in 1251 bytes} [Sun Aug 7 10:42:49 2022] GET /identity/v3/endpoints?service_id=91d79125f8fd4ad598f3141cdb345c72&interface=public®ion_id=RegionOne => generated 194 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 07 10:42:49.369048 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b816919e-2e4a-4207-9d82-62085623ce98 None admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 10:42:49.479282 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b816919e-2e4a-4207-9d82-62085623ce98 None admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 10:42:49.503411 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=100 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 10:42:49.517312 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14697773-3497-49a2-b17e-1e3bda625de3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 10:42:49.517929 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14697773-3497-49a2-b17e-1e3bda625de3 None admin] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:42:49.518210 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14697773-3497-49a2-b17e-1e3bda625de3 None admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:42:49.518484 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14697773-3497-49a2-b17e-1e3bda625de3 None admin] PATH_INFO: `/v3/services` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:42:49.519116 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14697773-3497-49a2-b17e-1e3bda625de3 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 10:42:49.520460 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14697773-3497-49a2-b17e-1e3bda625de3 None admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 10:42:49.531265 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 510/996] 10.222.0.25 () {56 vars in 1087 bytes} [Sun Aug 7 10:42:49 2022] GET /identity/v3/services => generated 1727 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 07 10:42:49.677785 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db4b4a79-ac43-488d-9bca-8a836ce83b13 None None] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:42:49.677995 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db4b4a79-ac43-488d-9bca-8a836ce83b13 None None] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:42:49.678272 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db4b4a79-ac43-488d-9bca-8a836ce83b13 None None] PATH_INFO: `/` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:42:49.679662 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 487/997] 10.222.0.25 () {54 vars in 927 bytes} [Sun Aug 7 10:42:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 07 10:42:49.685360 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-202b0d6b-57b1-460c-bbb6-7e8ac464b125 None None] REQUEST_METHOD: `POST` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:42:49.685710 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-202b0d6b-57b1-460c-bbb6-7e8ac464b125 None None] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:42:49.685995 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-202b0d6b-57b1-460c-bbb6-7e8ac464b125 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:42:49.705293 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.auth.core [None req-202b0d6b-57b1-460c-bbb6-7e8ac464b125 None None] MFA Rules not processed for user `bac4e6b76fe64217ac8fa44e71cffe6a`. Rule list: `[]` (Enabled: `True`). {{(pid=80144) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 10:42:49.706480 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.fernet_utils [None req-202b0d6b-57b1-460c-bbb6-7e8ac464b125 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80144) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 10:42:49.777982 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 511/998] 10.222.0.25 () {58 vars in 1024 bytes} [Sun Aug 7 10:42:49 2022] POST /identity/v3/auth/tokens => generated 1570 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 07 10:42:49.800583 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db4b4a79-ac43-488d-9bca-8a836ce83b13 None None] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 10:42:49.908686 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db4b4a79-ac43-488d-9bca-8a836ce83b13 None None] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 10:42:49.933405 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=100 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 10:42:49.946768 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bdbbda4-551a-47db-b167-2e7d4f45907e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 10:42:49.947549 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bdbbda4-551a-47db-b167-2e7d4f45907e None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:42:49.947835 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bdbbda4-551a-47db-b167-2e7d4f45907e None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:42:49.948112 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bdbbda4-551a-47db-b167-2e7d4f45907e None admin] PATH_INFO: `/v3/services/object-store` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:42:49.948693 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bdbbda4-551a-47db-b167-2e7d4f45907e None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 10:42:49.950052 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bdbbda4-551a-47db-b167-2e7d4f45907e None admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 10:42:49.958833 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application [None req-8bdbbda4-551a-47db-b167-2e7d4f45907e None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 07 10:42:49.958833 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 07 10:42:49.958833 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 07 10:42:49.958833 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 07 10:42:49.958833 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 07 10:42:49.958833 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 07 10:42:49.958833 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 07 10:42:49.958833 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 07 10:42:49.958833 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 07 10:42:49.958833 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application Aug 07 10:42:49.958833 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 07 10:42:49.958833 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application Aug 07 10:42:49.958833 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 07 10:42:49.958833 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 07 10:42:49.958833 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 07 10:42:49.958833 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 07 10:42:49.958833 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 07 10:42:49.959917 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 07 10:42:49.959917 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 07 10:42:49.959917 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 07 10:42:49.959917 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 07 10:42:49.959917 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 07 10:42:49.959917 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 07 10:42:49.959917 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 07 10:42:49.959917 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 07 10:42:49.959917 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 07 10:42:49.959917 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 07 10:42:49.959917 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 07 10:42:49.959917 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 07 10:42:49.959917 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 07 10:42:49.959917 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 07 10:42:49.959917 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 07 10:42:49.959917 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.get_or_create( Aug 07 10:42:49.960805 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 07 10:42:49.960805 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application with Lock( Aug 07 10:42:49.960805 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 07 10:42:49.960805 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self._enter() Aug 07 10:42:49.960805 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 07 10:42:49.960805 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 07 10:42:49.960805 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 07 10:42:49.960805 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.creator() Aug 07 10:42:49.960805 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 07 10:42:49.960805 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application created_value = creator( Aug 07 10:42:49.960805 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 07 10:42:49.960805 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 07 10:42:49.960805 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 07 10:42:49.960805 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application Aug 07 10:42:49.963233 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 488/999] 10.222.0.25 () {56 vars in 1126 bytes} [Sun Aug 7 10:42:49 2022] GET /identity/v3/services/object-store => generated 93 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 07 10:42:49.966222 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-202b0d6b-57b1-460c-bbb6-7e8ac464b125 None None] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 10:42:50.067944 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-202b0d6b-57b1-460c-bbb6-7e8ac464b125 None None] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 10:42:50.086598 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=100 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 10:42:50.105081 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f34e5fbe-767d-4284-bf66-43c115d13fda None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 10:42:50.105688 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f34e5fbe-767d-4284-bf66-43c115d13fda None admin] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:42:50.105965 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f34e5fbe-767d-4284-bf66-43c115d13fda None admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:42:50.106236 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f34e5fbe-767d-4284-bf66-43c115d13fda None admin] PATH_INFO: `/v3/services` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:42:50.106857 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f34e5fbe-767d-4284-bf66-43c115d13fda None admin] RBAC: Adding query filter params (name=object-store) {{(pid=80144) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 10:42:50.107187 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f34e5fbe-767d-4284-bf66-43c115d13fda None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 10:42:50.108519 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f34e5fbe-767d-4284-bf66-43c115d13fda None admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 10:42:50.118765 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 512/1000] 10.222.0.25 () {56 vars in 1122 bytes} [Sun Aug 7 10:42:49 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 10:42:50.119761 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bdbbda4-551a-47db-b167-2e7d4f45907e None admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 10:42:50.219975 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bdbbda4-551a-47db-b167-2e7d4f45907e None admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 10:42:50.254200 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-481c2ae3-6364-48fb-b072-d313c28eb3b0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 10:42:50.254811 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-481c2ae3-6364-48fb-b072-d313c28eb3b0 None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:42:50.255117 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-481c2ae3-6364-48fb-b072-d313c28eb3b0 None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:42:50.255385 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-481c2ae3-6364-48fb-b072-d313c28eb3b0 None admin] PATH_INFO: `/v3/services` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:42:50.256006 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-481c2ae3-6364-48fb-b072-d313c28eb3b0 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=80143) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 10:42:50.256308 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-481c2ae3-6364-48fb-b072-d313c28eb3b0 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 10:42:50.257652 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-481c2ae3-6364-48fb-b072-d313c28eb3b0 None admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 10:42:50.267813 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 489/1001] 10.222.0.25 () {56 vars in 1122 bytes} [Sun Aug 7 10:42:50 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 10:42:50.268809 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f34e5fbe-767d-4284-bf66-43c115d13fda None admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 10:42:50.370215 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f34e5fbe-767d-4284-bf66-43c115d13fda None admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 10:42:50.403666 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35c046be-613e-49c5-a6cf-4882b10f2733 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 10:42:50.404269 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35c046be-613e-49c5-a6cf-4882b10f2733 None admin] REQUEST_METHOD: `POST` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:42:50.404548 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35c046be-613e-49c5-a6cf-4882b10f2733 None admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:42:50.404820 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35c046be-613e-49c5-a6cf-4882b10f2733 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:42:50.405465 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35c046be-613e-49c5-a6cf-4882b10f2733 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 10:42:50.406806 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35c046be-613e-49c5-a6cf-4882b10f2733 None admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 10:42:50.428094 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 513/1002] 10.222.0.25 () {60 vars in 1144 bytes} [Sun Aug 7 10:42:50 2022] POST /identity/v3/endpoints => generated 325 bytes in 160 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 07 10:42:50.813044 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0c8bfa5-dbeb-43d8-b9c1-ffb84ce87491 None None] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:42:50.813560 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0c8bfa5-dbeb-43d8-b9c1-ffb84ce87491 None None] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:42:50.813638 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0c8bfa5-dbeb-43d8-b9c1-ffb84ce87491 None None] PATH_INFO: `/` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:42:50.814988 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 490/1003] 10.222.0.25 () {54 vars in 927 bytes} [Sun Aug 7 10:42:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 07 10:42:50.820099 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-082d374a-b6cd-4948-8eb0-fa6095156ec8 None None] REQUEST_METHOD: `POST` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:42:50.820473 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-082d374a-b6cd-4948-8eb0-fa6095156ec8 None None] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:42:50.820651 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-082d374a-b6cd-4948-8eb0-fa6095156ec8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:42:50.839199 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.auth.core [None req-082d374a-b6cd-4948-8eb0-fa6095156ec8 None None] MFA Rules not processed for user `bac4e6b76fe64217ac8fa44e71cffe6a`. Rule list: `[]` (Enabled: `True`). {{(pid=80144) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 10:42:50.840371 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.fernet_utils [None req-082d374a-b6cd-4948-8eb0-fa6095156ec8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80144) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 10:42:50.876067 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 10:42:50.916773 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 514/1004] 10.222.0.25 () {58 vars in 1024 bytes} [Sun Aug 7 10:42:50 2022] POST /identity/v3/auth/tokens => generated 1717 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 07 10:42:50.939131 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0c8bfa5-dbeb-43d8-b9c1-ffb84ce87491 None None] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 10:42:50.990252 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=100 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 10:42:51.044139 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0c8bfa5-dbeb-43d8-b9c1-ffb84ce87491 None None] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 10:42:51.078694 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66383dd2-848b-4dcb-91fa-67175db246cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 10:42:51.079274 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66383dd2-848b-4dcb-91fa-67175db246cf None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:42:51.079555 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66383dd2-848b-4dcb-91fa-67175db246cf None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:42:51.079825 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66383dd2-848b-4dcb-91fa-67175db246cf None admin] PATH_INFO: `/v3/services/image` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:42:51.080404 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66383dd2-848b-4dcb-91fa-67175db246cf None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 10:42:51.081771 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66383dd2-848b-4dcb-91fa-67175db246cf None admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 10:42:51.090832 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application [None req-66383dd2-848b-4dcb-91fa-67175db246cf None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Aug 07 10:42:51.090832 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 07 10:42:51.090832 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 07 10:42:51.090832 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 07 10:42:51.090832 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 07 10:42:51.090832 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 07 10:42:51.090832 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 07 10:42:51.090832 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 07 10:42:51.090832 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Aug 07 10:42:51.090832 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application Aug 07 10:42:51.090832 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 07 10:42:51.090832 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application Aug 07 10:42:51.090832 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 07 10:42:51.090832 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 07 10:42:51.090832 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 07 10:42:51.090832 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 07 10:42:51.090832 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 07 10:42:51.091864 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 07 10:42:51.091864 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 07 10:42:51.091864 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 07 10:42:51.091864 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 07 10:42:51.091864 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 07 10:42:51.091864 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 07 10:42:51.091864 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 07 10:42:51.091864 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 07 10:42:51.091864 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 07 10:42:51.091864 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 07 10:42:51.091864 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 07 10:42:51.091864 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 07 10:42:51.091864 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 07 10:42:51.091864 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 07 10:42:51.091864 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 07 10:42:51.091864 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.get_or_create( Aug 07 10:42:51.092697 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 07 10:42:51.092697 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application with Lock( Aug 07 10:42:51.092697 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 07 10:42:51.092697 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self._enter() Aug 07 10:42:51.092697 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 07 10:42:51.092697 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 07 10:42:51.092697 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 07 10:42:51.092697 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.creator() Aug 07 10:42:51.092697 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 07 10:42:51.092697 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application created_value = creator( Aug 07 10:42:51.092697 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 07 10:42:51.092697 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 07 10:42:51.092697 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Aug 07 10:42:51.092697 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application Aug 07 10:42:51.095005 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 491/1005] 10.222.0.25 () {56 vars in 1105 bytes} [Sun Aug 7 10:42:50 2022] GET /identity/v3/services/image => generated 86 bytes in 154 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 07 10:42:51.098209 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-082d374a-b6cd-4948-8eb0-fa6095156ec8 None None] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 10:42:51.199123 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-082d374a-b6cd-4948-8eb0-fa6095156ec8 None None] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 10:42:51.237421 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a85b91b5-b384-4481-af57-10c3c3706f57 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 10:42:51.238289 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a85b91b5-b384-4481-af57-10c3c3706f57 None admin] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:42:51.238567 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a85b91b5-b384-4481-af57-10c3c3706f57 None admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:42:51.238839 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a85b91b5-b384-4481-af57-10c3c3706f57 None admin] PATH_INFO: `/v3/services` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:42:51.239495 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a85b91b5-b384-4481-af57-10c3c3706f57 None admin] RBAC: Adding query filter params (name=image) {{(pid=80144) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 10:42:51.239800 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a85b91b5-b384-4481-af57-10c3c3706f57 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 10:42:51.241149 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a85b91b5-b384-4481-af57-10c3c3706f57 None admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 10:42:51.250641 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 515/1006] 10.222.0.25 () {56 vars in 1108 bytes} [Sun Aug 7 10:42:51 2022] GET /identity/v3/services?name=image => generated 122 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 10:42:51.251902 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66383dd2-848b-4dcb-91fa-67175db246cf None admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 10:42:51.356787 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66383dd2-848b-4dcb-91fa-67175db246cf None admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 10:42:51.370340 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=100 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 10:42:51.392537 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-384d9c65-c8b2-4079-a29e-7a5f23d32e8a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 10:42:51.393200 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-384d9c65-c8b2-4079-a29e-7a5f23d32e8a None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:42:51.393483 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-384d9c65-c8b2-4079-a29e-7a5f23d32e8a None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:42:51.393752 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-384d9c65-c8b2-4079-a29e-7a5f23d32e8a None admin] PATH_INFO: `/v3/services` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:42:51.394452 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-384d9c65-c8b2-4079-a29e-7a5f23d32e8a None admin] RBAC: Adding query filter params (type=image) {{(pid=80143) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 10:42:51.394754 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-384d9c65-c8b2-4079-a29e-7a5f23d32e8a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 10:42:51.396132 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-384d9c65-c8b2-4079-a29e-7a5f23d32e8a None admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 10:42:51.401221 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 492/1007] 10.222.0.25 () {56 vars in 1108 bytes} [Sun Aug 7 10:42:51 2022] GET /identity/v3/services?type=image => generated 122 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 10:42:51.574415 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-425366aa-f302-404c-93c2-8fc2fbfad5e0 None None] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:42:51.574702 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-425366aa-f302-404c-93c2-8fc2fbfad5e0 None None] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:42:51.575017 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-425366aa-f302-404c-93c2-8fc2fbfad5e0 None None] PATH_INFO: `/` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:42:51.576371 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 516/1008] 10.222.0.25 () {54 vars in 927 bytes} [Sun Aug 7 10:42:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 07 10:42:51.581772 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82f201b9-ad20-4c8f-a729-a7d9cc2fb64f None None] REQUEST_METHOD: `POST` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:42:51.582063 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82f201b9-ad20-4c8f-a729-a7d9cc2fb64f None None] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:42:51.582410 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82f201b9-ad20-4c8f-a729-a7d9cc2fb64f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:42:51.601228 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.auth.core [None req-82f201b9-ad20-4c8f-a729-a7d9cc2fb64f None None] MFA Rules not processed for user `bac4e6b76fe64217ac8fa44e71cffe6a`. Rule list: `[]` (Enabled: `True`). {{(pid=80143) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 10:42:51.602502 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-82f201b9-ad20-4c8f-a729-a7d9cc2fb64f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 10:42:51.671345 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 493/1009] 10.222.0.25 () {58 vars in 1024 bytes} [Sun Aug 7 10:42:51 2022] POST /identity/v3/auth/tokens => generated 1717 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 07 10:42:51.693471 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-425366aa-f302-404c-93c2-8fc2fbfad5e0 None None] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 10:42:51.725404 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=100 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 10:42:51.799342 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-425366aa-f302-404c-93c2-8fc2fbfad5e0 None None] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 10:42:51.832582 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-948fbfdd-7bad-4278-90f0-326b742d401e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 10:42:51.833225 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-948fbfdd-7bad-4278-90f0-326b742d401e None admin] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:42:51.833477 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-948fbfdd-7bad-4278-90f0-326b742d401e None admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:42:51.833742 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-948fbfdd-7bad-4278-90f0-326b742d401e None admin] PATH_INFO: `/v3/services/compute` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:42:51.834374 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-948fbfdd-7bad-4278-90f0-326b742d401e None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 10:42:51.835768 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-948fbfdd-7bad-4278-90f0-326b742d401e None admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 10:42:51.845049 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application [None req-948fbfdd-7bad-4278-90f0-326b742d401e None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Aug 07 10:42:51.845049 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 07 10:42:51.845049 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 07 10:42:51.845049 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 07 10:42:51.845049 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 07 10:42:51.845049 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 07 10:42:51.845049 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 07 10:42:51.845049 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 07 10:42:51.845049 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Aug 07 10:42:51.845049 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application Aug 07 10:42:51.845049 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 07 10:42:51.845049 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application Aug 07 10:42:51.845049 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 07 10:42:51.845049 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 07 10:42:51.845049 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 07 10:42:51.845049 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 07 10:42:51.845049 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 07 10:42:51.846011 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 07 10:42:51.846011 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 07 10:42:51.846011 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 07 10:42:51.846011 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 07 10:42:51.846011 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 07 10:42:51.846011 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 07 10:42:51.846011 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 07 10:42:51.846011 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 07 10:42:51.846011 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 07 10:42:51.846011 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 07 10:42:51.846011 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 07 10:42:51.846011 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 07 10:42:51.846011 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 07 10:42:51.846011 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 07 10:42:51.846011 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 07 10:42:51.846011 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self.get_or_create( Aug 07 10:42:51.846876 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 07 10:42:51.846876 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application with Lock( Aug 07 10:42:51.846876 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 07 10:42:51.846876 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self._enter() Aug 07 10:42:51.846876 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 07 10:42:51.846876 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 07 10:42:51.846876 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 07 10:42:51.846876 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self.creator() Aug 07 10:42:51.846876 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 07 10:42:51.846876 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application created_value = creator( Aug 07 10:42:51.846876 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 07 10:42:51.846876 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 07 10:42:51.846876 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Aug 07 10:42:51.846876 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application Aug 07 10:42:51.846876 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 517/1010] 10.222.0.25 () {56 vars in 1111 bytes} [Sun Aug 7 10:42:51 2022] GET /identity/v3/services/compute => generated 88 bytes in 153 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 07 10:42:51.854145 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82f201b9-ad20-4c8f-a729-a7d9cc2fb64f None None] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 10:42:51.954466 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82f201b9-ad20-4c8f-a729-a7d9cc2fb64f None None] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 10:42:51.964111 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=100 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 10:42:51.990156 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36939a35-abba-42d0-9bfb-e068e7072ad2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 10:42:51.990761 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36939a35-abba-42d0-9bfb-e068e7072ad2 None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:42:51.991067 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36939a35-abba-42d0-9bfb-e068e7072ad2 None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:42:51.991339 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36939a35-abba-42d0-9bfb-e068e7072ad2 None admin] PATH_INFO: `/v3/services` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:42:51.991959 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36939a35-abba-42d0-9bfb-e068e7072ad2 None admin] RBAC: Adding query filter params (name=compute) {{(pid=80143) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 10:42:51.992263 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36939a35-abba-42d0-9bfb-e068e7072ad2 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 10:42:51.994176 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36939a35-abba-42d0-9bfb-e068e7072ad2 None admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 10:42:52.003441 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 494/1011] 10.222.0.25 () {56 vars in 1112 bytes} [Sun Aug 7 10:42:51 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 10:42:52.004616 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-948fbfdd-7bad-4278-90f0-326b742d401e None admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 10:42:52.098044 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=100 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 10:42:52.106803 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-948fbfdd-7bad-4278-90f0-326b742d401e None admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 10:42:52.141995 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d499d73-b3ec-490e-9bf3-6aea62278cac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 10:42:52.142611 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d499d73-b3ec-490e-9bf3-6aea62278cac None admin] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:42:52.142897 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d499d73-b3ec-490e-9bf3-6aea62278cac None admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:42:52.143204 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d499d73-b3ec-490e-9bf3-6aea62278cac None admin] PATH_INFO: `/v3/services` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:42:52.143835 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d499d73-b3ec-490e-9bf3-6aea62278cac None admin] RBAC: Adding query filter params (type=compute) {{(pid=80144) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 10:42:52.144141 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d499d73-b3ec-490e-9bf3-6aea62278cac None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 10:42:52.145489 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d499d73-b3ec-490e-9bf3-6aea62278cac None admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 10:42:52.155734 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 518/1012] 10.222.0.25 () {56 vars in 1112 bytes} [Sun Aug 7 10:42:52 2022] GET /identity/v3/services?type=compute => generated 352 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 07 10:42:52.157300 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36939a35-abba-42d0-9bfb-e068e7072ad2 None admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 10:42:52.257361 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36939a35-abba-42d0-9bfb-e068e7072ad2 None admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 10:42:52.291155 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc6c99fa-8e21-4cca-9b06-0f559b379f54 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 10:42:52.291758 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc6c99fa-8e21-4cca-9b06-0f559b379f54 None admin] REQUEST_METHOD: `POST` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:42:52.292035 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc6c99fa-8e21-4cca-9b06-0f559b379f54 None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:42:52.292302 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc6c99fa-8e21-4cca-9b06-0f559b379f54 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:42:52.292947 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc6c99fa-8e21-4cca-9b06-0f559b379f54 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 10:42:52.294299 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc6c99fa-8e21-4cca-9b06-0f559b379f54 None admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 10:42:52.319977 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 495/1013] 10.222.0.25 () {60 vars in 1144 bytes} [Sun Aug 7 10:42:52 2022] POST /identity/v3/endpoints => generated 334 bytes in 157 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 07 10:42:52.480701 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b9977a9-4ee8-46d3-a698-555d6597aec5 None None] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:42:52.480985 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b9977a9-4ee8-46d3-a698-555d6597aec5 None None] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:42:52.481268 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b9977a9-4ee8-46d3-a698-555d6597aec5 None None] PATH_INFO: `/` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:42:52.482680 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 519/1014] 10.222.0.25 () {54 vars in 927 bytes} [Sun Aug 7 10:42:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 07 10:42:52.488101 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88748b79-8b08-46d3-a887-5ee6bc9cfd02 None None] REQUEST_METHOD: `POST` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:42:52.488389 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88748b79-8b08-46d3-a887-5ee6bc9cfd02 None None] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:42:52.488660 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88748b79-8b08-46d3-a887-5ee6bc9cfd02 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:42:52.509232 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.auth.core [None req-88748b79-8b08-46d3-a887-5ee6bc9cfd02 None None] MFA Rules not processed for user `bac4e6b76fe64217ac8fa44e71cffe6a`. Rule list: `[]` (Enabled: `True`). {{(pid=80143) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 10:42:52.510440 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-88748b79-8b08-46d3-a887-5ee6bc9cfd02 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 10:42:52.538435 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 10:42:52.594082 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 496/1015] 10.222.0.25 () {58 vars in 1024 bytes} [Sun Aug 7 10:42:52 2022] POST /identity/v3/auth/tokens => generated 1873 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 07 10:42:52.617315 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b9977a9-4ee8-46d3-a698-555d6597aec5 None None] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 10:42:52.724065 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b9977a9-4ee8-46d3-a698-555d6597aec5 None None] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 10:42:52.758282 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0917e3d2-b716-4c9f-9ae0-3a15678d0f61 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 10:42:52.759637 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0917e3d2-b716-4c9f-9ae0-3a15678d0f61 None admin] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:42:52.759745 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0917e3d2-b716-4c9f-9ae0-3a15678d0f61 None admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:42:52.759994 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0917e3d2-b716-4c9f-9ae0-3a15678d0f61 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:42:52.761208 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0917e3d2-b716-4c9f-9ae0-3a15678d0f61 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 10:42:52.762619 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0917e3d2-b716-4c9f-9ae0-3a15678d0f61 None admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 10:42:52.768853 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 520/1016] 10.222.0.25 () {56 vars in 1108 bytes} [Sun Aug 7 10:42:52 2022] GET /identity/v3/domains/default => generated 199 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 10:42:52.770337 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88748b79-8b08-46d3-a887-5ee6bc9cfd02 None None] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 10:42:52.871462 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88748b79-8b08-46d3-a887-5ee6bc9cfd02 None None] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 10:42:52.906470 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fc417d7-52df-4de8-9678-1b6d15524e27 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 10:42:52.907118 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fc417d7-52df-4de8-9678-1b6d15524e27 None admin] REQUEST_METHOD: `POST` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:42:52.907405 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fc417d7-52df-4de8-9678-1b6d15524e27 None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:42:52.907682 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fc417d7-52df-4de8-9678-1b6d15524e27 None admin] PATH_INFO: `/v3/projects` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:42:52.908339 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fc417d7-52df-4de8-9678-1b6d15524e27 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 10:42:52.909779 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fc417d7-52df-4de8-9678-1b6d15524e27 None admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 10:42:52.942966 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 497/1017] 10.222.0.25 () {60 vars in 1141 bytes} [Sun Aug 7 10:42:52 2022] POST /identity/v3/projects => generated 311 bytes in 169 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 07 10:42:53.441738 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26989406-e1ff-4612-b50a-e337c749b85d None None] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:42:53.442230 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26989406-e1ff-4612-b50a-e337c749b85d None None] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:42:53.442335 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26989406-e1ff-4612-b50a-e337c749b85d None None] PATH_INFO: `/` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:42:53.443834 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 521/1018] 10.222.0.25 () {54 vars in 927 bytes} [Sun Aug 7 10:42:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 07 10:42:53.448892 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed6cb686-36f1-4faf-ac47-aa5ab9e15186 None None] REQUEST_METHOD: `POST` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:42:53.449188 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed6cb686-36f1-4faf-ac47-aa5ab9e15186 None None] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:42:53.449470 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed6cb686-36f1-4faf-ac47-aa5ab9e15186 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:42:53.462359 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 10:42:53.468644 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.auth.core [None req-ed6cb686-36f1-4faf-ac47-aa5ab9e15186 None None] MFA Rules not processed for user `bac4e6b76fe64217ac8fa44e71cffe6a`. Rule list: `[]` (Enabled: `True`). {{(pid=80143) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 10:42:53.469833 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-ed6cb686-36f1-4faf-ac47-aa5ab9e15186 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 10:42:53.542735 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 498/1019] 10.222.0.25 () {58 vars in 1024 bytes} [Sun Aug 7 10:42:53 2022] POST /identity/v3/auth/tokens => generated 1873 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 07 10:42:53.564858 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26989406-e1ff-4612-b50a-e337c749b85d None None] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 10:42:53.592475 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=100 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 10:42:53.668703 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26989406-e1ff-4612-b50a-e337c749b85d None None] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 10:42:53.702733 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d529b16-41ca-4426-9af9-6f38b2674735 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 10:42:53.703289 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d529b16-41ca-4426-9af9-6f38b2674735 None admin] REQUEST_METHOD: `POST` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:42:53.703567 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d529b16-41ca-4426-9af9-6f38b2674735 None admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:42:53.703837 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d529b16-41ca-4426-9af9-6f38b2674735 None admin] PATH_INFO: `/v3/services` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:42:53.704478 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d529b16-41ca-4426-9af9-6f38b2674735 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 10:42:53.705838 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d529b16-41ca-4426-9af9-6f38b2674735 None admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 10:42:53.722192 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 522/1020] 10.222.0.25 () {60 vars in 1141 bytes} [Sun Aug 7 10:42:53 2022] POST /identity/v3/services => generated 242 bytes in 158 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 07 10:42:54.467910 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bbcfa00-478f-4df5-84a9-de9d0af79e4b None None] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:42:54.468451 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bbcfa00-478f-4df5-84a9-de9d0af79e4b None None] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:42:54.468451 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bbcfa00-478f-4df5-84a9-de9d0af79e4b None None] PATH_INFO: `/` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:42:54.469926 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 499/1021] 10.222.0.25 () {54 vars in 927 bytes} [Sun Aug 7 10:42:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 07 10:42:54.474997 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ba3ff9d-e5a4-4ea0-8de4-cc6779883743 None None] REQUEST_METHOD: `POST` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:42:54.475391 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ba3ff9d-e5a4-4ea0-8de4-cc6779883743 None None] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:42:54.475551 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ba3ff9d-e5a4-4ea0-8de4-cc6779883743 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:42:54.495265 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.auth.core [None req-6ba3ff9d-e5a4-4ea0-8de4-cc6779883743 None None] MFA Rules not processed for user `bac4e6b76fe64217ac8fa44e71cffe6a`. Rule list: `[]` (Enabled: `True`). {{(pid=80144) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 10:42:54.496469 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.fernet_utils [None req-6ba3ff9d-e5a4-4ea0-8de4-cc6779883743 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80144) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 10:42:54.571260 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 523/1022] 10.222.0.25 () {58 vars in 1024 bytes} [Sun Aug 7 10:42:54 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 07 10:42:54.594852 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bbcfa00-478f-4df5-84a9-de9d0af79e4b None None] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 10:42:54.697201 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bbcfa00-478f-4df5-84a9-de9d0af79e4b None None] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 10:42:54.732230 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13a9e71f-bd06-41a5-9ae5-f02a6c357a7a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 10:42:54.732865 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13a9e71f-bd06-41a5-9ae5-f02a6c357a7a None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:42:54.733156 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13a9e71f-bd06-41a5-9ae5-f02a6c357a7a None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:42:54.733433 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13a9e71f-bd06-41a5-9ae5-f02a6c357a7a None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:42:54.736122 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=100 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 10:42:54.745595 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13a9e71f-bd06-41a5-9ae5-f02a6c357a7a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 07 10:42:54.750318 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application [None req-13a9e71f-bd06-41a5-9ae5-f02a6c357a7a None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 07 10:42:54.750318 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 07 10:42:54.750318 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 07 10:42:54.750318 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 07 10:42:54.750318 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 07 10:42:54.750318 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 07 10:42:54.750318 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 07 10:42:54.750318 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 07 10:42:54.750318 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 07 10:42:54.750318 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 07 10:42:54.750318 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 07 10:42:54.750318 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 07 10:42:54.750318 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 07 10:42:54.750318 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 07 10:42:54.750318 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 07 10:42:54.750318 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 07 10:42:54.751245 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 07 10:42:54.751245 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 07 10:42:54.751245 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 07 10:42:54.751245 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 07 10:42:54.751245 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 07 10:42:54.751245 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 07 10:42:54.751245 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 07 10:42:54.751245 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 07 10:42:54.751245 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 07 10:42:54.751245 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.get_or_create( Aug 07 10:42:54.751245 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 07 10:42:54.751245 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application with Lock( Aug 07 10:42:54.751245 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 07 10:42:54.751245 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self._enter() Aug 07 10:42:54.751245 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 07 10:42:54.751245 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 07 10:42:54.752100 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 07 10:42:54.752100 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.creator() Aug 07 10:42:54.752100 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 07 10:42:54.752100 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application created_value = creator( Aug 07 10:42:54.752100 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 07 10:42:54.752100 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 07 10:42:54.752100 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 07 10:42:54.752100 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 07 10:42:54.752100 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 07 10:42:54.752100 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 07 10:42:54.752100 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 07 10:42:54.752100 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application Aug 07 10:42:54.752100 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 500/1023] 10.222.0.25 () {56 vars in 1120 bytes} [Sun Aug 7 10:42:54 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 07 10:42:54.758980 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ba3ff9d-e5a4-4ea0-8de4-cc6779883743 None None] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 10:42:54.772298 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 10:42:54.861584 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ba3ff9d-e5a4-4ea0-8de4-cc6779883743 None None] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 10:42:54.898423 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19f67267-c4ab-4287-b8bb-9bf27ea075e4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 10:42:54.899096 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19f67267-c4ab-4287-b8bb-9bf27ea075e4 None admin] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:42:54.899377 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19f67267-c4ab-4287-b8bb-9bf27ea075e4 None admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:42:54.899656 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19f67267-c4ab-4287-b8bb-9bf27ea075e4 None admin] PATH_INFO: `/v3/roles` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:42:54.900324 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19f67267-c4ab-4287-b8bb-9bf27ea075e4 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80144) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 10:42:54.900633 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19f67267-c4ab-4287-b8bb-9bf27ea075e4 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 10:42:54.901986 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19f67267-c4ab-4287-b8bb-9bf27ea075e4 None admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 10:42:54.915332 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 524/1024] 10.222.0.25 () {56 vars in 1115 bytes} [Sun Aug 7 10:42:54 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 07 10:42:54.916631 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13a9e71f-bd06-41a5-9ae5-f02a6c357a7a None admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 10:42:54.988104 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25e5b6ed-d9d1-42ff-855e-f21479cba83d None None] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:42:54.988348 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25e5b6ed-d9d1-42ff-855e-f21479cba83d None None] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:42:54.988582 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25e5b6ed-d9d1-42ff-855e-f21479cba83d None None] PATH_INFO: `/` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:42:54.989924 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 525/1025] 10.222.0.25 () {54 vars in 927 bytes} [Sun Aug 7 10:42:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 07 10:42:54.996289 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2842d71-5e50-4427-9b52-750688d136f8 None None] REQUEST_METHOD: `POST` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:42:54.996648 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2842d71-5e50-4427-9b52-750688d136f8 None None] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:42:54.996999 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2842d71-5e50-4427-9b52-750688d136f8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:42:55.015133 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.auth.core [None req-e2842d71-5e50-4427-9b52-750688d136f8 None None] MFA Rules not processed for user `bac4e6b76fe64217ac8fa44e71cffe6a`. Rule list: `[]` (Enabled: `True`). {{(pid=80144) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 10:42:55.016308 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.fernet_utils [None req-e2842d71-5e50-4427-9b52-750688d136f8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80144) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 10:42:55.024402 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13a9e71f-bd06-41a5-9ae5-f02a6c357a7a None admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 10:42:55.060457 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7899f296-ff32-4703-9c9f-1a38b82d614b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 10:42:55.061096 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7899f296-ff32-4703-9c9f-1a38b82d614b None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:42:55.061376 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7899f296-ff32-4703-9c9f-1a38b82d614b None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:42:55.061650 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7899f296-ff32-4703-9c9f-1a38b82d614b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:42:55.072990 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7899f296-ff32-4703-9c9f-1a38b82d614b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 10:42:55.074434 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7899f296-ff32-4703-9c9f-1a38b82d614b None admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 10:42:55.089549 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application [None req-7899f296-ff32-4703-9c9f-1a38b82d614b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 07 10:42:55.089549 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 07 10:42:55.089549 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 07 10:42:55.089549 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 07 10:42:55.089549 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 07 10:42:55.089549 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 07 10:42:55.089549 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 07 10:42:55.089549 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 07 10:42:55.089549 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 07 10:42:55.089549 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 07 10:42:55.089549 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 07 10:42:55.089549 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 07 10:42:55.089549 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 07 10:42:55.089549 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 07 10:42:55.089549 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 07 10:42:55.089549 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 07 10:42:55.090514 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 07 10:42:55.090514 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 07 10:42:55.090514 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 07 10:42:55.090514 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 07 10:42:55.090514 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 07 10:42:55.090514 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.get_or_create( Aug 07 10:42:55.090514 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 07 10:42:55.090514 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application with Lock( Aug 07 10:42:55.090514 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 07 10:42:55.090514 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self._enter() Aug 07 10:42:55.090514 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 07 10:42:55.090514 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 07 10:42:55.090514 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 07 10:42:55.090514 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.creator() Aug 07 10:42:55.090514 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 07 10:42:55.090514 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application created_value = creator( Aug 07 10:42:55.090514 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 07 10:42:55.091848 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 07 10:42:55.091848 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 07 10:42:55.091848 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application Aug 07 10:42:55.091848 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 501/1026] 10.222.0.25 () {56 vars in 1108 bytes} [Sun Aug 7 10:42:54 2022] GET /identity/v3/domains/Default => generated 87 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 07 10:42:55.098469 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 526/1027] 10.222.0.25 () {58 vars in 1024 bytes} [Sun Aug 7 10:42:54 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 07 10:42:55.098597 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7899f296-ff32-4703-9c9f-1a38b82d614b None admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 10:42:55.119280 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2842d71-5e50-4427-9b52-750688d136f8 None None] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 10:42:55.123637 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=100 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 10:42:55.150663 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=100 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 10:42:55.209622 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7899f296-ff32-4703-9c9f-1a38b82d614b None admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 10:42:55.225802 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2842d71-5e50-4427-9b52-750688d136f8 None None] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 10:42:55.246406 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90307205-dc41-4f51-a143-3c747f13396c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 10:42:55.247102 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90307205-dc41-4f51-a143-3c747f13396c None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:42:55.247385 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90307205-dc41-4f51-a143-3c747f13396c None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:42:55.247661 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90307205-dc41-4f51-a143-3c747f13396c None admin] PATH_INFO: `/v3/domains` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:42:55.248305 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90307205-dc41-4f51-a143-3c747f13396c None admin] RBAC: Adding query filter params (name=Default) {{(pid=80143) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 10:42:55.248611 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90307205-dc41-4f51-a143-3c747f13396c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 10:42:55.250185 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90307205-dc41-4f51-a143-3c747f13396c None admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 10:42:55.260420 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf595dfc-829a-4335-a599-4cf85baadbc9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 10:42:55.261078 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf595dfc-829a-4335-a599-4cf85baadbc9 None admin] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:42:55.261360 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf595dfc-829a-4335-a599-4cf85baadbc9 None admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:42:55.261631 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf595dfc-829a-4335-a599-4cf85baadbc9 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:42:55.262740 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf595dfc-829a-4335-a599-4cf85baadbc9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 10:42:55.264085 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 502/1028] 10.222.0.25 () {56 vars in 1109 bytes} [Sun Aug 7 10:42:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 10:42:55.264218 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf595dfc-829a-4335-a599-4cf85baadbc9 None admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 10:42:55.266970 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 527/1029] 10.222.0.25 () {56 vars in 1108 bytes} [Sun Aug 7 10:42:55 2022] GET /identity/v3/domains/default => generated 199 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 07 10:42:55.271968 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90307205-dc41-4f51-a143-3c747f13396c None admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 10:42:55.272929 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf595dfc-829a-4335-a599-4cf85baadbc9 None admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 10:42:55.346307 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=100 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 10:42:55.380245 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf595dfc-829a-4335-a599-4cf85baadbc9 None admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 10:42:55.381136 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90307205-dc41-4f51-a143-3c747f13396c None admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 10:42:55.392525 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=100 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 10:42:55.423513 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1326b94-dec8-473b-bd50-79ba26a6d951 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 10:42:55.424118 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1326b94-dec8-473b-bd50-79ba26a6d951 None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:42:55.424244 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cca6e11-4c53-41a2-a2eb-367e4d2b4760 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 10:42:55.424441 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1326b94-dec8-473b-bd50-79ba26a6d951 None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:42:55.424671 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1326b94-dec8-473b-bd50-79ba26a6d951 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:42:55.424767 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cca6e11-4c53-41a2-a2eb-367e4d2b4760 None admin] REQUEST_METHOD: `POST` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:42:55.425043 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cca6e11-4c53-41a2-a2eb-367e4d2b4760 None admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:42:55.425319 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cca6e11-4c53-41a2-a2eb-367e4d2b4760 None admin] PATH_INFO: `/v3/users` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:42:55.425963 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cca6e11-4c53-41a2-a2eb-367e4d2b4760 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 10:42:55.428117 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cca6e11-4c53-41a2-a2eb-367e4d2b4760 None admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 10:42:55.432573 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1326b94-dec8-473b-bd50-79ba26a6d951 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 10:42:55.434001 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1326b94-dec8-473b-bd50-79ba26a6d951 None admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 10:42:55.447437 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application [None req-b1326b94-dec8-473b-bd50-79ba26a6d951 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Aug 07 10:42:55.447437 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 07 10:42:55.447437 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 07 10:42:55.447437 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 07 10:42:55.447437 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 07 10:42:55.447437 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 07 10:42:55.447437 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 07 10:42:55.447437 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 07 10:42:55.447437 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 07 10:42:55.447437 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 07 10:42:55.447437 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 07 10:42:55.447437 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 07 10:42:55.447437 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 07 10:42:55.447437 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 07 10:42:55.447437 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 07 10:42:55.447437 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 07 10:42:55.448448 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 07 10:42:55.448448 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 07 10:42:55.448448 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 07 10:42:55.448448 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 07 10:42:55.448448 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 07 10:42:55.448448 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 07 10:42:55.448448 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 07 10:42:55.448448 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 07 10:42:55.448448 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 07 10:42:55.448448 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.get_or_create( Aug 07 10:42:55.448448 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 07 10:42:55.448448 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application with Lock( Aug 07 10:42:55.448448 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 07 10:42:55.448448 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self._enter() Aug 07 10:42:55.448448 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 07 10:42:55.448448 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 07 10:42:55.448448 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 07 10:42:55.448448 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.creator() Aug 07 10:42:55.449398 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 07 10:42:55.449398 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application created_value = creator( Aug 07 10:42:55.449398 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 07 10:42:55.449398 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 07 10:42:55.449398 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 07 10:42:55.449398 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 07 10:42:55.449398 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 07 10:42:55.449398 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 07 10:42:55.449398 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Aug 07 10:42:55.449398 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application Aug 07 10:42:55.449398 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 503/1030] 10.222.0.25 () {56 vars in 1093 bytes} [Sun Aug 7 10:42:55 2022] GET /identity/v3/users/nova => generated 82 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 07 10:42:55.454166 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1326b94-dec8-473b-bd50-79ba26a6d951 None admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 10:42:55.462957 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 528/1031] 10.222.0.25 () {60 vars in 1132 bytes} [Sun Aug 7 10:42:55 2022] POST /identity/v3/users => generated 285 bytes in 182 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 07 10:42:55.568357 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1326b94-dec8-473b-bd50-79ba26a6d951 None admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 10:42:55.607812 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01e04756-d3c5-4124-9233-74282aab478b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 10:42:55.608435 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01e04756-d3c5-4124-9233-74282aab478b None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:42:55.608718 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01e04756-d3c5-4124-9233-74282aab478b None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:42:55.608993 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01e04756-d3c5-4124-9233-74282aab478b None admin] PATH_INFO: `/v3/users` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:42:55.609878 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01e04756-d3c5-4124-9233-74282aab478b None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80143) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 10:42:55.610055 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01e04756-d3c5-4124-9233-74282aab478b None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 10:42:55.612012 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01e04756-d3c5-4124-9233-74282aab478b None admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 10:42:55.630835 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 504/1032] 10.222.0.25 () {56 vars in 1133 bytes} [Sun Aug 7 10:42:55 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 07 10:42:55.637550 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cca6e11-4c53-41a2-a2eb-367e4d2b4760 None admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 10:42:55.748376 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cca6e11-4c53-41a2-a2eb-367e4d2b4760 None admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 10:42:55.773692 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5156d0b5-4804-45f5-8554-bc99760fa7d3 None None] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:42:55.774074 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5156d0b5-4804-45f5-8554-bc99760fa7d3 None None] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:42:55.774302 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5156d0b5-4804-45f5-8554-bc99760fa7d3 None None] PATH_INFO: `/` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:42:55.774558 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 10:42:55.775733 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 505/1033] 10.222.0.25 () {54 vars in 927 bytes} [Sun Aug 7 10:42:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 07 10:42:55.781311 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78096244-775c-4b9a-a597-53ca45731b54 None None] REQUEST_METHOD: `POST` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:42:55.781709 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78096244-775c-4b9a-a597-53ca45731b54 None None] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:42:55.782085 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78096244-775c-4b9a-a597-53ca45731b54 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:42:55.787515 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14686363-7a42-4573-86b3-379cc87c7845 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 10:42:55.788162 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14686363-7a42-4573-86b3-379cc87c7845 None admin] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:42:55.788463 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14686363-7a42-4573-86b3-379cc87c7845 None admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:42:55.788723 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14686363-7a42-4573-86b3-379cc87c7845 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:42:55.800478 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.auth.core [None req-78096244-775c-4b9a-a597-53ca45731b54 None None] MFA Rules not processed for user `bac4e6b76fe64217ac8fa44e71cffe6a`. Rule list: `[]` (Enabled: `True`). {{(pid=80143) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 10:42:55.801653 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-78096244-775c-4b9a-a597-53ca45731b54 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 10:42:55.822724 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=100 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 10:42:55.862455 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14686363-7a42-4573-86b3-379cc87c7845 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 10:42:55.864189 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14686363-7a42-4573-86b3-379cc87c7845 None admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 10:42:55.871855 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 506/1034] 10.222.0.25 () {58 vars in 1024 bytes} [Sun Aug 7 10:42:55 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 07 10:42:55.894595 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78096244-775c-4b9a-a597-53ca45731b54 None None] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 10:42:55.972577 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 529/1035] 10.222.0.25 () {58 vars in 1282 bytes} [Sun Aug 7 10:42:55 2022] GET /identity/v3/auth/tokens => generated 1969 bytes in 335 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 07 10:42:55.973704 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14686363-7a42-4573-86b3-379cc87c7845 None admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 10:42:56.003626 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=100 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 10:42:56.005338 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78096244-775c-4b9a-a597-53ca45731b54 None None] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 10:42:56.042613 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab7c51b5-b404-42be-b787-441e75879019 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 10:42:56.043523 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab7c51b5-b404-42be-b787-441e75879019 None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:42:56.043806 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab7c51b5-b404-42be-b787-441e75879019 None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:42:56.044082 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab7c51b5-b404-42be-b787-441e75879019 None admin] PATH_INFO: `/v3/services/image` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:42:56.044666 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab7c51b5-b404-42be-b787-441e75879019 None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 10:42:56.046050 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab7c51b5-b404-42be-b787-441e75879019 None admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 10:42:56.054826 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application [None req-ab7c51b5-b404-42be-b787-441e75879019 None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Aug 07 10:42:56.054826 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 07 10:42:56.054826 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 07 10:42:56.054826 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 07 10:42:56.054826 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 07 10:42:56.054826 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 07 10:42:56.054826 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 07 10:42:56.054826 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 07 10:42:56.054826 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Aug 07 10:42:56.054826 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application Aug 07 10:42:56.054826 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 07 10:42:56.054826 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application Aug 07 10:42:56.054826 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 07 10:42:56.054826 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 07 10:42:56.054826 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 07 10:42:56.054826 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 07 10:42:56.054826 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 07 10:42:56.055882 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 07 10:42:56.055882 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 07 10:42:56.055882 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 07 10:42:56.055882 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 07 10:42:56.055882 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 07 10:42:56.055882 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 07 10:42:56.055882 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 07 10:42:56.055882 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 07 10:42:56.055882 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 07 10:42:56.055882 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 07 10:42:56.055882 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 07 10:42:56.055882 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 07 10:42:56.055882 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 07 10:42:56.055882 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 07 10:42:56.055882 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 07 10:42:56.055882 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.get_or_create( Aug 07 10:42:56.056702 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 07 10:42:56.056702 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application with Lock( Aug 07 10:42:56.056702 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 07 10:42:56.056702 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self._enter() Aug 07 10:42:56.056702 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 07 10:42:56.056702 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 07 10:42:56.056702 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 07 10:42:56.056702 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.creator() Aug 07 10:42:56.056702 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 07 10:42:56.056702 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application created_value = creator( Aug 07 10:42:56.056702 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 07 10:42:56.056702 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 07 10:42:56.056702 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Aug 07 10:42:56.056702 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application Aug 07 10:42:56.056702 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 507/1036] 10.222.0.25 () {56 vars in 1105 bytes} [Sun Aug 7 10:42:55 2022] GET /identity/v3/services/image => generated 86 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 07 10:42:56.062063 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab7c51b5-b404-42be-b787-441e75879019 None admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 10:42:56.080087 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14686363-7a42-4573-86b3-379cc87c7845 None admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 10:42:56.083783 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=100 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 10:42:56.120321 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e1d1480-d868-4b1c-873e-eb7934f22322 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 10:42:56.120952 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e1d1480-d868-4b1c-873e-eb7934f22322 None admin] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:42:56.121328 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e1d1480-d868-4b1c-873e-eb7934f22322 None admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:42:56.121500 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e1d1480-d868-4b1c-873e-eb7934f22322 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:42:56.131563 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e1d1480-d868-4b1c-873e-eb7934f22322 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 10:42:56.133682 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e1d1480-d868-4b1c-873e-eb7934f22322 None admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 10:42:56.148109 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application [None req-6e1d1480-d868-4b1c-873e-eb7934f22322 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 07 10:42:56.148109 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 07 10:42:56.148109 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 07 10:42:56.148109 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 07 10:42:56.148109 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 07 10:42:56.148109 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 07 10:42:56.148109 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 07 10:42:56.148109 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 07 10:42:56.148109 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 07 10:42:56.148109 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 07 10:42:56.148109 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 07 10:42:56.148109 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 07 10:42:56.148109 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 07 10:42:56.148109 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 07 10:42:56.148109 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 07 10:42:56.148109 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 07 10:42:56.149612 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 07 10:42:56.149612 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 07 10:42:56.149612 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 07 10:42:56.149612 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 07 10:42:56.149612 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 07 10:42:56.149612 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self.get_or_create( Aug 07 10:42:56.149612 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 07 10:42:56.149612 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application with Lock( Aug 07 10:42:56.149612 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 07 10:42:56.149612 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self._enter() Aug 07 10:42:56.149612 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 07 10:42:56.149612 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 07 10:42:56.149612 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 07 10:42:56.149612 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self.creator() Aug 07 10:42:56.149612 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 07 10:42:56.149612 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application created_value = creator( Aug 07 10:42:56.149612 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 07 10:42:56.150733 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 07 10:42:56.150733 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 07 10:42:56.150733 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application Aug 07 10:42:56.150733 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 530/1037] 10.222.0.25 () {56 vars in 1108 bytes} [Sun Aug 7 10:42:55 2022] GET /identity/v3/domains/Default => generated 87 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 07 10:42:56.155322 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e1d1480-d868-4b1c-873e-eb7934f22322 None admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 10:42:56.173297 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab7c51b5-b404-42be-b787-441e75879019 None admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 10:42:56.210851 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbb2cb90-0363-4bf6-805c-ae6e4bc271cd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 10:42:56.211586 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbb2cb90-0363-4bf6-805c-ae6e4bc271cd None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:42:56.212037 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbb2cb90-0363-4bf6-805c-ae6e4bc271cd None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:42:56.212157 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbb2cb90-0363-4bf6-805c-ae6e4bc271cd None admin] PATH_INFO: `/v3/services` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:42:56.212887 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbb2cb90-0363-4bf6-805c-ae6e4bc271cd None admin] RBAC: Adding query filter params (name=image) {{(pid=80143) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 10:42:56.213311 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbb2cb90-0363-4bf6-805c-ae6e4bc271cd None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 10:42:56.214696 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbb2cb90-0363-4bf6-805c-ae6e4bc271cd None admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 10:42:56.220196 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 508/1038] 10.222.0.25 () {56 vars in 1108 bytes} [Sun Aug 7 10:42:56 2022] GET /identity/v3/services?name=image => generated 122 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 07 10:42:56.225680 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbb2cb90-0363-4bf6-805c-ae6e4bc271cd None admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 10:42:56.238160 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=100 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 10:42:56.260315 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e1d1480-d868-4b1c-873e-eb7934f22322 None admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 10:42:56.295414 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d3ad563-931e-47fc-827f-6a3421bdbc4f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 10:42:56.296031 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d3ad563-931e-47fc-827f-6a3421bdbc4f None admin] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:42:56.296348 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d3ad563-931e-47fc-827f-6a3421bdbc4f None admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:42:56.296581 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d3ad563-931e-47fc-827f-6a3421bdbc4f None admin] PATH_INFO: `/v3/domains` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:42:56.297599 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d3ad563-931e-47fc-827f-6a3421bdbc4f None admin] RBAC: Adding query filter params (name=Default) {{(pid=80144) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 10:42:56.297599 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d3ad563-931e-47fc-827f-6a3421bdbc4f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 10:42:56.298845 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d3ad563-931e-47fc-827f-6a3421bdbc4f None admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 10:42:56.317457 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 531/1039] 10.222.0.25 () {56 vars in 1109 bytes} [Sun Aug 7 10:42:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 07 10:42:56.318468 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d3ad563-931e-47fc-827f-6a3421bdbc4f None admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 10:42:56.320540 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=100 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 10:42:56.337492 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbb2cb90-0363-4bf6-805c-ae6e4bc271cd None admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 10:42:56.372691 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5ab065e-f01e-410f-a154-fcfbfd7aa659 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 10:42:56.373296 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5ab065e-f01e-410f-a154-fcfbfd7aa659 None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:42:56.373572 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5ab065e-f01e-410f-a154-fcfbfd7aa659 None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:42:56.373842 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5ab065e-f01e-410f-a154-fcfbfd7aa659 None admin] PATH_INFO: `/v3/services` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:42:56.374465 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5ab065e-f01e-410f-a154-fcfbfd7aa659 None admin] RBAC: Adding query filter params (type=image) {{(pid=80143) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 10:42:56.374768 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5ab065e-f01e-410f-a154-fcfbfd7aa659 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 10:42:56.376149 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5ab065e-f01e-410f-a154-fcfbfd7aa659 None admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 10:42:56.380936 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 509/1040] 10.222.0.25 () {56 vars in 1108 bytes} [Sun Aug 7 10:42:56 2022] GET /identity/v3/services?type=image => generated 350 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 07 10:42:56.387301 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5ab065e-f01e-410f-a154-fcfbfd7aa659 None admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 10:42:56.437082 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d3ad563-931e-47fc-827f-6a3421bdbc4f None admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 10:42:56.472632 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7756f12c-6a2f-49cc-a48c-481f0547afed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 10:42:56.473258 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7756f12c-6a2f-49cc-a48c-481f0547afed None admin] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:42:56.473536 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7756f12c-6a2f-49cc-a48c-481f0547afed None admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:42:56.473812 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7756f12c-6a2f-49cc-a48c-481f0547afed None admin] PATH_INFO: `/v3/projects/service` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:42:56.481056 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7756f12c-6a2f-49cc-a48c-481f0547afed None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 10:42:56.482494 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7756f12c-6a2f-49cc-a48c-481f0547afed None admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 10:42:56.485015 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=100 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 10:42:56.494973 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5ab065e-f01e-410f-a154-fcfbfd7aa659 None admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 10:42:56.495165 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application [None req-7756f12c-6a2f-49cc-a48c-481f0547afed None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 07 10:42:56.495165 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 07 10:42:56.495165 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 07 10:42:56.495165 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 07 10:42:56.495165 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 07 10:42:56.495165 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 07 10:42:56.495165 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 07 10:42:56.495165 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 07 10:42:56.495165 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 07 10:42:56.495165 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 07 10:42:56.495165 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 07 10:42:56.495165 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 07 10:42:56.495165 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 07 10:42:56.495165 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 07 10:42:56.495165 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 07 10:42:56.496191 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 07 10:42:56.496191 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 07 10:42:56.496191 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 07 10:42:56.496191 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 07 10:42:56.496191 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 07 10:42:56.496191 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 07 10:42:56.496191 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self.get_or_create( Aug 07 10:42:56.496191 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 07 10:42:56.496191 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application with Lock( Aug 07 10:42:56.496191 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 07 10:42:56.496191 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self._enter() Aug 07 10:42:56.496191 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 07 10:42:56.496191 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 07 10:42:56.496191 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 07 10:42:56.496191 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self.creator() Aug 07 10:42:56.496191 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 07 10:42:56.496191 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application created_value = creator( Aug 07 10:42:56.497125 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 07 10:42:56.497125 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 07 10:42:56.497125 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 07 10:42:56.497125 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 07 10:42:56.497125 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 07 10:42:56.497125 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 07 10:42:56.497125 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 07 10:42:56.497125 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application Aug 07 10:42:56.497125 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 532/1041] 10.222.0.25 () {56 vars in 1111 bytes} [Sun Aug 7 10:42:56 2022] GET /identity/v3/projects/service => generated 88 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 07 10:42:56.501883 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7756f12c-6a2f-49cc-a48c-481f0547afed None admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 10:42:56.536233 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a76c6429-d38d-45f3-a297-f787c33ef31c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 10:42:56.536845 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a76c6429-d38d-45f3-a297-f787c33ef31c None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:42:56.537133 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a76c6429-d38d-45f3-a297-f787c33ef31c None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:42:56.537409 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a76c6429-d38d-45f3-a297-f787c33ef31c None admin] PATH_INFO: `/v3/endpoints` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:42:56.538118 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a76c6429-d38d-45f3-a297-f787c33ef31c None admin] RBAC: Adding query filter params (interface=public, service_id=afd952134cd64204990c71372f20d023, region_id=RegionOne) {{(pid=80143) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 10:42:56.538427 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a76c6429-d38d-45f3-a297-f787c33ef31c None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 10:42:56.540347 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a76c6429-d38d-45f3-a297-f787c33ef31c None admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 10:42:56.545571 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 510/1042] 10.222.0.25 () {56 vars in 1251 bytes} [Sun Aug 7 10:42:56 2022] GET /identity/v3/endpoints?service_id=afd952134cd64204990c71372f20d023&interface=public®ion_id=RegionOne => generated 194 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 07 10:42:56.550790 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a76c6429-d38d-45f3-a297-f787c33ef31c None admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 10:42:56.566751 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=100 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 10:42:56.611315 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7756f12c-6a2f-49cc-a48c-481f0547afed None admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 10:42:56.647627 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28b48366-71fb-4ff5-bca2-94d0ee25b7de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 10:42:56.649141 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28b48366-71fb-4ff5-bca2-94d0ee25b7de None admin] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:42:56.649582 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28b48366-71fb-4ff5-bca2-94d0ee25b7de None admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:42:56.650075 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28b48366-71fb-4ff5-bca2-94d0ee25b7de None admin] PATH_INFO: `/v3/projects` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:42:56.650851 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28b48366-71fb-4ff5-bca2-94d0ee25b7de None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80144) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 10:42:56.651193 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28b48366-71fb-4ff5-bca2-94d0ee25b7de None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 10:42:56.652656 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28b48366-71fb-4ff5-bca2-94d0ee25b7de None admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 10:42:56.661796 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a76c6429-d38d-45f3-a297-f787c33ef31c None admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 10:42:56.666968 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 533/1043] 10.222.0.25 () {56 vars in 1148 bytes} [Sun Aug 7 10:42:56 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 07 10:42:56.672664 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28b48366-71fb-4ff5-bca2-94d0ee25b7de None admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 10:42:56.700778 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55bd626d-aefc-4602-a76e-3d882208bf32 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 10:42:56.701590 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55bd626d-aefc-4602-a76e-3d882208bf32 None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:42:56.701870 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55bd626d-aefc-4602-a76e-3d882208bf32 None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:42:56.702458 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55bd626d-aefc-4602-a76e-3d882208bf32 None admin] PATH_INFO: `/v3/services` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:42:56.703353 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55bd626d-aefc-4602-a76e-3d882208bf32 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 10:42:56.705165 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55bd626d-aefc-4602-a76e-3d882208bf32 None admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 10:42:56.718970 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 511/1044] 10.222.0.25 () {56 vars in 1087 bytes} [Sun Aug 7 10:42:56 2022] GET /identity/v3/services => generated 1957 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 07 10:42:56.736698 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=100 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 10:42:56.784291 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28b48366-71fb-4ff5-bca2-94d0ee25b7de None admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 10:42:56.821520 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6e8d777-68f8-4702-a0aa-e5057d3c0330 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 10:42:56.822149 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6e8d777-68f8-4702-a0aa-e5057d3c0330 None admin] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:42:56.822430 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6e8d777-68f8-4702-a0aa-e5057d3c0330 None admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:42:56.822700 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6e8d777-68f8-4702-a0aa-e5057d3c0330 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:42:56.823542 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6e8d777-68f8-4702-a0aa-e5057d3c0330 None admin] RBAC: Adding query filter params (role.id=3f1b8463c86840fb848a4cdf7c3d98be, scope.project.id=6de1f3cac02c4bcdb05dd2337176033d, user.id=31d16d1b96104b4eb690398432b9829e) {{(pid=80144) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 10:42:56.823862 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6e8d777-68f8-4702-a0aa-e5057d3c0330 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 10:42:56.825292 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6e8d777-68f8-4702-a0aa-e5057d3c0330 None admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 10:42:56.839039 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 534/1045] 10.222.0.25 () {56 vars in 1374 bytes} [Sun Aug 7 10:42:56 2022] GET /identity/v3/role_assignments?user.id=31d16d1b96104b4eb690398432b9829e&scope.project.id=6de1f3cac02c4bcdb05dd2337176033d&role.id=3f1b8463c86840fb848a4cdf7c3d98be => generated 259 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 07 10:42:57.558576 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-171682a9-96b9-49eb-9c09-6b19e7755455 None None] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:42:57.558901 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-171682a9-96b9-49eb-9c09-6b19e7755455 None None] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:42:57.559158 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-171682a9-96b9-49eb-9c09-6b19e7755455 None None] PATH_INFO: `/` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:42:57.560501 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 512/1046] 10.222.0.25 () {54 vars in 927 bytes} [Sun Aug 7 10:42:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 07 10:42:57.565745 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b3286bd-cc89-4b40-af75-be2fd9ec0c34 None None] REQUEST_METHOD: `POST` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:42:57.566040 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b3286bd-cc89-4b40-af75-be2fd9ec0c34 None None] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:42:57.566318 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b3286bd-cc89-4b40-af75-be2fd9ec0c34 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:42:57.584186 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.auth.core [None req-0b3286bd-cc89-4b40-af75-be2fd9ec0c34 None None] MFA Rules not processed for user `bac4e6b76fe64217ac8fa44e71cffe6a`. Rule list: `[]` (Enabled: `True`). {{(pid=80144) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 10:42:57.585354 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.fernet_utils [None req-0b3286bd-cc89-4b40-af75-be2fd9ec0c34 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80144) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 10:42:57.651749 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 535/1047] 10.222.0.25 () {58 vars in 1024 bytes} [Sun Aug 7 10:42:57 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 07 10:42:57.674630 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-171682a9-96b9-49eb-9c09-6b19e7755455 None None] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 10:42:57.755164 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=100 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 10:42:57.780283 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-171682a9-96b9-49eb-9c09-6b19e7755455 None None] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 10:42:57.813993 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-017dd93f-be7a-4673-9709-280a2eea0fb1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 10:42:57.814622 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-017dd93f-be7a-4673-9709-280a2eea0fb1 None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:42:57.814901 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-017dd93f-be7a-4673-9709-280a2eea0fb1 None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:42:57.815204 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-017dd93f-be7a-4673-9709-280a2eea0fb1 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:42:57.826089 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-017dd93f-be7a-4673-9709-280a2eea0fb1 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 07 10:42:57.831107 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application [None req-017dd93f-be7a-4673-9709-280a2eea0fb1 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 07 10:42:57.831107 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 07 10:42:57.831107 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 07 10:42:57.831107 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 07 10:42:57.831107 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 07 10:42:57.831107 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 07 10:42:57.831107 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 07 10:42:57.831107 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 07 10:42:57.831107 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 07 10:42:57.831107 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 07 10:42:57.831107 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 07 10:42:57.831107 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 07 10:42:57.831107 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 07 10:42:57.831107 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 07 10:42:57.831107 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 07 10:42:57.831107 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 07 10:42:57.832101 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 07 10:42:57.832101 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 07 10:42:57.832101 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 07 10:42:57.832101 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 07 10:42:57.832101 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 07 10:42:57.832101 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 07 10:42:57.832101 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 07 10:42:57.832101 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 07 10:42:57.832101 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 07 10:42:57.832101 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.get_or_create( Aug 07 10:42:57.832101 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 07 10:42:57.832101 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application with Lock( Aug 07 10:42:57.832101 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 07 10:42:57.832101 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self._enter() Aug 07 10:42:57.832101 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 07 10:42:57.832101 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 07 10:42:57.832965 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 07 10:42:57.832965 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.creator() Aug 07 10:42:57.832965 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 07 10:42:57.832965 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application created_value = creator( Aug 07 10:42:57.832965 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 07 10:42:57.832965 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 07 10:42:57.832965 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 07 10:42:57.832965 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 07 10:42:57.832965 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 07 10:42:57.832965 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 07 10:42:57.832965 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 07 10:42:57.832965 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application Aug 07 10:42:57.835022 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 513/1048] 10.222.0.25 () {56 vars in 1096 bytes} [Sun Aug 7 10:42:57 2022] GET /identity/v3/roles/admin => generated 83 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 07 10:42:57.839206 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b3286bd-cc89-4b40-af75-be2fd9ec0c34 None None] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 10:42:57.894345 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=100 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 10:42:57.941564 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b3286bd-cc89-4b40-af75-be2fd9ec0c34 None None] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 10:42:57.976378 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70ca81ef-8a95-4bac-a871-d882db2b52e7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 10:42:57.976978 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70ca81ef-8a95-4bac-a871-d882db2b52e7 None admin] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:42:57.977289 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70ca81ef-8a95-4bac-a871-d882db2b52e7 None admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:42:57.977552 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70ca81ef-8a95-4bac-a871-d882db2b52e7 None admin] PATH_INFO: `/v3/roles` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:42:57.978203 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70ca81ef-8a95-4bac-a871-d882db2b52e7 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80144) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 10:42:57.978498 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70ca81ef-8a95-4bac-a871-d882db2b52e7 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 10:42:57.979856 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70ca81ef-8a95-4bac-a871-d882db2b52e7 None admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 10:42:57.988200 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 536/1049] 10.222.0.25 () {56 vars in 1099 bytes} [Sun Aug 7 10:42:57 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 10:42:57.993750 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-017dd93f-be7a-4673-9709-280a2eea0fb1 None admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 10:42:58.095090 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-017dd93f-be7a-4673-9709-280a2eea0fb1 None admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 10:42:58.128387 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77f36c80-d8c6-48a6-b957-64713059d3f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 10:42:58.129012 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77f36c80-d8c6-48a6-b957-64713059d3f5 None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:42:58.129291 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77f36c80-d8c6-48a6-b957-64713059d3f5 None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:42:58.129563 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77f36c80-d8c6-48a6-b957-64713059d3f5 None admin] PATH_INFO: `/v3/users/736e3f99fbd24cb79b20f3b520c1fb56` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:42:58.132357 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=100 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 10:42:58.144876 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77f36c80-d8c6-48a6-b957-64713059d3f5 None admin] RBAC: Authorizing `identity:get_user(user_id=736e3f99fbd24cb79b20f3b520c1fb56)` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 10:42:58.146308 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77f36c80-d8c6-48a6-b957-64713059d3f5 None admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 10:42:58.148111 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 514/1050] 10.222.0.25 () {56 vars in 1177 bytes} [Sun Aug 7 10:42:57 2022] GET /identity/v3/users/736e3f99fbd24cb79b20f3b520c1fb56 => generated 285 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 10:42:58.154234 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70ca81ef-8a95-4bac-a871-d882db2b52e7 None admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 10:42:58.253622 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70ca81ef-8a95-4bac-a871-d882db2b52e7 None admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 10:42:58.270294 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=100 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 10:42:58.288687 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-973e28b9-857c-4195-a9de-f52247f69eba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 10:42:58.289304 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-973e28b9-857c-4195-a9de-f52247f69eba None admin] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:42:58.289617 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-973e28b9-857c-4195-a9de-f52247f69eba None admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:42:58.289892 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-973e28b9-857c-4195-a9de-f52247f69eba None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:42:58.357441 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-973e28b9-857c-4195-a9de-f52247f69eba None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 10:42:58.358878 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-973e28b9-857c-4195-a9de-f52247f69eba None admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 10:42:58.471796 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 537/1051] 10.222.0.25 () {58 vars in 1282 bytes} [Sun Aug 7 10:42:58 2022] GET /identity/v3/auth/tokens => generated 1969 bytes in 314 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 07 10:42:58.473022 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77f36c80-d8c6-48a6-b957-64713059d3f5 None admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 10:42:58.574061 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77f36c80-d8c6-48a6-b957-64713059d3f5 None admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 10:42:58.608041 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a598a9f8-1fd4-4e1a-9691-c52a1e6f575f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 10:42:58.608670 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a598a9f8-1fd4-4e1a-9691-c52a1e6f575f None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:42:58.608947 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a598a9f8-1fd4-4e1a-9691-c52a1e6f575f None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:42:58.609215 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a598a9f8-1fd4-4e1a-9691-c52a1e6f575f None admin] PATH_INFO: `/v3/projects/4a1780a74fca49a89f96d3281c44dbc3` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:42:58.610324 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a598a9f8-1fd4-4e1a-9691-c52a1e6f575f None admin] RBAC: Authorizing `identity:get_project(project_id=4a1780a74fca49a89f96d3281c44dbc3)` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 10:42:58.611766 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a598a9f8-1fd4-4e1a-9691-c52a1e6f575f None admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 10:42:58.619124 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 515/1052] 10.222.0.25 () {56 vars in 1186 bytes} [Sun Aug 7 10:42:58 2022] GET /identity/v3/projects/4a1780a74fca49a89f96d3281c44dbc3 => generated 311 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 07 10:42:58.620363 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-973e28b9-857c-4195-a9de-f52247f69eba None admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 10:42:58.638443 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=100 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 10:42:58.720941 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-973e28b9-857c-4195-a9de-f52247f69eba None admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 10:42:58.737449 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c750a82e-f901-47cc-8159-4deeebfaad6b None None] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:42:58.738217 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c750a82e-f901-47cc-8159-4deeebfaad6b None None] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:42:58.738297 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c750a82e-f901-47cc-8159-4deeebfaad6b None None] PATH_INFO: `/` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:42:58.739616 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 516/1053] 10.222.0.25 () {54 vars in 927 bytes} [Sun Aug 7 10:42:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 07 10:42:58.747778 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9c7971e-698d-44be-b6eb-2c4764cb0495 None None] REQUEST_METHOD: `POST` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:42:58.748166 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9c7971e-698d-44be-b6eb-2c4764cb0495 None None] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:42:58.748520 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9c7971e-698d-44be-b6eb-2c4764cb0495 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:42:58.757037 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3d908ec-65a5-4722-9f54-6c57676185dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 10:42:58.757628 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3d908ec-65a5-4722-9f54-6c57676185dd None admin] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:42:58.757905 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3d908ec-65a5-4722-9f54-6c57676185dd None admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:42:58.758175 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3d908ec-65a5-4722-9f54-6c57676185dd None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:42:58.759527 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3d908ec-65a5-4722-9f54-6c57676185dd None admin] RBAC: Adding query filter params (role.id=81b1f1bcee2a473088fbbd327c1c5b23, scope.project.id=4a1780a74fca49a89f96d3281c44dbc3, user.id=736e3f99fbd24cb79b20f3b520c1fb56) {{(pid=80144) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 10:42:58.759839 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3d908ec-65a5-4722-9f54-6c57676185dd None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 10:42:58.761230 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3d908ec-65a5-4722-9f54-6c57676185dd None admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 10:42:58.766851 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 538/1054] 10.222.0.25 () {56 vars in 1374 bytes} [Sun Aug 7 10:42:58 2022] GET /identity/v3/role_assignments?user.id=736e3f99fbd24cb79b20f3b520c1fb56&scope.project.id=4a1780a74fca49a89f96d3281c44dbc3&role.id=81b1f1bcee2a473088fbbd327c1c5b23 => generated 259 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 10:42:58.771904 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.auth.core [None req-f9c7971e-698d-44be-b6eb-2c4764cb0495 None None] MFA Rules not processed for user `bac4e6b76fe64217ac8fa44e71cffe6a`. Rule list: `[]` (Enabled: `True`). {{(pid=80143) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 10:42:58.773273 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-f9c7971e-698d-44be-b6eb-2c4764cb0495 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 10:42:58.834543 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86c52be0-2599-458f-b1c6-0370534575d1 None None] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:42:58.835437 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86c52be0-2599-458f-b1c6-0370534575d1 None None] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:42:58.835725 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86c52be0-2599-458f-b1c6-0370534575d1 None None] PATH_INFO: `/` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:42:58.837999 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 539/1055] 10.222.0.25 () {54 vars in 927 bytes} [Sun Aug 7 10:42:58 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 07 10:42:58.842764 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=100 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 10:42:58.843521 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ac3323c-40aa-4e02-a1b1-f663483a41f5 None None] REQUEST_METHOD: `POST` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:42:58.843730 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ac3323c-40aa-4e02-a1b1-f663483a41f5 None None] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:42:58.844011 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ac3323c-40aa-4e02-a1b1-f663483a41f5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:42:58.853516 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 517/1056] 10.222.0.25 () {58 vars in 1024 bytes} [Sun Aug 7 10:42:58 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 07 10:42:58.865075 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.auth.core [None req-8ac3323c-40aa-4e02-a1b1-f663483a41f5 None None] MFA Rules not processed for user `bac4e6b76fe64217ac8fa44e71cffe6a`. Rule list: `[]` (Enabled: `True`). {{(pid=80144) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 10:42:58.866298 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.fernet_utils [None req-8ac3323c-40aa-4e02-a1b1-f663483a41f5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80144) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 10:42:58.876734 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9c7971e-698d-44be-b6eb-2c4764cb0495 None None] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 10:42:58.945349 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 540/1057] 10.222.0.25 () {58 vars in 1024 bytes} [Sun Aug 7 10:42:58 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 07 10:42:58.968714 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ac3323c-40aa-4e02-a1b1-f663483a41f5 None None] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 10:42:58.981095 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=100 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 10:42:58.983676 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9c7971e-698d-44be-b6eb-2c4764cb0495 None None] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 10:42:59.022231 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bffa18c-5990-470d-97d5-4eef19d29e74 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 10:42:59.022858 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bffa18c-5990-470d-97d5-4eef19d29e74 None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:42:59.023189 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bffa18c-5990-470d-97d5-4eef19d29e74 None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:42:59.023431 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bffa18c-5990-470d-97d5-4eef19d29e74 None admin] PATH_INFO: `/v3/services/image` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:42:59.024574 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bffa18c-5990-470d-97d5-4eef19d29e74 None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 10:42:59.026124 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bffa18c-5990-470d-97d5-4eef19d29e74 None admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 10:42:59.034714 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application [None req-0bffa18c-5990-470d-97d5-4eef19d29e74 None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Aug 07 10:42:59.034714 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 07 10:42:59.034714 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 07 10:42:59.034714 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 07 10:42:59.034714 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 07 10:42:59.034714 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 07 10:42:59.034714 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 07 10:42:59.034714 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 07 10:42:59.034714 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Aug 07 10:42:59.034714 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application Aug 07 10:42:59.034714 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 07 10:42:59.034714 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application Aug 07 10:42:59.034714 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 07 10:42:59.034714 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 07 10:42:59.034714 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 07 10:42:59.034714 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 07 10:42:59.034714 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 07 10:42:59.035956 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 07 10:42:59.035956 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 07 10:42:59.035956 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 07 10:42:59.035956 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 07 10:42:59.035956 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 07 10:42:59.035956 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 07 10:42:59.035956 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 07 10:42:59.035956 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 07 10:42:59.035956 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 07 10:42:59.035956 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 07 10:42:59.035956 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 07 10:42:59.035956 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 07 10:42:59.035956 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 07 10:42:59.035956 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 07 10:42:59.035956 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 07 10:42:59.035956 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.get_or_create( Aug 07 10:42:59.037299 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 07 10:42:59.037299 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application with Lock( Aug 07 10:42:59.037299 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 07 10:42:59.037299 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self._enter() Aug 07 10:42:59.037299 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 07 10:42:59.037299 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 07 10:42:59.037299 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 07 10:42:59.037299 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.creator() Aug 07 10:42:59.037299 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 07 10:42:59.037299 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application created_value = creator( Aug 07 10:42:59.037299 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 07 10:42:59.037299 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 07 10:42:59.037299 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Aug 07 10:42:59.037299 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application Aug 07 10:42:59.041013 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 518/1058] 10.222.0.25 () {56 vars in 1105 bytes} [Sun Aug 7 10:42:58 2022] GET /identity/v3/services/image => generated 86 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 07 10:42:59.041663 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bffa18c-5990-470d-97d5-4eef19d29e74 None admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 10:42:59.079270 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ac3323c-40aa-4e02-a1b1-f663483a41f5 None None] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 10:42:59.108494 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=100 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 10:42:59.116291 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f321a316-9430-458e-9571-2dc794ec67e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 10:42:59.116930 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f321a316-9430-458e-9571-2dc794ec67e2 None admin] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:42:59.117209 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f321a316-9430-458e-9571-2dc794ec67e2 None admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:42:59.117482 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f321a316-9430-458e-9571-2dc794ec67e2 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:42:59.128678 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f321a316-9430-458e-9571-2dc794ec67e2 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 07 10:42:59.133337 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application [None req-f321a316-9430-458e-9571-2dc794ec67e2 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 07 10:42:59.133337 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 07 10:42:59.133337 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 07 10:42:59.133337 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 07 10:42:59.133337 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 07 10:42:59.133337 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 07 10:42:59.133337 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 07 10:42:59.133337 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 07 10:42:59.133337 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 07 10:42:59.133337 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 07 10:42:59.133337 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 07 10:42:59.133337 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 07 10:42:59.133337 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 07 10:42:59.133337 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 07 10:42:59.133337 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 07 10:42:59.133337 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 07 10:42:59.134470 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 07 10:42:59.134470 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 07 10:42:59.134470 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 07 10:42:59.134470 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 07 10:42:59.134470 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 07 10:42:59.134470 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 07 10:42:59.134470 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 07 10:42:59.134470 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 07 10:42:59.134470 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 07 10:42:59.134470 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self.get_or_create( Aug 07 10:42:59.134470 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 07 10:42:59.134470 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application with Lock( Aug 07 10:42:59.134470 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 07 10:42:59.134470 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self._enter() Aug 07 10:42:59.134470 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 07 10:42:59.134470 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 07 10:42:59.136123 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 07 10:42:59.136123 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self.creator() Aug 07 10:42:59.136123 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 07 10:42:59.136123 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application created_value = creator( Aug 07 10:42:59.136123 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 07 10:42:59.136123 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 07 10:42:59.136123 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 07 10:42:59.136123 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 07 10:42:59.136123 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 07 10:42:59.136123 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 07 10:42:59.136123 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 07 10:42:59.136123 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application Aug 07 10:42:59.136123 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 541/1059] 10.222.0.25 () {56 vars in 1120 bytes} [Sun Aug 7 10:42:58 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 07 10:42:59.140168 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f321a316-9430-458e-9571-2dc794ec67e2 None admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 10:42:59.158573 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bffa18c-5990-470d-97d5-4eef19d29e74 None admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 10:42:59.195437 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68c371a7-a7d5-42a8-8228-a050cc7270a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 10:42:59.196408 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68c371a7-a7d5-42a8-8228-a050cc7270a6 None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:42:59.196854 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68c371a7-a7d5-42a8-8228-a050cc7270a6 None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:42:59.197130 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68c371a7-a7d5-42a8-8228-a050cc7270a6 None admin] PATH_INFO: `/v3/services` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:42:59.197751 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68c371a7-a7d5-42a8-8228-a050cc7270a6 None admin] RBAC: Adding query filter params (name=image) {{(pid=80143) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 10:42:59.198050 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68c371a7-a7d5-42a8-8228-a050cc7270a6 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 10:42:59.199502 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68c371a7-a7d5-42a8-8228-a050cc7270a6 None admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 10:42:59.204673 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 519/1060] 10.222.0.25 () {56 vars in 1108 bytes} [Sun Aug 7 10:42:59 2022] GET /identity/v3/services?name=image => generated 122 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 07 10:42:59.210078 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68c371a7-a7d5-42a8-8228-a050cc7270a6 None admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 10:42:59.216244 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=100 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 10:42:59.249505 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f321a316-9430-458e-9571-2dc794ec67e2 None admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 10:42:59.290845 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-632bd622-8ee0-453d-9666-fd5fc9ca6764 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 10:42:59.291990 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-632bd622-8ee0-453d-9666-fd5fc9ca6764 None admin] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:42:59.292783 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-632bd622-8ee0-453d-9666-fd5fc9ca6764 None admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:42:59.293061 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-632bd622-8ee0-453d-9666-fd5fc9ca6764 None admin] PATH_INFO: `/v3/roles` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:42:59.293780 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-632bd622-8ee0-453d-9666-fd5fc9ca6764 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80144) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 10:42:59.294086 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-632bd622-8ee0-453d-9666-fd5fc9ca6764 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 10:42:59.296317 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-632bd622-8ee0-453d-9666-fd5fc9ca6764 None admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 10:42:59.312507 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 542/1061] 10.222.0.25 () {56 vars in 1115 bytes} [Sun Aug 7 10:42:59 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 07 10:42:59.314104 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-632bd622-8ee0-453d-9666-fd5fc9ca6764 None admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 10:42:59.318876 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68c371a7-a7d5-42a8-8228-a050cc7270a6 None admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 10:42:59.340179 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=100 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 10:42:59.358829 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7f32849-aaee-4a95-8d0f-833a7a6b2524 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 10:42:59.359839 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7f32849-aaee-4a95-8d0f-833a7a6b2524 None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:42:59.360126 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7f32849-aaee-4a95-8d0f-833a7a6b2524 None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:42:59.360856 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7f32849-aaee-4a95-8d0f-833a7a6b2524 None admin] PATH_INFO: `/v3/services` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:42:59.361500 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7f32849-aaee-4a95-8d0f-833a7a6b2524 None admin] RBAC: Adding query filter params (type=image) {{(pid=80143) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 10:42:59.362099 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7f32849-aaee-4a95-8d0f-833a7a6b2524 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 10:42:59.363785 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7f32849-aaee-4a95-8d0f-833a7a6b2524 None admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 10:42:59.369887 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 520/1062] 10.222.0.25 () {56 vars in 1108 bytes} [Sun Aug 7 10:42:59 2022] GET /identity/v3/services?type=image => generated 350 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 07 10:42:59.375511 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7f32849-aaee-4a95-8d0f-833a7a6b2524 None admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 10:42:59.433596 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-632bd622-8ee0-453d-9666-fd5fc9ca6764 None admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 10:42:59.463092 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=100 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 10:42:59.468577 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05c882a0-cfe7-478d-bdce-66e4e81b9d2c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 10:42:59.469197 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05c882a0-cfe7-478d-bdce-66e4e81b9d2c None admin] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:42:59.469480 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05c882a0-cfe7-478d-bdce-66e4e81b9d2c None admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:42:59.469750 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05c882a0-cfe7-478d-bdce-66e4e81b9d2c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:42:59.482321 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05c882a0-cfe7-478d-bdce-66e4e81b9d2c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 10:42:59.483794 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05c882a0-cfe7-478d-bdce-66e4e81b9d2c None admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 10:42:59.487155 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7f32849-aaee-4a95-8d0f-833a7a6b2524 None admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 10:42:59.498996 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application [None req-05c882a0-cfe7-478d-bdce-66e4e81b9d2c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 07 10:42:59.498996 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 07 10:42:59.498996 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 07 10:42:59.498996 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 07 10:42:59.498996 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 07 10:42:59.498996 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 07 10:42:59.498996 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 07 10:42:59.498996 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 07 10:42:59.498996 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 07 10:42:59.498996 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 07 10:42:59.498996 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 07 10:42:59.498996 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 07 10:42:59.498996 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 07 10:42:59.498996 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 07 10:42:59.498996 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 07 10:42:59.498996 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 07 10:42:59.499964 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 07 10:42:59.499964 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 07 10:42:59.499964 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 07 10:42:59.499964 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 07 10:42:59.499964 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 07 10:42:59.499964 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self.get_or_create( Aug 07 10:42:59.499964 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 07 10:42:59.499964 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application with Lock( Aug 07 10:42:59.499964 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 07 10:42:59.499964 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self._enter() Aug 07 10:42:59.499964 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 07 10:42:59.499964 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 07 10:42:59.499964 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 07 10:42:59.499964 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self.creator() Aug 07 10:42:59.499964 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 07 10:42:59.499964 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application created_value = creator( Aug 07 10:42:59.499964 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 07 10:42:59.500849 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 07 10:42:59.500849 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 07 10:42:59.500849 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application Aug 07 10:42:59.500849 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 543/1063] 10.222.0.25 () {56 vars in 1108 bytes} [Sun Aug 7 10:42:59 2022] GET /identity/v3/domains/Default => generated 87 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 07 10:42:59.505820 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05c882a0-cfe7-478d-bdce-66e4e81b9d2c None admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 10:42:59.528267 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a29aa97b-c7bb-487e-af98-401c0d55d5f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 10:42:59.529182 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a29aa97b-c7bb-487e-af98-401c0d55d5f2 None admin] REQUEST_METHOD: `POST` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:42:59.529601 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a29aa97b-c7bb-487e-af98-401c0d55d5f2 None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:42:59.530151 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a29aa97b-c7bb-487e-af98-401c0d55d5f2 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:42:59.531089 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a29aa97b-c7bb-487e-af98-401c0d55d5f2 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 10:42:59.532499 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a29aa97b-c7bb-487e-af98-401c0d55d5f2 None admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 10:42:59.560559 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 521/1064] 10.222.0.25 () {60 vars in 1144 bytes} [Sun Aug 7 10:42:59 2022] POST /identity/v3/endpoints => generated 327 bytes in 179 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 07 10:42:59.625306 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05c882a0-cfe7-478d-bdce-66e4e81b9d2c None admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 10:42:59.668278 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-963eb4a3-bd81-46f2-b025-ef59e4f9d235 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 10:42:59.668905 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-963eb4a3-bd81-46f2-b025-ef59e4f9d235 None admin] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:42:59.669188 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-963eb4a3-bd81-46f2-b025-ef59e4f9d235 None admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:42:59.669460 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-963eb4a3-bd81-46f2-b025-ef59e4f9d235 None admin] PATH_INFO: `/v3/domains` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:42:59.670110 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-963eb4a3-bd81-46f2-b025-ef59e4f9d235 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80144) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 10:42:59.670421 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-963eb4a3-bd81-46f2-b025-ef59e4f9d235 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 10:42:59.671831 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-963eb4a3-bd81-46f2-b025-ef59e4f9d235 None admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 10:42:59.689176 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 544/1065] 10.222.0.25 () {56 vars in 1109 bytes} [Sun Aug 7 10:42:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 07 10:42:59.690820 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a29aa97b-c7bb-487e-af98-401c0d55d5f2 None admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 10:42:59.736339 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 10:42:59.801281 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a29aa97b-c7bb-487e-af98-401c0d55d5f2 None admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 10:42:59.837800 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-109d0dee-7003-411b-a2fb-234adfa492cb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 10:42:59.838443 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-109d0dee-7003-411b-a2fb-234adfa492cb None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:42:59.838746 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-109d0dee-7003-411b-a2fb-234adfa492cb None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:42:59.839035 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-109d0dee-7003-411b-a2fb-234adfa492cb None admin] PATH_INFO: `/v3/users/nova` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:42:59.846893 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-109d0dee-7003-411b-a2fb-234adfa492cb None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 10:42:59.848382 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-109d0dee-7003-411b-a2fb-234adfa492cb None admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 10:42:59.859579 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application [None req-109d0dee-7003-411b-a2fb-234adfa492cb None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Aug 07 10:42:59.859579 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 07 10:42:59.859579 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 07 10:42:59.859579 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 07 10:42:59.859579 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 07 10:42:59.859579 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 07 10:42:59.859579 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 07 10:42:59.859579 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 07 10:42:59.859579 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 07 10:42:59.859579 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 07 10:42:59.859579 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 07 10:42:59.859579 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 07 10:42:59.859579 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 07 10:42:59.859579 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 07 10:42:59.859579 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 07 10:42:59.859579 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 07 10:42:59.860540 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 07 10:42:59.860540 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 07 10:42:59.860540 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 07 10:42:59.860540 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 07 10:42:59.860540 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 07 10:42:59.860540 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 07 10:42:59.860540 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 07 10:42:59.860540 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 07 10:42:59.860540 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 07 10:42:59.860540 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.get_or_create( Aug 07 10:42:59.860540 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 07 10:42:59.860540 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application with Lock( Aug 07 10:42:59.860540 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 07 10:42:59.860540 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self._enter() Aug 07 10:42:59.860540 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 07 10:42:59.860540 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 07 10:42:59.860540 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 07 10:42:59.860540 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.creator() Aug 07 10:42:59.861491 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 07 10:42:59.861491 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application created_value = creator( Aug 07 10:42:59.861491 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 07 10:42:59.861491 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 07 10:42:59.861491 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 07 10:42:59.861491 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 07 10:42:59.861491 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 07 10:42:59.861491 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 07 10:42:59.861491 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Aug 07 10:42:59.861491 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application Aug 07 10:42:59.861491 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 522/1066] 10.222.0.25 () {56 vars in 1093 bytes} [Sun Aug 7 10:42:59 2022] GET /identity/v3/users/nova => generated 82 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 07 10:42:59.867048 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-963eb4a3-bd81-46f2-b025-ef59e4f9d235 None admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 10:42:59.914433 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=100 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 10:42:59.973476 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-963eb4a3-bd81-46f2-b025-ef59e4f9d235 None admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 10:43:00.007331 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9c8fd90-3b08-4a4a-897c-386a1452054b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 10:43:00.007938 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9c8fd90-3b08-4a4a-897c-386a1452054b None admin] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:43:00.008264 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9c8fd90-3b08-4a4a-897c-386a1452054b None admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:43:00.013026 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9c8fd90-3b08-4a4a-897c-386a1452054b None admin] PATH_INFO: `/v3/users` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:43:00.013760 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9c8fd90-3b08-4a4a-897c-386a1452054b None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80144) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 10:43:00.014069 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9c8fd90-3b08-4a4a-897c-386a1452054b None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 10:43:00.015510 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9c8fd90-3b08-4a4a-897c-386a1452054b None admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 10:43:00.036925 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 545/1067] 10.222.0.25 () {56 vars in 1133 bytes} [Sun Aug 7 10:42:59 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 07 10:43:00.039292 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-109d0dee-7003-411b-a2fb-234adfa492cb None admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 10:43:00.143398 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=100 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 10:43:00.147105 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-109d0dee-7003-411b-a2fb-234adfa492cb None admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 10:43:00.183364 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3f655b1-cabd-43d9-95ae-ab9018066a31 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 10:43:00.183927 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3f655b1-cabd-43d9-95ae-ab9018066a31 None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:43:00.184204 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3f655b1-cabd-43d9-95ae-ab9018066a31 None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:43:00.184475 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3f655b1-cabd-43d9-95ae-ab9018066a31 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:43:00.194823 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3f655b1-cabd-43d9-95ae-ab9018066a31 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 10:43:00.197002 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3f655b1-cabd-43d9-95ae-ab9018066a31 None admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 10:43:00.210928 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application [None req-f3f655b1-cabd-43d9-95ae-ab9018066a31 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 07 10:43:00.210928 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 07 10:43:00.210928 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 07 10:43:00.210928 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 07 10:43:00.210928 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 07 10:43:00.210928 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 07 10:43:00.210928 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 07 10:43:00.210928 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 07 10:43:00.210928 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 07 10:43:00.210928 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 07 10:43:00.210928 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 07 10:43:00.210928 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 07 10:43:00.210928 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 07 10:43:00.210928 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 07 10:43:00.210928 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 07 10:43:00.210928 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 07 10:43:00.212143 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 07 10:43:00.212143 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 07 10:43:00.212143 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 07 10:43:00.212143 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 07 10:43:00.212143 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 07 10:43:00.212143 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.get_or_create( Aug 07 10:43:00.212143 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 07 10:43:00.212143 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application with Lock( Aug 07 10:43:00.212143 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 07 10:43:00.212143 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self._enter() Aug 07 10:43:00.212143 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 07 10:43:00.212143 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 07 10:43:00.212143 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 07 10:43:00.212143 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.creator() Aug 07 10:43:00.212143 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 07 10:43:00.212143 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application created_value = creator( Aug 07 10:43:00.212143 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 07 10:43:00.213035 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 07 10:43:00.213035 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 07 10:43:00.213035 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application Aug 07 10:43:00.213882 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 523/1068] 10.222.0.25 () {56 vars in 1108 bytes} [Sun Aug 7 10:43:00 2022] GET /identity/v3/domains/Default => generated 87 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 07 10:43:00.218506 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9c8fd90-3b08-4a4a-897c-386a1452054b None admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 10:43:00.321914 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=100 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 10:43:00.326002 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9c8fd90-3b08-4a4a-897c-386a1452054b None admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 10:43:00.362203 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-419bb09c-3eb5-4796-b7ac-c2d02e90e927 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 10:43:00.363104 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-419bb09c-3eb5-4796-b7ac-c2d02e90e927 None admin] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:43:00.363388 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-419bb09c-3eb5-4796-b7ac-c2d02e90e927 None admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:43:00.363659 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-419bb09c-3eb5-4796-b7ac-c2d02e90e927 None admin] PATH_INFO: `/v3/domains` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:43:00.364297 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-419bb09c-3eb5-4796-b7ac-c2d02e90e927 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80144) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 10:43:00.364602 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-419bb09c-3eb5-4796-b7ac-c2d02e90e927 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 10:43:00.365950 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-419bb09c-3eb5-4796-b7ac-c2d02e90e927 None admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 10:43:00.382343 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 546/1069] 10.222.0.25 () {56 vars in 1109 bytes} [Sun Aug 7 10:43:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 10:43:00.385436 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3f655b1-cabd-43d9-95ae-ab9018066a31 None admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 10:43:00.497880 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3f655b1-cabd-43d9-95ae-ab9018066a31 None admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 10:43:00.531093 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c45b62e0-dadd-4cb8-b590-aa209d5d9eff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 10:43:00.531727 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c45b62e0-dadd-4cb8-b590-aa209d5d9eff None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:43:00.532008 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c45b62e0-dadd-4cb8-b590-aa209d5d9eff None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:43:00.532278 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c45b62e0-dadd-4cb8-b590-aa209d5d9eff None admin] PATH_INFO: `/v3/projects/service` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:43:00.538969 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c45b62e0-dadd-4cb8-b590-aa209d5d9eff None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 10:43:00.540470 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c45b62e0-dadd-4cb8-b590-aa209d5d9eff None admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 10:43:00.551488 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application [None req-c45b62e0-dadd-4cb8-b590-aa209d5d9eff None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 07 10:43:00.551488 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 07 10:43:00.551488 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 07 10:43:00.551488 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 07 10:43:00.551488 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 07 10:43:00.551488 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 07 10:43:00.551488 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 07 10:43:00.551488 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 07 10:43:00.551488 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 07 10:43:00.551488 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 07 10:43:00.551488 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 07 10:43:00.551488 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 07 10:43:00.551488 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 07 10:43:00.551488 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 07 10:43:00.551488 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 07 10:43:00.552382 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 07 10:43:00.552382 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 07 10:43:00.552382 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 07 10:43:00.552382 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 07 10:43:00.552382 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 07 10:43:00.552382 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 07 10:43:00.552382 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.get_or_create( Aug 07 10:43:00.552382 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 07 10:43:00.552382 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application with Lock( Aug 07 10:43:00.552382 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 07 10:43:00.552382 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self._enter() Aug 07 10:43:00.552382 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 07 10:43:00.552382 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 07 10:43:00.552382 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 07 10:43:00.552382 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.creator() Aug 07 10:43:00.552382 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 07 10:43:00.552382 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application created_value = creator( Aug 07 10:43:00.558468 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 07 10:43:00.558468 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 07 10:43:00.558468 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 07 10:43:00.558468 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 07 10:43:00.558468 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 07 10:43:00.558468 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 07 10:43:00.558468 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 07 10:43:00.558468 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application Aug 07 10:43:00.558468 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 524/1070] 10.222.0.25 () {56 vars in 1111 bytes} [Sun Aug 7 10:43:00 2022] GET /identity/v3/projects/service => generated 88 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 07 10:43:00.560854 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-419bb09c-3eb5-4796-b7ac-c2d02e90e927 None admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 10:43:00.664805 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-419bb09c-3eb5-4796-b7ac-c2d02e90e927 None admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 10:43:00.699378 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1da313c-adb2-4096-8fed-a77bad96369f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 10:43:00.700012 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1da313c-adb2-4096-8fed-a77bad96369f None admin] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:43:00.700297 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1da313c-adb2-4096-8fed-a77bad96369f None admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:43:00.700571 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1da313c-adb2-4096-8fed-a77bad96369f None admin] PATH_INFO: `/v3/projects` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:43:00.701279 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1da313c-adb2-4096-8fed-a77bad96369f None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80144) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 10:43:00.701591 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1da313c-adb2-4096-8fed-a77bad96369f None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 10:43:00.703068 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1da313c-adb2-4096-8fed-a77bad96369f None admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 10:43:00.721665 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 547/1071] 10.222.0.25 () {56 vars in 1148 bytes} [Sun Aug 7 10:43:00 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 10:43:00.723040 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c45b62e0-dadd-4cb8-b590-aa209d5d9eff None admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 10:43:00.740038 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=100 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 10:43:00.780675 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fdc1b6b-2e7e-4dd8-aae5-5bc04666fadf None None] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:43:00.781026 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fdc1b6b-2e7e-4dd8-aae5-5bc04666fadf None None] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:43:00.781248 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fdc1b6b-2e7e-4dd8-aae5-5bc04666fadf None None] PATH_INFO: `/` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:43:00.782591 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 548/1072] 10.222.0.25 () {54 vars in 927 bytes} [Sun Aug 7 10:43:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 07 10:43:00.788203 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37ff5861-0be3-405f-b483-4fcb05becb56 None None] REQUEST_METHOD: `POST` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:43:00.788488 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37ff5861-0be3-405f-b483-4fcb05becb56 None None] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:43:00.788765 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37ff5861-0be3-405f-b483-4fcb05becb56 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:43:00.807374 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.auth.core [None req-37ff5861-0be3-405f-b483-4fcb05becb56 None None] MFA Rules not processed for user `bac4e6b76fe64217ac8fa44e71cffe6a`. Rule list: `[]` (Enabled: `True`). {{(pid=80144) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 10:43:00.808637 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.fernet_utils [None req-37ff5861-0be3-405f-b483-4fcb05becb56 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80144) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 10:43:00.822634 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=100 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 10:43:00.834954 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c45b62e0-dadd-4cb8-b590-aa209d5d9eff None admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 10:43:00.871807 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ba37cab-d78c-462d-a533-f27d836d7bb8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 10:43:00.873112 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ba37cab-d78c-462d-a533-f27d836d7bb8 None admin] REQUEST_METHOD: `PUT` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:43:00.873407 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ba37cab-d78c-462d-a533-f27d836d7bb8 None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:43:00.873682 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ba37cab-d78c-462d-a533-f27d836d7bb8 None admin] PATH_INFO: `/v3/projects/6de1f3cac02c4bcdb05dd2337176033d/users/31d16d1b96104b4eb690398432b9829e/roles/3f1b8463c86840fb848a4cdf7c3d98be` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:43:00.876710 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ba37cab-d78c-462d-a533-f27d836d7bb8 None admin] RBAC: Authorizing `identity:create_grant(project_id=6de1f3cac02c4bcdb05dd2337176033d, user_id=31d16d1b96104b4eb690398432b9829e, role_id=3f1b8463c86840fb848a4cdf7c3d98be)` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 10:43:00.879102 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ba37cab-d78c-462d-a533-f27d836d7bb8 None admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 10:43:00.882540 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 549/1073] 10.222.0.25 () {58 vars in 1024 bytes} [Sun Aug 7 10:43:00 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 07 10:43:00.893942 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 525/1074] 10.222.0.25 () {58 vars in 1439 bytes} [Sun Aug 7 10:43:00 2022] PUT /identity/v3/projects/6de1f3cac02c4bcdb05dd2337176033d/users/31d16d1b96104b4eb690398432b9829e/roles/3f1b8463c86840fb848a4cdf7c3d98be => generated 0 bytes in 171 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 07 10:43:00.905154 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37ff5861-0be3-405f-b483-4fcb05becb56 None None] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 10:43:01.015843 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37ff5861-0be3-405f-b483-4fcb05becb56 None None] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 10:43:01.062121 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-553ebd70-2f8a-4290-bede-2ade533572e4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 10:43:01.062679 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-553ebd70-2f8a-4290-bede-2ade533572e4 None admin] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:43:01.063016 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-553ebd70-2f8a-4290-bede-2ade533572e4 None admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:43:01.063292 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-553ebd70-2f8a-4290-bede-2ade533572e4 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:43:01.073775 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-553ebd70-2f8a-4290-bede-2ade533572e4 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 07 10:43:01.078793 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application [None req-553ebd70-2f8a-4290-bede-2ade533572e4 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 07 10:43:01.078793 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 07 10:43:01.078793 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 07 10:43:01.078793 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 07 10:43:01.078793 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 07 10:43:01.078793 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 07 10:43:01.078793 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 07 10:43:01.078793 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 07 10:43:01.078793 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 07 10:43:01.078793 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 07 10:43:01.078793 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 07 10:43:01.078793 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 07 10:43:01.078793 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 07 10:43:01.078793 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 07 10:43:01.078793 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 07 10:43:01.078793 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 07 10:43:01.079820 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 07 10:43:01.079820 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 07 10:43:01.079820 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 07 10:43:01.079820 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 07 10:43:01.079820 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 07 10:43:01.079820 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 07 10:43:01.079820 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 07 10:43:01.079820 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 07 10:43:01.079820 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 07 10:43:01.079820 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self.get_or_create( Aug 07 10:43:01.079820 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 07 10:43:01.079820 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application with Lock( Aug 07 10:43:01.079820 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 07 10:43:01.079820 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self._enter() Aug 07 10:43:01.079820 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 07 10:43:01.079820 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 07 10:43:01.080649 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 07 10:43:01.080649 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self.creator() Aug 07 10:43:01.080649 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 07 10:43:01.080649 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application created_value = creator( Aug 07 10:43:01.080649 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 07 10:43:01.080649 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 07 10:43:01.080649 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 07 10:43:01.080649 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 07 10:43:01.080649 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 07 10:43:01.080649 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 07 10:43:01.080649 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 07 10:43:01.080649 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application Aug 07 10:43:01.080649 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 550/1075] 10.222.0.25 () {56 vars in 1096 bytes} [Sun Aug 7 10:43:00 2022] GET /identity/v3/roles/admin => generated 83 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 07 10:43:01.086997 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ba37cab-d78c-462d-a533-f27d836d7bb8 None admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 10:43:01.160120 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 10:43:01.193545 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ba37cab-d78c-462d-a533-f27d836d7bb8 None admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 10:43:01.228446 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2c329ff-0269-48cf-ae70-284b40903aef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 10:43:01.229057 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2c329ff-0269-48cf-ae70-284b40903aef None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:43:01.229340 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2c329ff-0269-48cf-ae70-284b40903aef None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:43:01.229614 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2c329ff-0269-48cf-ae70-284b40903aef None admin] PATH_INFO: `/v3/roles` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:43:01.230281 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2c329ff-0269-48cf-ae70-284b40903aef None admin] RBAC: Adding query filter params (name=admin) {{(pid=80143) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 10:43:01.230588 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2c329ff-0269-48cf-ae70-284b40903aef None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 10:43:01.231998 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2c329ff-0269-48cf-ae70-284b40903aef None admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 10:43:01.241313 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 526/1076] 10.222.0.25 () {56 vars in 1099 bytes} [Sun Aug 7 10:43:01 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 10:43:01.247914 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-553ebd70-2f8a-4290-bede-2ade533572e4 None admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 10:43:01.259598 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=100 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 10:43:01.357639 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-553ebd70-2f8a-4290-bede-2ade533572e4 None admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 10:43:01.393644 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e42de3f-2e97-4e9b-b369-f73b6664532b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 10:43:01.394273 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e42de3f-2e97-4e9b-b369-f73b6664532b None admin] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:43:01.394539 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e42de3f-2e97-4e9b-b369-f73b6664532b None admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:43:01.394815 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e42de3f-2e97-4e9b-b369-f73b6664532b None admin] PATH_INFO: `/v3/users/736e3f99fbd24cb79b20f3b520c1fb56` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:43:01.396019 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e42de3f-2e97-4e9b-b369-f73b6664532b None admin] RBAC: Authorizing `identity:get_user(user_id=736e3f99fbd24cb79b20f3b520c1fb56)` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 10:43:01.397509 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e42de3f-2e97-4e9b-b369-f73b6664532b None admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 10:43:01.403999 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 551/1077] 10.222.0.25 () {56 vars in 1177 bytes} [Sun Aug 7 10:43:01 2022] GET /identity/v3/users/736e3f99fbd24cb79b20f3b520c1fb56 => generated 285 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 10:43:01.404990 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2c329ff-0269-48cf-ae70-284b40903aef None admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 10:43:01.512392 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2c329ff-0269-48cf-ae70-284b40903aef None admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 10:43:01.545962 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37090738-2d51-41e8-8be4-20488112cba5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 10:43:01.546590 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37090738-2d51-41e8-8be4-20488112cba5 None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:43:01.546868 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37090738-2d51-41e8-8be4-20488112cba5 None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:43:01.547165 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37090738-2d51-41e8-8be4-20488112cba5 None admin] PATH_INFO: `/v3/projects/4a1780a74fca49a89f96d3281c44dbc3` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:43:01.548268 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37090738-2d51-41e8-8be4-20488112cba5 None admin] RBAC: Authorizing `identity:get_project(project_id=4a1780a74fca49a89f96d3281c44dbc3)` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 10:43:01.549724 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37090738-2d51-41e8-8be4-20488112cba5 None admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 10:43:01.556830 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 527/1078] 10.222.0.25 () {56 vars in 1186 bytes} [Sun Aug 7 10:43:01 2022] GET /identity/v3/projects/4a1780a74fca49a89f96d3281c44dbc3 => generated 311 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 07 10:43:01.557795 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e42de3f-2e97-4e9b-b369-f73b6664532b None admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 10:43:01.603665 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75b42ebb-fc50-40c0-ad16-3ef76e333f63 None None] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:43:01.603942 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75b42ebb-fc50-40c0-ad16-3ef76e333f63 None None] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:43:01.604216 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75b42ebb-fc50-40c0-ad16-3ef76e333f63 None None] PATH_INFO: `/` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:43:01.605605 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 528/1079] 10.222.0.25 () {54 vars in 927 bytes} [Sun Aug 7 10:43:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 07 10:43:01.610396 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a8eb1b9-2cd0-415a-a917-6da264fca905 None None] REQUEST_METHOD: `POST` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:43:01.610625 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a8eb1b9-2cd0-415a-a917-6da264fca905 None None] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:43:01.610898 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a8eb1b9-2cd0-415a-a917-6da264fca905 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:43:01.618128 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=100 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 10:43:01.631987 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.auth.core [None req-3a8eb1b9-2cd0-415a-a917-6da264fca905 None None] MFA Rules not processed for user `bac4e6b76fe64217ac8fa44e71cffe6a`. Rule list: `[]` (Enabled: `True`). {{(pid=80143) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 10:43:01.633200 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-3a8eb1b9-2cd0-415a-a917-6da264fca905 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 10:43:01.640088 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=100 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 10:43:01.664377 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e42de3f-2e97-4e9b-b369-f73b6664532b None admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 10:43:01.700596 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3fd84a4-5358-4a07-979f-e8ad4f5d3e6e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 10:43:01.701219 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3fd84a4-5358-4a07-979f-e8ad4f5d3e6e None admin] REQUEST_METHOD: `PUT` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:43:01.701509 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3fd84a4-5358-4a07-979f-e8ad4f5d3e6e None admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:43:01.701765 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3fd84a4-5358-4a07-979f-e8ad4f5d3e6e None admin] PATH_INFO: `/v3/projects/4a1780a74fca49a89f96d3281c44dbc3/users/736e3f99fbd24cb79b20f3b520c1fb56/roles/81b1f1bcee2a473088fbbd327c1c5b23` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:43:01.703133 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 529/1080] 10.222.0.25 () {58 vars in 1024 bytes} [Sun Aug 7 10:43:01 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 07 10:43:01.703780 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3fd84a4-5358-4a07-979f-e8ad4f5d3e6e None admin] RBAC: Authorizing `identity:create_grant(project_id=4a1780a74fca49a89f96d3281c44dbc3, user_id=736e3f99fbd24cb79b20f3b520c1fb56, role_id=81b1f1bcee2a473088fbbd327c1c5b23)` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 10:43:01.705180 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3fd84a4-5358-4a07-979f-e8ad4f5d3e6e None admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 10:43:01.722978 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 552/1081] 10.222.0.25 () {58 vars in 1439 bytes} [Sun Aug 7 10:43:01 2022] PUT /identity/v3/projects/4a1780a74fca49a89f96d3281c44dbc3/users/736e3f99fbd24cb79b20f3b520c1fb56/roles/81b1f1bcee2a473088fbbd327c1c5b23 => generated 0 bytes in 165 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 07 10:43:01.724298 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a8eb1b9-2cd0-415a-a917-6da264fca905 None None] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 10:43:01.838819 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a8eb1b9-2cd0-415a-a917-6da264fca905 None None] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 10:43:01.870581 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=100 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 10:43:01.877408 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d337560-3fe6-47e5-b5fb-2fb3e8b540fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 10:43:01.878039 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d337560-3fe6-47e5-b5fb-2fb3e8b540fc None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:43:01.878322 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d337560-3fe6-47e5-b5fb-2fb3e8b540fc None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:43:01.878596 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d337560-3fe6-47e5-b5fb-2fb3e8b540fc None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:43:01.949505 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d337560-3fe6-47e5-b5fb-2fb3e8b540fc None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 10:43:01.950980 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d337560-3fe6-47e5-b5fb-2fb3e8b540fc None admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 10:43:02.057668 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 530/1082] 10.222.0.25 () {58 vars in 1282 bytes} [Sun Aug 7 10:43:01 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 328 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 07 10:43:02.058906 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3fd84a4-5358-4a07-979f-e8ad4f5d3e6e None admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 10:43:02.159291 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3fd84a4-5358-4a07-979f-e8ad4f5d3e6e None admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 10:43:02.192431 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f73bb65-d739-40bb-9931-91768942fc44 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 10:43:02.193059 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f73bb65-d739-40bb-9931-91768942fc44 None admin] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:43:02.193339 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f73bb65-d739-40bb-9931-91768942fc44 None admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:43:02.193613 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f73bb65-d739-40bb-9931-91768942fc44 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:43:02.196534 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 10:43:02.206569 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f73bb65-d739-40bb-9931-91768942fc44 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 10:43:02.208047 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f73bb65-d739-40bb-9931-91768942fc44 None admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 10:43:02.222812 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application [None req-4f73bb65-d739-40bb-9931-91768942fc44 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 07 10:43:02.222812 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 07 10:43:02.222812 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 07 10:43:02.222812 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 07 10:43:02.222812 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 07 10:43:02.222812 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 07 10:43:02.222812 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 07 10:43:02.222812 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 07 10:43:02.222812 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 07 10:43:02.222812 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 07 10:43:02.222812 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 07 10:43:02.222812 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 07 10:43:02.222812 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 07 10:43:02.222812 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 07 10:43:02.222812 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 07 10:43:02.222812 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 07 10:43:02.223812 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 07 10:43:02.223812 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 07 10:43:02.223812 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 07 10:43:02.223812 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 07 10:43:02.223812 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 07 10:43:02.223812 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self.get_or_create( Aug 07 10:43:02.223812 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 07 10:43:02.223812 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application with Lock( Aug 07 10:43:02.223812 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 07 10:43:02.223812 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self._enter() Aug 07 10:43:02.223812 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 07 10:43:02.223812 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 07 10:43:02.223812 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 07 10:43:02.223812 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self.creator() Aug 07 10:43:02.223812 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 07 10:43:02.223812 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application created_value = creator( Aug 07 10:43:02.223812 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 07 10:43:02.224728 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 07 10:43:02.224728 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 07 10:43:02.224728 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application Aug 07 10:43:02.227724 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 553/1083] 10.222.0.25 () {56 vars in 1108 bytes} [Sun Aug 7 10:43:02 2022] GET /identity/v3/domains/Default => generated 87 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 07 10:43:02.230665 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d337560-3fe6-47e5-b5fb-2fb3e8b540fc None admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 10:43:02.262410 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=100 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 10:43:02.332871 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d337560-3fe6-47e5-b5fb-2fb3e8b540fc None admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 10:43:02.366194 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edcf3dee-81f1-4f44-9ef9-e1907bfe2f8f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 10:43:02.366804 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edcf3dee-81f1-4f44-9ef9-e1907bfe2f8f None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:43:02.367116 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edcf3dee-81f1-4f44-9ef9-e1907bfe2f8f None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:43:02.367392 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edcf3dee-81f1-4f44-9ef9-e1907bfe2f8f None admin] PATH_INFO: `/v3/domains` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:43:02.368554 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edcf3dee-81f1-4f44-9ef9-e1907bfe2f8f None admin] RBAC: Adding query filter params (name=Default) {{(pid=80143) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 10:43:02.368852 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edcf3dee-81f1-4f44-9ef9-e1907bfe2f8f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 10:43:02.370209 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edcf3dee-81f1-4f44-9ef9-e1907bfe2f8f None admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 10:43:02.386955 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 531/1084] 10.222.0.25 () {56 vars in 1109 bytes} [Sun Aug 7 10:43:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 07 10:43:02.924705 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c2d3771-c5b4-4cfb-9a38-2cf9c6d4b939 None None] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:43:02.924997 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c2d3771-c5b4-4cfb-9a38-2cf9c6d4b939 None None] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:43:02.925273 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c2d3771-c5b4-4cfb-9a38-2cf9c6d4b939 None None] PATH_INFO: `/` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:43:02.926755 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 554/1085] 10.222.0.25 () {54 vars in 927 bytes} [Sun Aug 7 10:43:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 07 10:43:02.932340 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35a30268-1625-4451-a995-74ef0db2753f None None] REQUEST_METHOD: `POST` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:43:02.932777 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35a30268-1625-4451-a995-74ef0db2753f None None] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:43:02.932880 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35a30268-1625-4451-a995-74ef0db2753f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:43:02.951086 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.auth.core [None req-35a30268-1625-4451-a995-74ef0db2753f None None] MFA Rules not processed for user `bac4e6b76fe64217ac8fa44e71cffe6a`. Rule list: `[]` (Enabled: `True`). {{(pid=80143) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 10:43:02.952242 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-35a30268-1625-4451-a995-74ef0db2753f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 10:43:03.016526 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 532/1086] 10.222.0.25 () {58 vars in 1024 bytes} [Sun Aug 7 10:43:02 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 07 10:43:03.038787 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c2d3771-c5b4-4cfb-9a38-2cf9c6d4b939 None None] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 10:43:03.139122 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c2d3771-c5b4-4cfb-9a38-2cf9c6d4b939 None None] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 10:43:03.172527 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c046b4f2-3851-4bcb-9cf8-4947956d0fd7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 10:43:03.173153 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c046b4f2-3851-4bcb-9cf8-4947956d0fd7 None admin] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:43:03.173460 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c046b4f2-3851-4bcb-9cf8-4947956d0fd7 None admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:43:03.173708 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c046b4f2-3851-4bcb-9cf8-4947956d0fd7 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:43:03.184018 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c046b4f2-3851-4bcb-9cf8-4947956d0fd7 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 07 10:43:03.188647 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application [None req-c046b4f2-3851-4bcb-9cf8-4947956d0fd7 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 07 10:43:03.188647 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 07 10:43:03.188647 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 07 10:43:03.188647 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 07 10:43:03.188647 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 07 10:43:03.188647 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 07 10:43:03.188647 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 07 10:43:03.188647 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 07 10:43:03.188647 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 07 10:43:03.188647 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 07 10:43:03.188647 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 07 10:43:03.188647 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 07 10:43:03.188647 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 07 10:43:03.188647 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 07 10:43:03.188647 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 07 10:43:03.188647 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 07 10:43:03.189712 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 07 10:43:03.189712 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 07 10:43:03.189712 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 07 10:43:03.189712 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 07 10:43:03.189712 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 07 10:43:03.189712 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 07 10:43:03.189712 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 07 10:43:03.189712 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 07 10:43:03.189712 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 07 10:43:03.189712 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self.get_or_create( Aug 07 10:43:03.189712 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 07 10:43:03.189712 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application with Lock( Aug 07 10:43:03.189712 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 07 10:43:03.189712 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self._enter() Aug 07 10:43:03.189712 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 07 10:43:03.189712 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 07 10:43:03.190576 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 07 10:43:03.190576 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self.creator() Aug 07 10:43:03.190576 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 07 10:43:03.190576 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application created_value = creator( Aug 07 10:43:03.190576 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 07 10:43:03.190576 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 07 10:43:03.190576 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 07 10:43:03.190576 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 07 10:43:03.190576 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 07 10:43:03.190576 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 07 10:43:03.190576 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 07 10:43:03.190576 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application Aug 07 10:43:03.190576 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 555/1087] 10.222.0.25 () {56 vars in 1120 bytes} [Sun Aug 7 10:43:03 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 151 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 07 10:43:03.196462 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35a30268-1625-4451-a995-74ef0db2753f None None] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 10:43:03.219879 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=100 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 10:43:03.292738 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35a30268-1625-4451-a995-74ef0db2753f None None] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 10:43:03.327664 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1aadaa95-19af-4ec3-abdb-f6152e87c9bb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 10:43:03.328270 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aadaa95-19af-4ec3-abdb-f6152e87c9bb None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:43:03.328548 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aadaa95-19af-4ec3-abdb-f6152e87c9bb None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:43:03.328819 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aadaa95-19af-4ec3-abdb-f6152e87c9bb None admin] PATH_INFO: `/v3/roles` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:43:03.329476 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1aadaa95-19af-4ec3-abdb-f6152e87c9bb None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80143) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 10:43:03.329776 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1aadaa95-19af-4ec3-abdb-f6152e87c9bb None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 10:43:03.331160 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1aadaa95-19af-4ec3-abdb-f6152e87c9bb None admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 10:43:03.340116 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 533/1088] 10.222.0.25 () {56 vars in 1115 bytes} [Sun Aug 7 10:43:03 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 10:43:03.345710 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c046b4f2-3851-4bcb-9cf8-4947956d0fd7 None admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 10:43:03.395236 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=100 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 10:43:03.451239 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c046b4f2-3851-4bcb-9cf8-4947956d0fd7 None admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 10:43:03.489470 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-821d00c5-1d0f-4f50-8047-1bd64e574e00 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 10:43:03.490092 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-821d00c5-1d0f-4f50-8047-1bd64e574e00 None admin] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:43:03.490373 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-821d00c5-1d0f-4f50-8047-1bd64e574e00 None admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:43:03.490644 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-821d00c5-1d0f-4f50-8047-1bd64e574e00 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:43:03.500961 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-821d00c5-1d0f-4f50-8047-1bd64e574e00 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 10:43:03.502389 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-821d00c5-1d0f-4f50-8047-1bd64e574e00 None admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 10:43:03.518196 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application [None req-821d00c5-1d0f-4f50-8047-1bd64e574e00 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 07 10:43:03.518196 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 07 10:43:03.518196 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 07 10:43:03.518196 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 07 10:43:03.518196 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 07 10:43:03.518196 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 07 10:43:03.518196 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 07 10:43:03.518196 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 07 10:43:03.518196 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 07 10:43:03.518196 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 07 10:43:03.518196 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 07 10:43:03.518196 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 07 10:43:03.518196 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 07 10:43:03.518196 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 07 10:43:03.518196 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 07 10:43:03.518196 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 07 10:43:03.519370 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 07 10:43:03.519370 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 07 10:43:03.519370 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 07 10:43:03.519370 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 07 10:43:03.519370 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 07 10:43:03.519370 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self.get_or_create( Aug 07 10:43:03.519370 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 07 10:43:03.519370 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application with Lock( Aug 07 10:43:03.519370 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 07 10:43:03.519370 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self._enter() Aug 07 10:43:03.519370 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 07 10:43:03.519370 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 07 10:43:03.519370 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 07 10:43:03.519370 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self.creator() Aug 07 10:43:03.519370 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 07 10:43:03.519370 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application created_value = creator( Aug 07 10:43:03.519370 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 07 10:43:03.520260 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 07 10:43:03.520260 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 07 10:43:03.520260 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application Aug 07 10:43:03.520260 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 556/1089] 10.222.0.25 () {56 vars in 1108 bytes} [Sun Aug 7 10:43:03 2022] GET /identity/v3/domains/Default => generated 87 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 07 10:43:03.525626 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1aadaa95-19af-4ec3-abdb-f6152e87c9bb None admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 10:43:03.607706 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=100 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 10:43:03.624240 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1aadaa95-19af-4ec3-abdb-f6152e87c9bb None admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 10:43:03.657180 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b903e51-69d0-4145-a788-232fe902284d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 10:43:03.657784 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b903e51-69d0-4145-a788-232fe902284d None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:43:03.658062 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b903e51-69d0-4145-a788-232fe902284d None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:43:03.658330 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b903e51-69d0-4145-a788-232fe902284d None admin] PATH_INFO: `/v3/domains` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:43:03.658990 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b903e51-69d0-4145-a788-232fe902284d None admin] RBAC: Adding query filter params (name=Default) {{(pid=80143) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 10:43:03.659294 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b903e51-69d0-4145-a788-232fe902284d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 10:43:03.660725 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b903e51-69d0-4145-a788-232fe902284d None admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 10:43:03.672107 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 534/1090] 10.222.0.25 () {56 vars in 1109 bytes} [Sun Aug 7 10:43:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 10:43:03.677711 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-821d00c5-1d0f-4f50-8047-1bd64e574e00 None admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 10:43:03.755768 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9566bcbf-a17f-463e-8ae1-0cc384dd7b89 None None] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:43:03.756048 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9566bcbf-a17f-463e-8ae1-0cc384dd7b89 None None] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:43:03.756317 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9566bcbf-a17f-463e-8ae1-0cc384dd7b89 None None] PATH_INFO: `/` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:43:03.757797 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 535/1091] 10.222.0.25 () {54 vars in 927 bytes} [Sun Aug 7 10:43:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 07 10:43:03.762496 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0458fab6-0f44-4ecd-aacf-c112a4d78fb6 None None] REQUEST_METHOD: `POST` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:43:03.762776 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0458fab6-0f44-4ecd-aacf-c112a4d78fb6 None None] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:43:03.763098 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0458fab6-0f44-4ecd-aacf-c112a4d78fb6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:43:03.777963 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=100 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 10:43:03.781174 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-821d00c5-1d0f-4f50-8047-1bd64e574e00 None admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 10:43:03.787738 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.auth.core [None req-0458fab6-0f44-4ecd-aacf-c112a4d78fb6 None None] MFA Rules not processed for user `bac4e6b76fe64217ac8fa44e71cffe6a`. Rule list: `[]` (Enabled: `True`). {{(pid=80143) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 10:43:03.788908 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-0458fab6-0f44-4ecd-aacf-c112a4d78fb6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 10:43:03.814831 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e90d1cc-e257-4202-a028-8b7df84c0dcc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 10:43:03.815493 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e90d1cc-e257-4202-a028-8b7df84c0dcc None admin] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:43:03.815772 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e90d1cc-e257-4202-a028-8b7df84c0dcc None admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:43:03.816044 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e90d1cc-e257-4202-a028-8b7df84c0dcc None admin] PATH_INFO: `/v3/users/nova` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:43:03.823501 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e90d1cc-e257-4202-a028-8b7df84c0dcc None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 10:43:03.824931 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e90d1cc-e257-4202-a028-8b7df84c0dcc None admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 10:43:03.835614 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application [None req-4e90d1cc-e257-4202-a028-8b7df84c0dcc None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Aug 07 10:43:03.835614 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 07 10:43:03.835614 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 07 10:43:03.835614 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 07 10:43:03.835614 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 07 10:43:03.835614 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 07 10:43:03.835614 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 07 10:43:03.835614 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 07 10:43:03.835614 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 07 10:43:03.835614 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 07 10:43:03.835614 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 07 10:43:03.835614 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 07 10:43:03.835614 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 07 10:43:03.835614 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 07 10:43:03.835614 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 07 10:43:03.835614 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 07 10:43:03.837358 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 07 10:43:03.837358 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 07 10:43:03.837358 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 07 10:43:03.837358 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 07 10:43:03.837358 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 07 10:43:03.837358 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 07 10:43:03.837358 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 07 10:43:03.837358 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 07 10:43:03.837358 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 07 10:43:03.837358 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self.get_or_create( Aug 07 10:43:03.837358 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 07 10:43:03.837358 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application with Lock( Aug 07 10:43:03.837358 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 07 10:43:03.837358 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self._enter() Aug 07 10:43:03.837358 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 07 10:43:03.837358 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 07 10:43:03.837358 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 07 10:43:03.837358 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self.creator() Aug 07 10:43:03.839046 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 07 10:43:03.839046 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application created_value = creator( Aug 07 10:43:03.839046 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 07 10:43:03.839046 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 07 10:43:03.839046 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 07 10:43:03.839046 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 07 10:43:03.839046 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 07 10:43:03.839046 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 07 10:43:03.839046 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Aug 07 10:43:03.839046 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application Aug 07 10:43:03.839046 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 557/1092] 10.222.0.25 () {56 vars in 1093 bytes} [Sun Aug 7 10:43:03 2022] GET /identity/v3/users/nova => generated 82 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 07 10:43:03.842836 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e90d1cc-e257-4202-a028-8b7df84c0dcc None admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 10:43:03.859376 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 536/1093] 10.222.0.25 () {58 vars in 1024 bytes} [Sun Aug 7 10:43:03 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 07 10:43:03.882404 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0458fab6-0f44-4ecd-aacf-c112a4d78fb6 None None] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 10:43:03.941494 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e90d1cc-e257-4202-a028-8b7df84c0dcc None admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 10:43:03.957797 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=100 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 10:43:03.975047 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bbba12f-2537-473b-968d-6258aefee0ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 10:43:03.975655 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bbba12f-2537-473b-968d-6258aefee0ec None admin] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:43:03.975931 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bbba12f-2537-473b-968d-6258aefee0ec None admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:43:03.976201 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bbba12f-2537-473b-968d-6258aefee0ec None admin] PATH_INFO: `/v3/users` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:43:03.977116 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bbba12f-2537-473b-968d-6258aefee0ec None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80144) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 10:43:03.977428 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bbba12f-2537-473b-968d-6258aefee0ec None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 10:43:03.978826 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bbba12f-2537-473b-968d-6258aefee0ec None admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 10:43:03.984134 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0458fab6-0f44-4ecd-aacf-c112a4d78fb6 None None] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 10:43:03.995655 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 558/1094] 10.222.0.25 () {56 vars in 1133 bytes} [Sun Aug 7 10:43:03 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 07 10:43:04.001241 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bbba12f-2537-473b-968d-6258aefee0ec None admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 10:43:04.018082 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-191d4d8a-7623-47c5-8657-6503c70171f8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 10:43:04.018644 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-191d4d8a-7623-47c5-8657-6503c70171f8 None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:43:04.018951 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-191d4d8a-7623-47c5-8657-6503c70171f8 None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:43:04.019385 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-191d4d8a-7623-47c5-8657-6503c70171f8 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:43:04.019621 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=100 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 10:43:04.030698 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-191d4d8a-7623-47c5-8657-6503c70171f8 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 07 10:43:04.035361 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application [None req-191d4d8a-7623-47c5-8657-6503c70171f8 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 07 10:43:04.035361 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 07 10:43:04.035361 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 07 10:43:04.035361 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 07 10:43:04.035361 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 07 10:43:04.035361 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 07 10:43:04.035361 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 07 10:43:04.035361 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 07 10:43:04.035361 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 07 10:43:04.035361 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 07 10:43:04.035361 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 07 10:43:04.035361 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 07 10:43:04.035361 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 07 10:43:04.035361 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 07 10:43:04.035361 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 07 10:43:04.035361 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 07 10:43:04.036388 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 07 10:43:04.036388 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 07 10:43:04.036388 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 07 10:43:04.036388 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 07 10:43:04.036388 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 07 10:43:04.036388 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 07 10:43:04.036388 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 07 10:43:04.036388 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 07 10:43:04.036388 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 07 10:43:04.036388 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.get_or_create( Aug 07 10:43:04.036388 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 07 10:43:04.036388 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application with Lock( Aug 07 10:43:04.036388 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 07 10:43:04.036388 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self._enter() Aug 07 10:43:04.036388 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 07 10:43:04.036388 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 07 10:43:04.037233 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 07 10:43:04.037233 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.creator() Aug 07 10:43:04.037233 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 07 10:43:04.037233 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application created_value = creator( Aug 07 10:43:04.037233 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 07 10:43:04.037233 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 07 10:43:04.037233 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 07 10:43:04.037233 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 07 10:43:04.037233 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 07 10:43:04.037233 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 07 10:43:04.037233 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 07 10:43:04.037233 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application Aug 07 10:43:04.037233 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 537/1095] 10.222.0.25 () {56 vars in 1096 bytes} [Sun Aug 7 10:43:03 2022] GET /identity/v3/roles/admin => generated 83 bytes in 155 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 07 10:43:04.042286 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-191d4d8a-7623-47c5-8657-6503c70171f8 None admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 10:43:04.105388 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bbba12f-2537-473b-968d-6258aefee0ec None admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 10:43:04.140300 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-050abd9e-2ba4-42c5-af91-41d3c6dacbfe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 10:43:04.140964 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-191d4d8a-7623-47c5-8657-6503c70171f8 None admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 10:43:04.141073 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-050abd9e-2ba4-42c5-af91-41d3c6dacbfe None admin] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:43:04.141162 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-050abd9e-2ba4-42c5-af91-41d3c6dacbfe None admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:43:04.141419 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-050abd9e-2ba4-42c5-af91-41d3c6dacbfe None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:43:04.176530 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b1e999e-f88d-4419-af52-f763f188ced7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 10:43:04.177133 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b1e999e-f88d-4419-af52-f763f188ced7 None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:43:04.177415 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b1e999e-f88d-4419-af52-f763f188ced7 None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:43:04.177688 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b1e999e-f88d-4419-af52-f763f188ced7 None admin] PATH_INFO: `/v3/roles` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:43:04.178349 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b1e999e-f88d-4419-af52-f763f188ced7 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80143) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 10:43:04.178654 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b1e999e-f88d-4419-af52-f763f188ced7 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 10:43:04.180150 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b1e999e-f88d-4419-af52-f763f188ced7 None admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 10:43:04.181845 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=100 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 10:43:04.189037 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 538/1096] 10.222.0.25 () {56 vars in 1099 bytes} [Sun Aug 7 10:43:04 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 07 10:43:04.194718 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b1e999e-f88d-4419-af52-f763f188ced7 None admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 10:43:04.207970 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-050abd9e-2ba4-42c5-af91-41d3c6dacbfe None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 10:43:04.209452 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-050abd9e-2ba4-42c5-af91-41d3c6dacbfe None admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 10:43:04.227165 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=100 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 10:43:04.294833 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b1e999e-f88d-4419-af52-f763f188ced7 None admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 10:43:04.305722 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 559/1097] 10.222.0.25 () {58 vars in 1282 bytes} [Sun Aug 7 10:43:04 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 305 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 07 10:43:04.311819 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-050abd9e-2ba4-42c5-af91-41d3c6dacbfe None admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 10:43:04.329633 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bead63bb-7e05-4d69-acdb-be8323dfb2e0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 10:43:04.330259 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bead63bb-7e05-4d69-acdb-be8323dfb2e0 None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:43:04.330533 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bead63bb-7e05-4d69-acdb-be8323dfb2e0 None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:43:04.330802 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bead63bb-7e05-4d69-acdb-be8323dfb2e0 None admin] PATH_INFO: `/v3/users/736e3f99fbd24cb79b20f3b520c1fb56` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:43:04.331942 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bead63bb-7e05-4d69-acdb-be8323dfb2e0 None admin] RBAC: Authorizing `identity:get_user(user_id=736e3f99fbd24cb79b20f3b520c1fb56)` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 10:43:04.333350 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bead63bb-7e05-4d69-acdb-be8323dfb2e0 None admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 10:43:04.334904 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 539/1098] 10.222.0.25 () {56 vars in 1177 bytes} [Sun Aug 7 10:43:04 2022] GET /identity/v3/users/736e3f99fbd24cb79b20f3b520c1fb56 => generated 285 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 07 10:43:04.340603 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bead63bb-7e05-4d69-acdb-be8323dfb2e0 None admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 10:43:04.401627 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=100 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 10:43:04.412213 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-050abd9e-2ba4-42c5-af91-41d3c6dacbfe None admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 10:43:04.433706 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=100 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 10:43:04.442275 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bead63bb-7e05-4d69-acdb-be8323dfb2e0 None admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 10:43:04.451509 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-220bfd8a-2382-443a-9d24-20882ae3da2b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 10:43:04.452139 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-220bfd8a-2382-443a-9d24-20882ae3da2b None admin] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:43:04.452419 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-220bfd8a-2382-443a-9d24-20882ae3da2b None admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:43:04.452692 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-220bfd8a-2382-443a-9d24-20882ae3da2b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:43:04.463245 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-220bfd8a-2382-443a-9d24-20882ae3da2b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 10:43:04.464686 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-220bfd8a-2382-443a-9d24-20882ae3da2b None admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 10:43:04.475558 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7322f59f-58ba-4772-8072-2c8ed44d15a8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 10:43:04.476163 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7322f59f-58ba-4772-8072-2c8ed44d15a8 None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:43:04.476444 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7322f59f-58ba-4772-8072-2c8ed44d15a8 None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:43:04.476719 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7322f59f-58ba-4772-8072-2c8ed44d15a8 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:43:04.479400 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application [None req-220bfd8a-2382-443a-9d24-20882ae3da2b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 07 10:43:04.479400 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 07 10:43:04.479400 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 07 10:43:04.479400 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 07 10:43:04.479400 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 07 10:43:04.479400 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 07 10:43:04.479400 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 07 10:43:04.479400 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 07 10:43:04.479400 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 07 10:43:04.479400 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 07 10:43:04.479400 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 07 10:43:04.479400 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 07 10:43:04.479400 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 07 10:43:04.479400 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 07 10:43:04.479400 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 07 10:43:04.479400 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 07 10:43:04.480510 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 07 10:43:04.480510 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 07 10:43:04.480510 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 07 10:43:04.480510 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 07 10:43:04.480510 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 07 10:43:04.480510 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self.get_or_create( Aug 07 10:43:04.480510 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 07 10:43:04.480510 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application with Lock( Aug 07 10:43:04.480510 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 07 10:43:04.480510 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self._enter() Aug 07 10:43:04.480510 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 07 10:43:04.480510 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 07 10:43:04.480510 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 07 10:43:04.480510 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self.creator() Aug 07 10:43:04.480510 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 07 10:43:04.480510 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application created_value = creator( Aug 07 10:43:04.480510 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 07 10:43:04.481406 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 07 10:43:04.481406 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 07 10:43:04.481406 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application Aug 07 10:43:04.481406 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 560/1099] 10.222.0.25 () {56 vars in 1108 bytes} [Sun Aug 7 10:43:04 2022] GET /identity/v3/domains/Default => generated 87 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 07 10:43:04.486195 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-220bfd8a-2382-443a-9d24-20882ae3da2b None admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 10:43:04.544078 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7322f59f-58ba-4772-8072-2c8ed44d15a8 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 10:43:04.545580 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7322f59f-58ba-4772-8072-2c8ed44d15a8 None admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 10:43:04.583608 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-220bfd8a-2382-443a-9d24-20882ae3da2b None admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 10:43:04.605055 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=100 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 10:43:04.616231 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16bebd7d-439b-4f40-9343-4d4852d92a73 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 10:43:04.616833 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16bebd7d-439b-4f40-9343-4d4852d92a73 None admin] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:43:04.617110 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16bebd7d-439b-4f40-9343-4d4852d92a73 None admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:43:04.617380 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16bebd7d-439b-4f40-9343-4d4852d92a73 None admin] PATH_INFO: `/v3/domains` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:43:04.618653 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16bebd7d-439b-4f40-9343-4d4852d92a73 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80144) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 10:43:04.618986 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16bebd7d-439b-4f40-9343-4d4852d92a73 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 10:43:04.620321 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16bebd7d-439b-4f40-9343-4d4852d92a73 None admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 10:43:04.631571 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 561/1100] 10.222.0.25 () {56 vars in 1109 bytes} [Sun Aug 7 10:43:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 07 10:43:04.637958 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16bebd7d-439b-4f40-9343-4d4852d92a73 None admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 10:43:04.649560 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 540/1101] 10.222.0.25 () {58 vars in 1282 bytes} [Sun Aug 7 10:43:04 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 305 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Aug 07 10:43:04.650571 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7322f59f-58ba-4772-8072-2c8ed44d15a8 None admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 10:43:04.666887 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=100 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 10:43:04.736612 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16bebd7d-439b-4f40-9343-4d4852d92a73 None admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 10:43:04.749115 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7322f59f-58ba-4772-8072-2c8ed44d15a8 None admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 10:43:04.770248 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b80ca0da-b1e4-474c-97ee-1a41255004fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 10:43:04.770876 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b80ca0da-b1e4-474c-97ee-1a41255004fd None admin] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:43:04.771186 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b80ca0da-b1e4-474c-97ee-1a41255004fd None admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:43:04.771459 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b80ca0da-b1e4-474c-97ee-1a41255004fd None admin] PATH_INFO: `/v3/projects/service` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:43:04.778705 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b80ca0da-b1e4-474c-97ee-1a41255004fd None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 10:43:04.780176 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b80ca0da-b1e4-474c-97ee-1a41255004fd None admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 10:43:04.783386 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3d26787-ee67-4685-b5f6-24cc39f8fe6d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 10:43:04.784008 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3d26787-ee67-4685-b5f6-24cc39f8fe6d None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:43:04.784290 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3d26787-ee67-4685-b5f6-24cc39f8fe6d None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:43:04.784563 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3d26787-ee67-4685-b5f6-24cc39f8fe6d None admin] PATH_INFO: `/v3/projects/4a1780a74fca49a89f96d3281c44dbc3` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:43:04.785675 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3d26787-ee67-4685-b5f6-24cc39f8fe6d None admin] RBAC: Authorizing `identity:get_project(project_id=4a1780a74fca49a89f96d3281c44dbc3)` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 10:43:04.787125 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3d26787-ee67-4685-b5f6-24cc39f8fe6d None admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 10:43:04.788733 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 541/1102] 10.222.0.25 () {56 vars in 1186 bytes} [Sun Aug 7 10:43:04 2022] GET /identity/v3/projects/4a1780a74fca49a89f96d3281c44dbc3 => generated 311 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 10:43:04.791430 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application [None req-b80ca0da-b1e4-474c-97ee-1a41255004fd None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 07 10:43:04.791430 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 07 10:43:04.791430 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 07 10:43:04.791430 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 07 10:43:04.791430 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 07 10:43:04.791430 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 07 10:43:04.791430 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 07 10:43:04.791430 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 07 10:43:04.791430 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 07 10:43:04.791430 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 07 10:43:04.791430 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 07 10:43:04.791430 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 07 10:43:04.791430 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 07 10:43:04.791430 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 07 10:43:04.791430 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 07 10:43:04.792441 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 07 10:43:04.792441 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 07 10:43:04.792441 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 07 10:43:04.792441 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 07 10:43:04.792441 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 07 10:43:04.792441 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 07 10:43:04.792441 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self.get_or_create( Aug 07 10:43:04.792441 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 07 10:43:04.792441 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application with Lock( Aug 07 10:43:04.792441 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 07 10:43:04.792441 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self._enter() Aug 07 10:43:04.792441 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 07 10:43:04.792441 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 07 10:43:04.792441 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 07 10:43:04.792441 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self.creator() Aug 07 10:43:04.792441 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 07 10:43:04.792441 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application created_value = creator( Aug 07 10:43:04.793549 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 07 10:43:04.793549 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 07 10:43:04.793549 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 07 10:43:04.793549 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 07 10:43:04.793549 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 07 10:43:04.793549 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 07 10:43:04.793549 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 07 10:43:04.793549 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application Aug 07 10:43:04.794692 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3d26787-ee67-4685-b5f6-24cc39f8fe6d None admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 10:43:04.795070 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 562/1103] 10.222.0.25 () {56 vars in 1111 bytes} [Sun Aug 7 10:43:04 2022] GET /identity/v3/projects/service => generated 88 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 07 10:43:04.798288 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b80ca0da-b1e4-474c-97ee-1a41255004fd None admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 10:43:04.825671 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=100 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 10:43:04.885792 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=100 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 10:43:04.894970 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3d26787-ee67-4685-b5f6-24cc39f8fe6d None admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 10:43:04.895265 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b80ca0da-b1e4-474c-97ee-1a41255004fd None admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 10:43:04.928090 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5de0377-b4df-4014-ae10-d46a9a1a3171 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 10:43:04.928690 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5de0377-b4df-4014-ae10-d46a9a1a3171 None admin] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:43:04.928966 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5de0377-b4df-4014-ae10-d46a9a1a3171 None admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:43:04.929234 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5de0377-b4df-4014-ae10-d46a9a1a3171 None admin] PATH_INFO: `/v3/projects` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:43:04.929808 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-674835d6-27d2-450e-86eb-4eddd47dfb0e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 10:43:04.929917 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5de0377-b4df-4014-ae10-d46a9a1a3171 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80144) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 10:43:04.930875 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5de0377-b4df-4014-ae10-d46a9a1a3171 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 10:43:04.931234 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-674835d6-27d2-450e-86eb-4eddd47dfb0e None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:43:04.931574 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-674835d6-27d2-450e-86eb-4eddd47dfb0e None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:43:04.931851 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-674835d6-27d2-450e-86eb-4eddd47dfb0e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:43:04.932285 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5de0377-b4df-4014-ae10-d46a9a1a3171 None admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 10:43:04.932576 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-674835d6-27d2-450e-86eb-4eddd47dfb0e None admin] RBAC: Adding query filter params (role.id=81b1f1bcee2a473088fbbd327c1c5b23, scope.project.id=4a1780a74fca49a89f96d3281c44dbc3, user.id=736e3f99fbd24cb79b20f3b520c1fb56) {{(pid=80143) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 10:43:04.932885 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-674835d6-27d2-450e-86eb-4eddd47dfb0e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 10:43:04.934279 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-674835d6-27d2-450e-86eb-4eddd47dfb0e None admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 10:43:04.939919 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 542/1104] 10.222.0.25 () {56 vars in 1374 bytes} [Sun Aug 7 10:43:04 2022] GET /identity/v3/role_assignments?user.id=736e3f99fbd24cb79b20f3b520c1fb56&scope.project.id=4a1780a74fca49a89f96d3281c44dbc3&role.id=81b1f1bcee2a473088fbbd327c1c5b23 => generated 608 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 10:43:04.945632 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 563/1105] 10.222.0.25 () {56 vars in 1148 bytes} [Sun Aug 7 10:43:04 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 07 10:43:04.952011 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-674835d6-27d2-450e-86eb-4eddd47dfb0e None admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 10:43:05.053850 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=100 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 10:43:05.060845 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-674835d6-27d2-450e-86eb-4eddd47dfb0e None admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 10:43:05.096422 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc8be4d8-374e-40fc-af88-73e02a41e6c6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 10:43:05.097027 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc8be4d8-374e-40fc-af88-73e02a41e6c6 None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:43:05.097311 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc8be4d8-374e-40fc-af88-73e02a41e6c6 None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:43:05.097580 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc8be4d8-374e-40fc-af88-73e02a41e6c6 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:43:05.098291 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc8be4d8-374e-40fc-af88-73e02a41e6c6 None admin] RBAC: Adding query filter params (role.id=3f1b8463c86840fb848a4cdf7c3d98be, scope.project.id=6de1f3cac02c4bcdb05dd2337176033d, user.id=31d16d1b96104b4eb690398432b9829e) {{(pid=80143) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 10:43:05.099106 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc8be4d8-374e-40fc-af88-73e02a41e6c6 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 10:43:05.100035 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc8be4d8-374e-40fc-af88-73e02a41e6c6 None admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 10:43:05.105902 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 543/1106] 10.222.0.25 () {56 vars in 1374 bytes} [Sun Aug 7 10:43:04 2022] GET /identity/v3/role_assignments?user.id=31d16d1b96104b4eb690398432b9829e&scope.project.id=6de1f3cac02c4bcdb05dd2337176033d&role.id=3f1b8463c86840fb848a4cdf7c3d98be => generated 608 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 10:43:07.057453 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0acbc3e-5a52-4f9d-8092-0a0ea801a299 None None] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:43:07.057990 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0acbc3e-5a52-4f9d-8092-0a0ea801a299 None None] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:43:07.057990 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0acbc3e-5a52-4f9d-8092-0a0ea801a299 None None] PATH_INFO: `/` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:43:07.059342 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 564/1107] 10.222.0.25 () {54 vars in 927 bytes} [Sun Aug 7 10:43:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 07 10:43:07.064380 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55d01f29-9d20-476c-ac93-d918bb00e54d None None] REQUEST_METHOD: `POST` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:43:07.064802 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55d01f29-9d20-476c-ac93-d918bb00e54d None None] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:43:07.064934 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55d01f29-9d20-476c-ac93-d918bb00e54d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:43:07.084715 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.auth.core [None req-55d01f29-9d20-476c-ac93-d918bb00e54d None None] MFA Rules not processed for user `bac4e6b76fe64217ac8fa44e71cffe6a`. Rule list: `[]` (Enabled: `True`). {{(pid=80143) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 10:43:07.086093 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-55d01f29-9d20-476c-ac93-d918bb00e54d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 10:43:07.152679 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 544/1108] 10.222.0.25 () {58 vars in 1024 bytes} [Sun Aug 7 10:43:07 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 07 10:43:07.176606 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0acbc3e-5a52-4f9d-8092-0a0ea801a299 None None] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 10:43:07.275634 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0acbc3e-5a52-4f9d-8092-0a0ea801a299 None None] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 10:43:07.308937 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a88ebeb-cc35-4512-a6ee-62661620bee2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 10:43:07.309484 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a88ebeb-cc35-4512-a6ee-62661620bee2 None admin] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:43:07.309758 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a88ebeb-cc35-4512-a6ee-62661620bee2 None admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:43:07.310026 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a88ebeb-cc35-4512-a6ee-62661620bee2 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:43:07.311135 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a88ebeb-cc35-4512-a6ee-62661620bee2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 10:43:07.312542 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a88ebeb-cc35-4512-a6ee-62661620bee2 None admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 10:43:07.314069 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 565/1109] 10.222.0.25 () {56 vars in 1108 bytes} [Sun Aug 7 10:43:07 2022] GET /identity/v3/domains/default => generated 199 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 10:43:07.319858 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55d01f29-9d20-476c-ac93-d918bb00e54d None None] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 10:43:07.406641 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=100 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 10:43:07.421590 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55d01f29-9d20-476c-ac93-d918bb00e54d None None] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 10:43:07.455600 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8098e486-7988-4897-9fd4-1ce4bbeba775 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 10:43:07.456203 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8098e486-7988-4897-9fd4-1ce4bbeba775 None admin] REQUEST_METHOD: `POST` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:43:07.456488 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8098e486-7988-4897-9fd4-1ce4bbeba775 None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:43:07.456762 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8098e486-7988-4897-9fd4-1ce4bbeba775 None admin] PATH_INFO: `/v3/users` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:43:07.457412 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8098e486-7988-4897-9fd4-1ce4bbeba775 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 10:43:07.458823 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8098e486-7988-4897-9fd4-1ce4bbeba775 None admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 10:43:07.480890 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 545/1110] 10.222.0.25 () {60 vars in 1132 bytes} [Sun Aug 7 10:43:07 2022] POST /identity/v3/users => generated 286 bytes in 161 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 07 10:43:09.495029 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-382077d8-58d8-45f8-98de-ab604fc4338c None None] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:43:09.495543 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-382077d8-58d8-45f8-98de-ab604fc4338c None None] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:43:09.495543 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-382077d8-58d8-45f8-98de-ab604fc4338c None None] PATH_INFO: `/` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:43:09.496890 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 566/1111] 10.222.0.25 () {54 vars in 927 bytes} [Sun Aug 7 10:43:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 07 10:43:09.501872 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ff502d6-f5a5-4778-8931-e61365d51b3c None None] REQUEST_METHOD: `POST` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:43:09.502288 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ff502d6-f5a5-4778-8931-e61365d51b3c None None] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:43:09.502415 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ff502d6-f5a5-4778-8931-e61365d51b3c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:43:09.519932 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.auth.core [None req-3ff502d6-f5a5-4778-8931-e61365d51b3c None None] MFA Rules not processed for user `bac4e6b76fe64217ac8fa44e71cffe6a`. Rule list: `[]` (Enabled: `True`). {{(pid=80143) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 10:43:09.521092 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-3ff502d6-f5a5-4778-8931-e61365d51b3c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 10:43:09.586756 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 546/1112] 10.222.0.25 () {58 vars in 1024 bytes} [Sun Aug 7 10:43:09 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 07 10:43:09.609795 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-382077d8-58d8-45f8-98de-ab604fc4338c None None] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 10:43:09.628494 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=100 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 10:43:09.707326 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-382077d8-58d8-45f8-98de-ab604fc4338c None None] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 10:43:09.739992 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8c5ab64-77b2-4a9f-92ba-e4311718b0cd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 10:43:09.740608 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8c5ab64-77b2-4a9f-92ba-e4311718b0cd None admin] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:43:09.740887 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8c5ab64-77b2-4a9f-92ba-e4311718b0cd None admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:43:09.741158 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8c5ab64-77b2-4a9f-92ba-e4311718b0cd None admin] PATH_INFO: `/v3/roles/1512d531ebe248ea8e2d7d56d754b230` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:43:09.743171 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8c5ab64-77b2-4a9f-92ba-e4311718b0cd None admin] RBAC: Authorizing `identity:get_role(role_id=1512d531ebe248ea8e2d7d56d754b230)` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 10:43:09.744490 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8c5ab64-77b2-4a9f-92ba-e4311718b0cd None admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 10:43:09.745779 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 567/1113] 10.222.0.25 () {56 vars in 1177 bytes} [Sun Aug 7 10:43:09 2022] GET /identity/v3/roles/1512d531ebe248ea8e2d7d56d754b230 => generated 223 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 10:43:09.751584 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ff502d6-f5a5-4778-8931-e61365d51b3c None None] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 10:43:09.824977 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 10:43:09.847543 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ff502d6-f5a5-4778-8931-e61365d51b3c None None] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 10:43:09.880304 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92a8127a-03ac-490c-8574-df4c129cadb9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 10:43:09.880920 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92a8127a-03ac-490c-8574-df4c129cadb9 None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:43:09.881200 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92a8127a-03ac-490c-8574-df4c129cadb9 None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:43:09.881472 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92a8127a-03ac-490c-8574-df4c129cadb9 None admin] PATH_INFO: `/v3/users/9183f5bd6ad2497a85a0c0ccffce4981` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:43:09.894441 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92a8127a-03ac-490c-8574-df4c129cadb9 None admin] RBAC: Authorizing `identity:get_user(user_id=9183f5bd6ad2497a85a0c0ccffce4981)` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 10:43:09.895903 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92a8127a-03ac-490c-8574-df4c129cadb9 None admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 10:43:09.897472 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 547/1114] 10.222.0.25 () {56 vars in 1177 bytes} [Sun Aug 7 10:43:09 2022] GET /identity/v3/users/9183f5bd6ad2497a85a0c0ccffce4981 => generated 286 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 10:43:09.903539 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8c5ab64-77b2-4a9f-92ba-e4311718b0cd None admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 10:43:09.985250 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=100 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 10:43:10.000203 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8c5ab64-77b2-4a9f-92ba-e4311718b0cd None admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 10:43:10.031833 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21ec6abc-d7e3-4c4e-9137-9a4407e03ce3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 10:43:10.032419 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21ec6abc-d7e3-4c4e-9137-9a4407e03ce3 None admin] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:43:10.032699 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21ec6abc-d7e3-4c4e-9137-9a4407e03ce3 None admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:43:10.032972 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21ec6abc-d7e3-4c4e-9137-9a4407e03ce3 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:43:10.098746 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21ec6abc-d7e3-4c4e-9137-9a4407e03ce3 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 10:43:10.100192 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21ec6abc-d7e3-4c4e-9137-9a4407e03ce3 None admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 10:43:10.178734 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=100 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 10:43:10.193809 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 568/1115] 10.222.0.25 () {58 vars in 1282 bytes} [Sun Aug 7 10:43:09 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 291 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 07 10:43:10.200191 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92a8127a-03ac-490c-8574-df4c129cadb9 None admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 10:43:10.295678 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92a8127a-03ac-490c-8574-df4c129cadb9 None admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 10:43:10.329257 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f4587a2-fa1e-4d0e-b77e-7eeedc00a4c5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 10:43:10.329879 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f4587a2-fa1e-4d0e-b77e-7eeedc00a4c5 None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:43:10.330163 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f4587a2-fa1e-4d0e-b77e-7eeedc00a4c5 None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:43:10.330438 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f4587a2-fa1e-4d0e-b77e-7eeedc00a4c5 None admin] PATH_INFO: `/v3/projects/4a1780a74fca49a89f96d3281c44dbc3` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:43:10.331561 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f4587a2-fa1e-4d0e-b77e-7eeedc00a4c5 None admin] RBAC: Authorizing `identity:get_project(project_id=4a1780a74fca49a89f96d3281c44dbc3)` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 10:43:10.332983 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f4587a2-fa1e-4d0e-b77e-7eeedc00a4c5 None admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 10:43:10.334595 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 548/1116] 10.222.0.25 () {56 vars in 1186 bytes} [Sun Aug 7 10:43:10 2022] GET /identity/v3/projects/4a1780a74fca49a89f96d3281c44dbc3 => generated 311 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 10:43:10.341275 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21ec6abc-d7e3-4c4e-9137-9a4407e03ce3 None admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 10:43:10.434819 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21ec6abc-d7e3-4c4e-9137-9a4407e03ce3 None admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 10:43:10.467066 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a751b0a3-268c-4f64-a389-10da3afb6d08 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 10:43:10.467660 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a751b0a3-268c-4f64-a389-10da3afb6d08 None admin] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:43:10.467937 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a751b0a3-268c-4f64-a389-10da3afb6d08 None admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:43:10.468206 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a751b0a3-268c-4f64-a389-10da3afb6d08 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:43:10.468912 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a751b0a3-268c-4f64-a389-10da3afb6d08 None admin] RBAC: Adding query filter params (role.id=1512d531ebe248ea8e2d7d56d754b230, scope.project.id=4a1780a74fca49a89f96d3281c44dbc3, user.id=9183f5bd6ad2497a85a0c0ccffce4981) {{(pid=80144) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 10:43:10.469215 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a751b0a3-268c-4f64-a389-10da3afb6d08 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 10:43:10.470596 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a751b0a3-268c-4f64-a389-10da3afb6d08 None admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 10:43:10.475742 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 569/1117] 10.222.0.25 () {56 vars in 1374 bytes} [Sun Aug 7 10:43:10 2022] GET /identity/v3/role_assignments?user.id=9183f5bd6ad2497a85a0c0ccffce4981&scope.project.id=4a1780a74fca49a89f96d3281c44dbc3&role.id=1512d531ebe248ea8e2d7d56d754b230 => generated 259 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 10:43:12.411792 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32f20dfd-41e7-42c8-81e8-47074fb9826b None None] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:43:12.412305 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32f20dfd-41e7-42c8-81e8-47074fb9826b None None] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:43:12.412397 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32f20dfd-41e7-42c8-81e8-47074fb9826b None None] PATH_INFO: `/` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:43:12.413704 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 549/1118] 10.222.0.25 () {54 vars in 927 bytes} [Sun Aug 7 10:43:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 07 10:43:12.419286 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb6ad743-ad10-4219-8821-63108a76daa3 None None] REQUEST_METHOD: `POST` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:43:12.419706 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb6ad743-ad10-4219-8821-63108a76daa3 None None] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:43:12.419815 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb6ad743-ad10-4219-8821-63108a76daa3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:43:12.437940 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.auth.core [None req-bb6ad743-ad10-4219-8821-63108a76daa3 None None] MFA Rules not processed for user `bac4e6b76fe64217ac8fa44e71cffe6a`. Rule list: `[]` (Enabled: `True`). {{(pid=80144) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 10:43:12.439144 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.fernet_utils [None req-bb6ad743-ad10-4219-8821-63108a76daa3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80144) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 10:43:12.484376 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=100 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 10:43:12.505349 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 570/1119] 10.222.0.25 () {58 vars in 1024 bytes} [Sun Aug 7 10:43:12 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 07 10:43:12.527893 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32f20dfd-41e7-42c8-81e8-47074fb9826b None None] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 10:43:12.530730 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=100 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 10:43:12.627027 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32f20dfd-41e7-42c8-81e8-47074fb9826b None None] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 10:43:12.660008 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80370b4d-7738-434a-85ce-dd9daa32465a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 10:43:12.660628 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80370b4d-7738-434a-85ce-dd9daa32465a None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:43:12.660908 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80370b4d-7738-434a-85ce-dd9daa32465a None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:43:12.661180 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80370b4d-7738-434a-85ce-dd9daa32465a None admin] PATH_INFO: `/v3/roles/1512d531ebe248ea8e2d7d56d754b230` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:43:12.662610 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80370b4d-7738-434a-85ce-dd9daa32465a None admin] RBAC: Authorizing `identity:get_role(role_id=1512d531ebe248ea8e2d7d56d754b230)` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 10:43:12.663942 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80370b4d-7738-434a-85ce-dd9daa32465a None admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 10:43:12.665791 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 550/1120] 10.222.0.25 () {56 vars in 1177 bytes} [Sun Aug 7 10:43:12 2022] GET /identity/v3/roles/1512d531ebe248ea8e2d7d56d754b230 => generated 223 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 10:43:12.671496 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb6ad743-ad10-4219-8821-63108a76daa3 None None] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 10:43:12.766308 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb6ad743-ad10-4219-8821-63108a76daa3 None None] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 10:43:12.799282 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1e865c3-565f-471c-ae90-0ba20683ecc8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 10:43:12.799903 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1e865c3-565f-471c-ae90-0ba20683ecc8 None admin] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:43:12.800185 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1e865c3-565f-471c-ae90-0ba20683ecc8 None admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:43:12.800461 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1e865c3-565f-471c-ae90-0ba20683ecc8 None admin] PATH_INFO: `/v3/users/9183f5bd6ad2497a85a0c0ccffce4981` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:43:12.801565 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1e865c3-565f-471c-ae90-0ba20683ecc8 None admin] RBAC: Authorizing `identity:get_user(user_id=9183f5bd6ad2497a85a0c0ccffce4981)` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 10:43:12.802997 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1e865c3-565f-471c-ae90-0ba20683ecc8 None admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 10:43:12.804539 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 571/1121] 10.222.0.25 () {56 vars in 1177 bytes} [Sun Aug 7 10:43:12 2022] GET /identity/v3/users/9183f5bd6ad2497a85a0c0ccffce4981 => generated 286 bytes in 134 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 10:43:12.810248 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80370b4d-7738-434a-85ce-dd9daa32465a None admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 10:43:12.877681 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=100 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 10:43:12.908511 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80370b4d-7738-434a-85ce-dd9daa32465a None admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 10:43:12.940826 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7e39442-1dfa-49d2-b49b-c1c01a7fe2e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 10:43:12.941380 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7e39442-1dfa-49d2-b49b-c1c01a7fe2e6 None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:43:12.941654 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7e39442-1dfa-49d2-b49b-c1c01a7fe2e6 None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:43:12.941924 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7e39442-1dfa-49d2-b49b-c1c01a7fe2e6 None admin] PATH_INFO: `/v3/projects/4a1780a74fca49a89f96d3281c44dbc3` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:43:12.943044 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7e39442-1dfa-49d2-b49b-c1c01a7fe2e6 None admin] RBAC: Authorizing `identity:get_project(project_id=4a1780a74fca49a89f96d3281c44dbc3)` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 10:43:12.944456 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7e39442-1dfa-49d2-b49b-c1c01a7fe2e6 None admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 10:43:12.946683 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 551/1122] 10.222.0.25 () {56 vars in 1186 bytes} [Sun Aug 7 10:43:12 2022] GET /identity/v3/projects/4a1780a74fca49a89f96d3281c44dbc3 => generated 311 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 07 10:43:12.952679 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1e865c3-565f-471c-ae90-0ba20683ecc8 None admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 10:43:13.002375 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=100 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 10:43:13.052468 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1e865c3-565f-471c-ae90-0ba20683ecc8 None admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 10:43:13.084733 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-958f0d51-a5fc-4fc2-8a63-c78aca22187b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 10:43:13.085379 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-958f0d51-a5fc-4fc2-8a63-c78aca22187b None admin] REQUEST_METHOD: `PUT` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:43:13.085654 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-958f0d51-a5fc-4fc2-8a63-c78aca22187b None admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:43:13.085921 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-958f0d51-a5fc-4fc2-8a63-c78aca22187b None admin] PATH_INFO: `/v3/projects/4a1780a74fca49a89f96d3281c44dbc3/users/9183f5bd6ad2497a85a0c0ccffce4981/roles/1512d531ebe248ea8e2d7d56d754b230` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:43:13.088702 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-958f0d51-a5fc-4fc2-8a63-c78aca22187b None admin] RBAC: Authorizing `identity:create_grant(project_id=4a1780a74fca49a89f96d3281c44dbc3, user_id=9183f5bd6ad2497a85a0c0ccffce4981, role_id=1512d531ebe248ea8e2d7d56d754b230)` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 10:43:13.090093 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-958f0d51-a5fc-4fc2-8a63-c78aca22187b None admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 10:43:13.101903 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 572/1123] 10.222.0.25 () {58 vars in 1439 bytes} [Sun Aug 7 10:43:12 2022] PUT /identity/v3/projects/4a1780a74fca49a89f96d3281c44dbc3/users/9183f5bd6ad2497a85a0c0ccffce4981/roles/1512d531ebe248ea8e2d7d56d754b230 => generated 0 bytes in 150 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 07 10:43:15.057375 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-201ae391-1e76-4a4d-b063-475001c9fe24 None None] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:43:15.058011 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-201ae391-1e76-4a4d-b063-475001c9fe24 None None] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:43:15.058011 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-201ae391-1e76-4a4d-b063-475001c9fe24 None None] PATH_INFO: `/` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:43:15.059280 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 552/1124] 10.222.0.25 () {54 vars in 927 bytes} [Sun Aug 7 10:43:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 07 10:43:15.064525 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8feb2ea8-8a82-4fe2-ba70-e9aed493003c None None] REQUEST_METHOD: `POST` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:43:15.064943 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8feb2ea8-8a82-4fe2-ba70-e9aed493003c None None] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:43:15.065082 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8feb2ea8-8a82-4fe2-ba70-e9aed493003c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:43:15.083897 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.auth.core [None req-8feb2ea8-8a82-4fe2-ba70-e9aed493003c None None] MFA Rules not processed for user `bac4e6b76fe64217ac8fa44e71cffe6a`. Rule list: `[]` (Enabled: `True`). {{(pid=80144) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 10:43:15.085065 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.fernet_utils [None req-8feb2ea8-8a82-4fe2-ba70-e9aed493003c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80144) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 10:43:15.151084 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 573/1125] 10.222.0.25 () {58 vars in 1024 bytes} [Sun Aug 7 10:43:15 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 07 10:43:15.175306 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-201ae391-1e76-4a4d-b063-475001c9fe24 None None] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 10:43:15.274005 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-201ae391-1e76-4a4d-b063-475001c9fe24 None None] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 10:43:15.306916 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7102899d-7b39-4a73-800e-750ca17384d3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 10:43:15.307608 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7102899d-7b39-4a73-800e-750ca17384d3 None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:43:15.307839 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7102899d-7b39-4a73-800e-750ca17384d3 None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:43:15.308169 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7102899d-7b39-4a73-800e-750ca17384d3 None admin] PATH_INFO: `/v3/roles/1512d531ebe248ea8e2d7d56d754b230` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:43:15.309493 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7102899d-7b39-4a73-800e-750ca17384d3 None admin] RBAC: Authorizing `identity:get_role(role_id=1512d531ebe248ea8e2d7d56d754b230)` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 10:43:15.310839 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7102899d-7b39-4a73-800e-750ca17384d3 None admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 10:43:15.312155 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 553/1126] 10.222.0.25 () {56 vars in 1177 bytes} [Sun Aug 7 10:43:15 2022] GET /identity/v3/roles/1512d531ebe248ea8e2d7d56d754b230 => generated 223 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 10:43:15.318034 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8feb2ea8-8a82-4fe2-ba70-e9aed493003c None None] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 10:43:15.355964 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 10:43:15.414820 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8feb2ea8-8a82-4fe2-ba70-e9aed493003c None None] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 10:43:15.448115 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d546860-54d6-4dd0-8da2-f6c8f67a8565 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 10:43:15.448740 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d546860-54d6-4dd0-8da2-f6c8f67a8565 None admin] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:43:15.449023 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d546860-54d6-4dd0-8da2-f6c8f67a8565 None admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:43:15.449300 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d546860-54d6-4dd0-8da2-f6c8f67a8565 None admin] PATH_INFO: `/v3/users/9183f5bd6ad2497a85a0c0ccffce4981` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:43:15.450418 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d546860-54d6-4dd0-8da2-f6c8f67a8565 None admin] RBAC: Authorizing `identity:get_user(user_id=9183f5bd6ad2497a85a0c0ccffce4981)` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 10:43:15.451847 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d546860-54d6-4dd0-8da2-f6c8f67a8565 None admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 10:43:15.453418 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 574/1127] 10.222.0.25 () {56 vars in 1177 bytes} [Sun Aug 7 10:43:15 2022] GET /identity/v3/users/9183f5bd6ad2497a85a0c0ccffce4981 => generated 286 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 10:43:15.459166 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7102899d-7b39-4a73-800e-750ca17384d3 None admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 10:43:15.467915 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=100 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 10:43:15.560949 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7102899d-7b39-4a73-800e-750ca17384d3 None admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 10:43:15.593105 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac4e2ac7-f8b9-4fb9-95aa-0d25cd83de98 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 10:43:15.593702 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac4e2ac7-f8b9-4fb9-95aa-0d25cd83de98 None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:43:15.593976 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac4e2ac7-f8b9-4fb9-95aa-0d25cd83de98 None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:43:15.594243 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac4e2ac7-f8b9-4fb9-95aa-0d25cd83de98 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:43:15.658439 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac4e2ac7-f8b9-4fb9-95aa-0d25cd83de98 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 10:43:15.659880 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac4e2ac7-f8b9-4fb9-95aa-0d25cd83de98 None admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 10:43:15.667603 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=100 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 10:43:15.753806 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 554/1128] 10.222.0.25 () {58 vars in 1282 bytes} [Sun Aug 7 10:43:15 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 295 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 07 10:43:15.759778 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d546860-54d6-4dd0-8da2-f6c8f67a8565 None admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 10:43:15.855350 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d546860-54d6-4dd0-8da2-f6c8f67a8565 None admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 10:43:15.861881 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=100 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 10:43:15.888613 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00cb5f7a-f4ce-4c66-9182-d091e5c2fe27 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 10:43:15.889229 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00cb5f7a-f4ce-4c66-9182-d091e5c2fe27 None admin] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:43:15.889506 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00cb5f7a-f4ce-4c66-9182-d091e5c2fe27 None admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:43:15.889776 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00cb5f7a-f4ce-4c66-9182-d091e5c2fe27 None admin] PATH_INFO: `/v3/projects/4a1780a74fca49a89f96d3281c44dbc3` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:43:15.890862 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00cb5f7a-f4ce-4c66-9182-d091e5c2fe27 None admin] RBAC: Authorizing `identity:get_project(project_id=4a1780a74fca49a89f96d3281c44dbc3)` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 10:43:15.892287 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00cb5f7a-f4ce-4c66-9182-d091e5c2fe27 None admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 10:43:15.893876 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 575/1129] 10.222.0.25 () {56 vars in 1186 bytes} [Sun Aug 7 10:43:15 2022] GET /identity/v3/projects/4a1780a74fca49a89f96d3281c44dbc3 => generated 311 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 10:43:15.900721 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac4e2ac7-f8b9-4fb9-95aa-0d25cd83de98 None admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 10:43:15.994391 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac4e2ac7-f8b9-4fb9-95aa-0d25cd83de98 None admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 10:43:16.001394 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=100 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 10:43:16.027820 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5d3749b-81d8-4361-bf17-5f159a819c88 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 10:43:16.028407 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5d3749b-81d8-4361-bf17-5f159a819c88 None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:43:16.028684 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5d3749b-81d8-4361-bf17-5f159a819c88 None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:43:16.028955 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5d3749b-81d8-4361-bf17-5f159a819c88 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:43:16.029665 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5d3749b-81d8-4361-bf17-5f159a819c88 None admin] RBAC: Adding query filter params (role.id=1512d531ebe248ea8e2d7d56d754b230, scope.project.id=4a1780a74fca49a89f96d3281c44dbc3, user.id=9183f5bd6ad2497a85a0c0ccffce4981) {{(pid=80143) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 10:43:16.029970 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5d3749b-81d8-4361-bf17-5f159a819c88 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 10:43:16.031403 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5d3749b-81d8-4361-bf17-5f159a819c88 None admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 10:43:16.036835 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 555/1130] 10.222.0.25 () {56 vars in 1374 bytes} [Sun Aug 7 10:43:15 2022] GET /identity/v3/role_assignments?user.id=9183f5bd6ad2497a85a0c0ccffce4981&scope.project.id=4a1780a74fca49a89f96d3281c44dbc3&role.id=1512d531ebe248ea8e2d7d56d754b230 => generated 608 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 10:43:18.063409 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fb21030-a250-434d-ae8f-5fd47d6e0c51 None None] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:43:18.063938 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fb21030-a250-434d-ae8f-5fd47d6e0c51 None None] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:43:18.063938 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fb21030-a250-434d-ae8f-5fd47d6e0c51 None None] PATH_INFO: `/` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:43:18.065267 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 576/1131] 10.222.0.25 () {54 vars in 927 bytes} [Sun Aug 7 10:43:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 07 10:43:18.070836 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-891d3b52-e2bf-4b7c-9436-4e99d66414bc None None] REQUEST_METHOD: `POST` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:43:18.071298 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-891d3b52-e2bf-4b7c-9436-4e99d66414bc None None] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:43:18.071477 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-891d3b52-e2bf-4b7c-9436-4e99d66414bc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:43:18.090557 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.auth.core [None req-891d3b52-e2bf-4b7c-9436-4e99d66414bc None None] MFA Rules not processed for user `bac4e6b76fe64217ac8fa44e71cffe6a`. Rule list: `[]` (Enabled: `True`). {{(pid=80143) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 10:43:18.092570 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-891d3b52-e2bf-4b7c-9436-4e99d66414bc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 10:43:18.159602 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 556/1132] 10.222.0.25 () {58 vars in 1024 bytes} [Sun Aug 7 10:43:18 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 07 10:43:18.183741 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fb21030-a250-434d-ae8f-5fd47d6e0c51 None None] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 10:43:18.280560 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fb21030-a250-434d-ae8f-5fd47d6e0c51 None None] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 10:43:18.313193 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acfab25a-cd56-481c-9307-170023c4244a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 10:43:18.313856 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acfab25a-cd56-481c-9307-170023c4244a None admin] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:43:18.314081 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acfab25a-cd56-481c-9307-170023c4244a None admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:43:18.314351 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acfab25a-cd56-481c-9307-170023c4244a None admin] PATH_INFO: `/v3/domains/default` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:43:18.315458 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acfab25a-cd56-481c-9307-170023c4244a None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 10:43:18.316863 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acfab25a-cd56-481c-9307-170023c4244a None admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 10:43:18.318374 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 577/1133] 10.222.0.25 () {56 vars in 1108 bytes} [Sun Aug 7 10:43:18 2022] GET /identity/v3/domains/default => generated 199 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 10:43:18.324781 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-891d3b52-e2bf-4b7c-9436-4e99d66414bc None None] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 10:43:18.417193 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=100 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 10:43:18.420378 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-891d3b52-e2bf-4b7c-9436-4e99d66414bc None None] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 10:43:18.454809 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d4eceaa-b1af-46fc-b27d-f4e5ca86ee56 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 10:43:18.455443 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d4eceaa-b1af-46fc-b27d-f4e5ca86ee56 None admin] REQUEST_METHOD: `POST` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:43:18.455724 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d4eceaa-b1af-46fc-b27d-f4e5ca86ee56 None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:43:18.455997 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d4eceaa-b1af-46fc-b27d-f4e5ca86ee56 None admin] PATH_INFO: `/v3/projects` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:43:18.456632 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d4eceaa-b1af-46fc-b27d-f4e5ca86ee56 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 10:43:18.458046 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d4eceaa-b1af-46fc-b27d-f4e5ca86ee56 None admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 10:43:18.484250 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 557/1134] 10.222.0.25 () {60 vars in 1141 bytes} [Sun Aug 7 10:43:18 2022] POST /identity/v3/projects => generated 311 bytes in 160 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 07 10:43:20.484903 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77f459c3-b824-4d88-8826-aa9502351c61 None None] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:43:20.485400 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77f459c3-b824-4d88-8826-aa9502351c61 None None] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:43:20.485504 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77f459c3-b824-4d88-8826-aa9502351c61 None None] PATH_INFO: `/` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:43:20.486760 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 578/1135] 10.222.0.25 () {54 vars in 927 bytes} [Sun Aug 7 10:43:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 07 10:43:20.492874 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c711cd27-f664-4103-bb2d-eaa1b1713038 None None] REQUEST_METHOD: `POST` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:43:20.493274 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c711cd27-f664-4103-bb2d-eaa1b1713038 None None] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:43:20.493516 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c711cd27-f664-4103-bb2d-eaa1b1713038 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:43:20.511358 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.auth.core [None req-c711cd27-f664-4103-bb2d-eaa1b1713038 None None] MFA Rules not processed for user `bac4e6b76fe64217ac8fa44e71cffe6a`. Rule list: `[]` (Enabled: `True`). {{(pid=80143) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 10:43:20.512534 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-c711cd27-f664-4103-bb2d-eaa1b1713038 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 10:43:20.577524 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 558/1136] 10.222.0.25 () {58 vars in 1024 bytes} [Sun Aug 7 10:43:20 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 07 10:43:20.602363 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77f459c3-b824-4d88-8826-aa9502351c61 None None] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 10:43:20.645370 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=100 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 10:43:20.700311 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77f459c3-b824-4d88-8826-aa9502351c61 None None] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 10:43:20.732539 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4123d636-dedc-42b8-a960-ac39f4ca7532 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 10:43:20.733148 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4123d636-dedc-42b8-a960-ac39f4ca7532 None admin] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:43:20.733422 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4123d636-dedc-42b8-a960-ac39f4ca7532 None admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:43:20.733692 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4123d636-dedc-42b8-a960-ac39f4ca7532 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:43:20.734775 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4123d636-dedc-42b8-a960-ac39f4ca7532 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 10:43:20.736203 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4123d636-dedc-42b8-a960-ac39f4ca7532 None admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 10:43:20.737724 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 579/1137] 10.222.0.25 () {56 vars in 1108 bytes} [Sun Aug 7 10:43:20 2022] GET /identity/v3/domains/default => generated 199 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 10:43:20.744419 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c711cd27-f664-4103-bb2d-eaa1b1713038 None None] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 10:43:20.821290 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 10:43:20.839428 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c711cd27-f664-4103-bb2d-eaa1b1713038 None None] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 10:43:20.872270 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3d6cbf5-cbf3-4c71-9e76-3b5d6957da04 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 10:43:20.872861 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3d6cbf5-cbf3-4c71-9e76-3b5d6957da04 None admin] REQUEST_METHOD: `POST` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:43:20.873140 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3d6cbf5-cbf3-4c71-9e76-3b5d6957da04 None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:43:20.873413 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3d6cbf5-cbf3-4c71-9e76-3b5d6957da04 None admin] PATH_INFO: `/v3/users` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:43:20.874056 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3d6cbf5-cbf3-4c71-9e76-3b5d6957da04 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 10:43:20.875499 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3d6cbf5-cbf3-4c71-9e76-3b5d6957da04 None admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 10:43:20.894888 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 559/1138] 10.222.0.25 () {60 vars in 1132 bytes} [Sun Aug 7 10:43:20 2022] POST /identity/v3/users => generated 286 bytes in 151 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 07 10:43:22.925965 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15c841df-6fac-474c-ba8d-e97ab98b5c31 None None] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:43:22.926494 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15c841df-6fac-474c-ba8d-e97ab98b5c31 None None] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:43:22.926494 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15c841df-6fac-474c-ba8d-e97ab98b5c31 None None] PATH_INFO: `/` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:43:22.927867 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 580/1139] 10.222.0.25 () {54 vars in 927 bytes} [Sun Aug 7 10:43:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 07 10:43:22.933156 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04f628c8-3783-4ac9-b452-c04a5d9d8d38 None None] REQUEST_METHOD: `POST` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:43:22.933551 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04f628c8-3783-4ac9-b452-c04a5d9d8d38 None None] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:43:22.933710 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04f628c8-3783-4ac9-b452-c04a5d9d8d38 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:43:22.952058 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.auth.core [None req-04f628c8-3783-4ac9-b452-c04a5d9d8d38 None None] MFA Rules not processed for user `bac4e6b76fe64217ac8fa44e71cffe6a`. Rule list: `[]` (Enabled: `True`). {{(pid=80143) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 10:43:22.953243 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-04f628c8-3783-4ac9-b452-c04a5d9d8d38 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 10:43:23.017712 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 560/1140] 10.222.0.25 () {58 vars in 1024 bytes} [Sun Aug 7 10:43:22 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 07 10:43:23.041018 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15c841df-6fac-474c-ba8d-e97ab98b5c31 None None] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 10:43:23.141189 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15c841df-6fac-474c-ba8d-e97ab98b5c31 None None] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 10:43:23.151697 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=100 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 10:43:23.174511 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc36bb34-9140-4ebd-87e8-31ccb86d601f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 10:43:23.175152 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc36bb34-9140-4ebd-87e8-31ccb86d601f None admin] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:43:23.175436 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc36bb34-9140-4ebd-87e8-31ccb86d601f None admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:43:23.175709 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc36bb34-9140-4ebd-87e8-31ccb86d601f None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:43:23.186281 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc36bb34-9140-4ebd-87e8-31ccb86d601f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 07 10:43:23.190819 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application [None req-cc36bb34-9140-4ebd-87e8-31ccb86d601f None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 07 10:43:23.190819 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 07 10:43:23.190819 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 07 10:43:23.190819 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 07 10:43:23.190819 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 07 10:43:23.190819 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 07 10:43:23.190819 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 07 10:43:23.190819 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 07 10:43:23.190819 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 07 10:43:23.190819 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 07 10:43:23.190819 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 07 10:43:23.190819 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 07 10:43:23.190819 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 07 10:43:23.190819 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 07 10:43:23.190819 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 07 10:43:23.190819 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 07 10:43:23.191981 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 07 10:43:23.191981 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 07 10:43:23.191981 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 07 10:43:23.191981 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 07 10:43:23.191981 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 07 10:43:23.191981 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 07 10:43:23.191981 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 07 10:43:23.191981 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 07 10:43:23.191981 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 07 10:43:23.191981 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self.get_or_create( Aug 07 10:43:23.191981 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 07 10:43:23.191981 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application with Lock( Aug 07 10:43:23.191981 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 07 10:43:23.191981 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self._enter() Aug 07 10:43:23.191981 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 07 10:43:23.191981 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 07 10:43:23.193058 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 07 10:43:23.193058 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self.creator() Aug 07 10:43:23.193058 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 07 10:43:23.193058 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application created_value = creator( Aug 07 10:43:23.193058 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 07 10:43:23.193058 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 07 10:43:23.193058 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 07 10:43:23.193058 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 07 10:43:23.193058 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 07 10:43:23.193058 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 07 10:43:23.193058 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 07 10:43:23.193058 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application Aug 07 10:43:23.193058 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 581/1141] 10.222.0.25 () {56 vars in 1096 bytes} [Sun Aug 7 10:43:23 2022] GET /identity/v3/roles/admin => generated 83 bytes in 151 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 07 10:43:23.198660 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04f628c8-3783-4ac9-b452-c04a5d9d8d38 None None] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 10:43:23.263091 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 10:43:23.295496 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04f628c8-3783-4ac9-b452-c04a5d9d8d38 None None] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 10:43:23.328908 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-068fe198-1056-447b-97d5-fb55320ec63d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 10:43:23.329503 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-068fe198-1056-447b-97d5-fb55320ec63d None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:43:23.329780 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-068fe198-1056-447b-97d5-fb55320ec63d None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:43:23.330051 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-068fe198-1056-447b-97d5-fb55320ec63d None admin] PATH_INFO: `/v3/roles` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:43:23.330705 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-068fe198-1056-447b-97d5-fb55320ec63d None admin] RBAC: Adding query filter params (name=admin) {{(pid=80143) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 10:43:23.331032 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-068fe198-1056-447b-97d5-fb55320ec63d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 10:43:23.332384 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-068fe198-1056-447b-97d5-fb55320ec63d None admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 10:43:23.340312 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 561/1142] 10.222.0.25 () {56 vars in 1099 bytes} [Sun Aug 7 10:43:23 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 10:43:23.346133 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc36bb34-9140-4ebd-87e8-31ccb86d601f None admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 10:43:23.444275 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc36bb34-9140-4ebd-87e8-31ccb86d601f None admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 10:43:23.476170 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e200d559-9a56-407d-bf96-fb82938c429a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 10:43:23.477310 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e200d559-9a56-407d-bf96-fb82938c429a None admin] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:43:23.477569 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e200d559-9a56-407d-bf96-fb82938c429a None admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:43:23.477841 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e200d559-9a56-407d-bf96-fb82938c429a None admin] PATH_INFO: `/v3/users/b7910a921d314003aad1ffd2c944d672` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:43:23.490718 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e200d559-9a56-407d-bf96-fb82938c429a None admin] RBAC: Authorizing `identity:get_user(user_id=b7910a921d314003aad1ffd2c944d672)` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 10:43:23.492732 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e200d559-9a56-407d-bf96-fb82938c429a None admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 10:43:23.494290 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 582/1143] 10.222.0.25 () {56 vars in 1177 bytes} [Sun Aug 7 10:43:23 2022] GET /identity/v3/users/b7910a921d314003aad1ffd2c944d672 => generated 286 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 10:43:23.500121 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-068fe198-1056-447b-97d5-fb55320ec63d None admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 10:43:23.597272 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-068fe198-1056-447b-97d5-fb55320ec63d None admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 10:43:23.624914 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=100 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 10:43:23.631090 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1480842-f76a-4b17-a8bf-4f5bab0bb71b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 10:43:23.631684 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1480842-f76a-4b17-a8bf-4f5bab0bb71b None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:43:23.631958 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1480842-f76a-4b17-a8bf-4f5bab0bb71b None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:43:23.632225 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1480842-f76a-4b17-a8bf-4f5bab0bb71b None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:43:23.696355 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1480842-f76a-4b17-a8bf-4f5bab0bb71b None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 10:43:23.697705 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1480842-f76a-4b17-a8bf-4f5bab0bb71b None admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 10:43:23.795096 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 562/1144] 10.222.0.25 () {58 vars in 1282 bytes} [Sun Aug 7 10:43:23 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 295 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 07 10:43:23.801003 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e200d559-9a56-407d-bf96-fb82938c429a None admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 10:43:23.833281 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=100 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 10:43:23.906207 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e200d559-9a56-407d-bf96-fb82938c429a None admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 10:43:23.940330 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-783d54c1-5d93-43f5-a6ba-ec0504d363a3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 10:43:23.941005 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-783d54c1-5d93-43f5-a6ba-ec0504d363a3 None admin] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:43:23.941236 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-783d54c1-5d93-43f5-a6ba-ec0504d363a3 None admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:43:23.941507 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-783d54c1-5d93-43f5-a6ba-ec0504d363a3 None admin] PATH_INFO: `/v3/projects/f0ec96974da54a71801a169bbd28c43d` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:43:23.942603 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-783d54c1-5d93-43f5-a6ba-ec0504d363a3 None admin] RBAC: Authorizing `identity:get_project(project_id=f0ec96974da54a71801a169bbd28c43d)` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 10:43:23.944037 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-783d54c1-5d93-43f5-a6ba-ec0504d363a3 None admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 10:43:23.945810 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 583/1145] 10.222.0.25 () {56 vars in 1186 bytes} [Sun Aug 7 10:43:23 2022] GET /identity/v3/projects/f0ec96974da54a71801a169bbd28c43d => generated 311 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 10:43:23.952005 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1480842-f76a-4b17-a8bf-4f5bab0bb71b None admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 10:43:23.982511 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=100 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 10:43:24.050870 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1480842-f76a-4b17-a8bf-4f5bab0bb71b None admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 10:43:24.082618 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1df9b410-799f-42c5-9b15-1aa557804f55 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 10:43:24.083238 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1df9b410-799f-42c5-9b15-1aa557804f55 None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:43:24.083536 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1df9b410-799f-42c5-9b15-1aa557804f55 None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:43:24.083791 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1df9b410-799f-42c5-9b15-1aa557804f55 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:43:24.084500 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1df9b410-799f-42c5-9b15-1aa557804f55 None admin] RBAC: Adding query filter params (role.id=81b1f1bcee2a473088fbbd327c1c5b23, scope.project.id=f0ec96974da54a71801a169bbd28c43d, user.id=b7910a921d314003aad1ffd2c944d672) {{(pid=80143) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 10:43:24.084805 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1df9b410-799f-42c5-9b15-1aa557804f55 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 10:43:24.086910 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1df9b410-799f-42c5-9b15-1aa557804f55 None admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 10:43:24.092102 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 563/1146] 10.222.0.25 () {56 vars in 1374 bytes} [Sun Aug 7 10:43:23 2022] GET /identity/v3/role_assignments?user.id=b7910a921d314003aad1ffd2c944d672&scope.project.id=f0ec96974da54a71801a169bbd28c43d&role.id=81b1f1bcee2a473088fbbd327c1c5b23 => generated 259 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 10:43:26.028237 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8241bac-fb18-4ec1-b70f-3ac250ecdaf0 None None] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:43:26.028806 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8241bac-fb18-4ec1-b70f-3ac250ecdaf0 None None] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:43:26.028806 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8241bac-fb18-4ec1-b70f-3ac250ecdaf0 None None] PATH_INFO: `/` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:43:26.030140 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 584/1147] 10.222.0.25 () {54 vars in 927 bytes} [Sun Aug 7 10:43:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 07 10:43:26.035147 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b7db060-1588-483d-bf0e-3911cfecf781 None None] REQUEST_METHOD: `POST` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:43:26.035564 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b7db060-1588-483d-bf0e-3911cfecf781 None None] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:43:26.035707 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b7db060-1588-483d-bf0e-3911cfecf781 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:43:26.053413 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.auth.core [None req-9b7db060-1588-483d-bf0e-3911cfecf781 None None] MFA Rules not processed for user `bac4e6b76fe64217ac8fa44e71cffe6a`. Rule list: `[]` (Enabled: `True`). {{(pid=80143) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 10:43:26.055152 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-9b7db060-1588-483d-bf0e-3911cfecf781 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 10:43:26.121090 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 564/1148] 10.222.0.25 () {58 vars in 1024 bytes} [Sun Aug 7 10:43:26 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 07 10:43:26.143630 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8241bac-fb18-4ec1-b70f-3ac250ecdaf0 None None] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 10:43:26.235265 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=100 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 10:43:26.243350 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8241bac-fb18-4ec1-b70f-3ac250ecdaf0 None None] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 10:43:26.276761 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9dc1000-747a-41e4-aad9-397046875848 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 10:43:26.277372 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9dc1000-747a-41e4-aad9-397046875848 None admin] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:43:26.277656 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9dc1000-747a-41e4-aad9-397046875848 None admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:43:26.277931 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9dc1000-747a-41e4-aad9-397046875848 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:43:26.287762 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9dc1000-747a-41e4-aad9-397046875848 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 07 10:43:26.292326 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application [None req-f9dc1000-747a-41e4-aad9-397046875848 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 07 10:43:26.292326 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 07 10:43:26.292326 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 07 10:43:26.292326 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 07 10:43:26.292326 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 07 10:43:26.292326 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 07 10:43:26.292326 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 07 10:43:26.292326 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 07 10:43:26.292326 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 07 10:43:26.292326 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 07 10:43:26.292326 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 07 10:43:26.292326 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 07 10:43:26.292326 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 07 10:43:26.292326 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 07 10:43:26.292326 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 07 10:43:26.292326 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 07 10:43:26.293414 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 07 10:43:26.293414 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 07 10:43:26.293414 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 07 10:43:26.293414 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 07 10:43:26.293414 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 07 10:43:26.293414 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 07 10:43:26.293414 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 07 10:43:26.293414 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 07 10:43:26.293414 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 07 10:43:26.293414 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self.get_or_create( Aug 07 10:43:26.293414 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 07 10:43:26.293414 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application with Lock( Aug 07 10:43:26.293414 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 07 10:43:26.293414 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self._enter() Aug 07 10:43:26.293414 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 07 10:43:26.293414 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 07 10:43:26.294579 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 07 10:43:26.294579 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self.creator() Aug 07 10:43:26.294579 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 07 10:43:26.294579 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application created_value = creator( Aug 07 10:43:26.294579 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 07 10:43:26.294579 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 07 10:43:26.294579 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 07 10:43:26.294579 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 07 10:43:26.294579 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 07 10:43:26.294579 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 07 10:43:26.294579 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 07 10:43:26.294579 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application Aug 07 10:43:26.294579 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 585/1149] 10.222.0.25 () {56 vars in 1096 bytes} [Sun Aug 7 10:43:26 2022] GET /identity/v3/roles/admin => generated 83 bytes in 150 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 07 10:43:26.299398 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b7db060-1588-483d-bf0e-3911cfecf781 None None] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 10:43:26.324424 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=100 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 10:43:26.397134 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b7db060-1588-483d-bf0e-3911cfecf781 None None] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 10:43:26.430947 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e61f3bf6-bfc2-4a6c-b5d5-3dc7f49910da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 10:43:26.431571 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e61f3bf6-bfc2-4a6c-b5d5-3dc7f49910da None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:43:26.431850 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e61f3bf6-bfc2-4a6c-b5d5-3dc7f49910da None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:43:26.432120 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e61f3bf6-bfc2-4a6c-b5d5-3dc7f49910da None admin] PATH_INFO: `/v3/roles` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:43:26.432777 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e61f3bf6-bfc2-4a6c-b5d5-3dc7f49910da None admin] RBAC: Adding query filter params (name=admin) {{(pid=80143) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 10:43:26.433083 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e61f3bf6-bfc2-4a6c-b5d5-3dc7f49910da None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 10:43:26.434430 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e61f3bf6-bfc2-4a6c-b5d5-3dc7f49910da None admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 10:43:26.442149 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 565/1150] 10.222.0.25 () {56 vars in 1099 bytes} [Sun Aug 7 10:43:26 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 10:43:26.448024 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9dc1000-747a-41e4-aad9-397046875848 None admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 10:43:26.543930 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9dc1000-747a-41e4-aad9-397046875848 None admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 10:43:26.576217 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18b3750c-ea3f-4c60-a5c8-a50296ff46da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 10:43:26.576825 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18b3750c-ea3f-4c60-a5c8-a50296ff46da None admin] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:43:26.577105 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18b3750c-ea3f-4c60-a5c8-a50296ff46da None admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:43:26.577377 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18b3750c-ea3f-4c60-a5c8-a50296ff46da None admin] PATH_INFO: `/v3/users/b7910a921d314003aad1ffd2c944d672` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:43:26.578471 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18b3750c-ea3f-4c60-a5c8-a50296ff46da None admin] RBAC: Authorizing `identity:get_user(user_id=b7910a921d314003aad1ffd2c944d672)` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 10:43:26.579914 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18b3750c-ea3f-4c60-a5c8-a50296ff46da None admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 10:43:26.581451 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 586/1151] 10.222.0.25 () {56 vars in 1177 bytes} [Sun Aug 7 10:43:26 2022] GET /identity/v3/users/b7910a921d314003aad1ffd2c944d672 => generated 286 bytes in 134 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 10:43:26.587707 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e61f3bf6-bfc2-4a6c-b5d5-3dc7f49910da None admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 10:43:26.671429 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=100 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 10:43:26.686275 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e61f3bf6-bfc2-4a6c-b5d5-3dc7f49910da None admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 10:43:26.719490 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ad15dba-4209-4c0a-8717-66e4ded2e7f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 10:43:26.720111 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ad15dba-4209-4c0a-8717-66e4ded2e7f5 None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:43:26.720393 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ad15dba-4209-4c0a-8717-66e4ded2e7f5 None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:43:26.720669 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ad15dba-4209-4c0a-8717-66e4ded2e7f5 None admin] PATH_INFO: `/v3/projects/f0ec96974da54a71801a169bbd28c43d` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:43:26.721778 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ad15dba-4209-4c0a-8717-66e4ded2e7f5 None admin] RBAC: Authorizing `identity:get_project(project_id=f0ec96974da54a71801a169bbd28c43d)` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 10:43:26.723227 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ad15dba-4209-4c0a-8717-66e4ded2e7f5 None admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 10:43:26.724822 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 566/1152] 10.222.0.25 () {56 vars in 1186 bytes} [Sun Aug 7 10:43:26 2022] GET /identity/v3/projects/f0ec96974da54a71801a169bbd28c43d => generated 311 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 10:43:26.730734 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18b3750c-ea3f-4c60-a5c8-a50296ff46da None admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 10:43:26.755673 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=100 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 10:43:26.828272 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18b3750c-ea3f-4c60-a5c8-a50296ff46da None admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 10:43:26.859691 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3e73a1e-f86e-4837-bc13-786ecedeb148 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 10:43:26.860334 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3e73a1e-f86e-4837-bc13-786ecedeb148 None admin] REQUEST_METHOD: `PUT` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:43:26.860632 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3e73a1e-f86e-4837-bc13-786ecedeb148 None admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:43:26.860890 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3e73a1e-f86e-4837-bc13-786ecedeb148 None admin] PATH_INFO: `/v3/projects/f0ec96974da54a71801a169bbd28c43d/users/b7910a921d314003aad1ffd2c944d672/roles/81b1f1bcee2a473088fbbd327c1c5b23` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:43:26.862755 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3e73a1e-f86e-4837-bc13-786ecedeb148 None admin] RBAC: Authorizing `identity:create_grant(project_id=f0ec96974da54a71801a169bbd28c43d, user_id=b7910a921d314003aad1ffd2c944d672, role_id=81b1f1bcee2a473088fbbd327c1c5b23)` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 10:43:26.864451 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3e73a1e-f86e-4837-bc13-786ecedeb148 None admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 10:43:26.878216 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 587/1153] 10.222.0.25 () {58 vars in 1439 bytes} [Sun Aug 7 10:43:26 2022] PUT /identity/v3/projects/f0ec96974da54a71801a169bbd28c43d/users/b7910a921d314003aad1ffd2c944d672/roles/81b1f1bcee2a473088fbbd327c1c5b23 => generated 0 bytes in 148 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 07 10:43:28.821860 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d1e667e-d91d-49ad-b5de-4af76f146e1a None None] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:43:28.822382 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d1e667e-d91d-49ad-b5de-4af76f146e1a None None] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:43:28.822382 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d1e667e-d91d-49ad-b5de-4af76f146e1a None None] PATH_INFO: `/` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:43:28.823741 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 567/1154] 10.222.0.25 () {54 vars in 927 bytes} [Sun Aug 7 10:43:28 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 07 10:43:28.828733 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fc3f91b-9aff-4df6-ad68-e496633738ad None None] REQUEST_METHOD: `POST` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:43:28.829127 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fc3f91b-9aff-4df6-ad68-e496633738ad None None] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:43:28.829285 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fc3f91b-9aff-4df6-ad68-e496633738ad None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:43:28.847768 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.auth.core [None req-4fc3f91b-9aff-4df6-ad68-e496633738ad None None] MFA Rules not processed for user `bac4e6b76fe64217ac8fa44e71cffe6a`. Rule list: `[]` (Enabled: `True`). {{(pid=80144) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 10:43:28.848943 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.fernet_utils [None req-4fc3f91b-9aff-4df6-ad68-e496633738ad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80144) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 10:43:28.915869 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 588/1155] 10.222.0.25 () {58 vars in 1024 bytes} [Sun Aug 7 10:43:28 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 07 10:43:28.938073 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d1e667e-d91d-49ad-b5de-4af76f146e1a None None] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 10:43:29.036106 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d1e667e-d91d-49ad-b5de-4af76f146e1a None None] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 10:43:29.068299 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f61205d-1938-4a12-bae4-e11076bb7d8d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 10:43:29.068902 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f61205d-1938-4a12-bae4-e11076bb7d8d None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:43:29.069177 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f61205d-1938-4a12-bae4-e11076bb7d8d None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:43:29.069444 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f61205d-1938-4a12-bae4-e11076bb7d8d None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:43:29.080436 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f61205d-1938-4a12-bae4-e11076bb7d8d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 07 10:43:29.085155 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application [None req-8f61205d-1938-4a12-bae4-e11076bb7d8d None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 07 10:43:29.085155 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 07 10:43:29.085155 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 07 10:43:29.085155 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 07 10:43:29.085155 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 07 10:43:29.085155 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 07 10:43:29.085155 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 07 10:43:29.085155 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 07 10:43:29.085155 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 07 10:43:29.085155 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 07 10:43:29.085155 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 07 10:43:29.085155 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 07 10:43:29.085155 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 07 10:43:29.085155 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 07 10:43:29.085155 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 07 10:43:29.085155 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 07 10:43:29.086334 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 07 10:43:29.086334 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 07 10:43:29.086334 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 07 10:43:29.086334 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 07 10:43:29.086334 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 07 10:43:29.086334 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 07 10:43:29.086334 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 07 10:43:29.086334 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 07 10:43:29.086334 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 07 10:43:29.086334 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.get_or_create( Aug 07 10:43:29.086334 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 07 10:43:29.086334 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application with Lock( Aug 07 10:43:29.086334 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 07 10:43:29.086334 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self._enter() Aug 07 10:43:29.086334 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 07 10:43:29.086334 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 07 10:43:29.087478 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 07 10:43:29.087478 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.creator() Aug 07 10:43:29.087478 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 07 10:43:29.087478 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application created_value = creator( Aug 07 10:43:29.087478 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 07 10:43:29.087478 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 07 10:43:29.087478 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 07 10:43:29.087478 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 07 10:43:29.087478 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 07 10:43:29.087478 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 07 10:43:29.087478 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 07 10:43:29.087478 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application Aug 07 10:43:29.087478 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 568/1156] 10.222.0.25 () {56 vars in 1096 bytes} [Sun Aug 7 10:43:28 2022] GET /identity/v3/roles/admin => generated 83 bytes in 149 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 07 10:43:29.092988 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fc3f91b-9aff-4df6-ad68-e496633738ad None None] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 10:43:29.107669 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 10:43:29.189821 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fc3f91b-9aff-4df6-ad68-e496633738ad None None] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 10:43:29.222562 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e08ab565-bee4-4cd2-bc4a-b51f0385f555 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 10:43:29.223190 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e08ab565-bee4-4cd2-bc4a-b51f0385f555 None admin] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:43:29.223474 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e08ab565-bee4-4cd2-bc4a-b51f0385f555 None admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:43:29.223769 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e08ab565-bee4-4cd2-bc4a-b51f0385f555 None admin] PATH_INFO: `/v3/roles` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:43:29.224427 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e08ab565-bee4-4cd2-bc4a-b51f0385f555 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80144) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 10:43:29.224731 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e08ab565-bee4-4cd2-bc4a-b51f0385f555 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 10:43:29.226071 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e08ab565-bee4-4cd2-bc4a-b51f0385f555 None admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 10:43:29.233935 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 589/1157] 10.222.0.25 () {56 vars in 1099 bytes} [Sun Aug 7 10:43:29 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 10:43:29.240019 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f61205d-1938-4a12-bae4-e11076bb7d8d None admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 10:43:29.256215 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=100 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 10:43:29.337677 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f61205d-1938-4a12-bae4-e11076bb7d8d None admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 10:43:29.370096 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6788e6a1-d1bc-4dea-954a-eff30cf788b6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 10:43:29.370712 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6788e6a1-d1bc-4dea-954a-eff30cf788b6 None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:43:29.371011 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6788e6a1-d1bc-4dea-954a-eff30cf788b6 None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:43:29.371283 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6788e6a1-d1bc-4dea-954a-eff30cf788b6 None admin] PATH_INFO: `/v3/users/b7910a921d314003aad1ffd2c944d672` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:43:29.372377 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6788e6a1-d1bc-4dea-954a-eff30cf788b6 None admin] RBAC: Authorizing `identity:get_user(user_id=b7910a921d314003aad1ffd2c944d672)` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 10:43:29.373794 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6788e6a1-d1bc-4dea-954a-eff30cf788b6 None admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 10:43:29.375430 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 569/1158] 10.222.0.25 () {56 vars in 1177 bytes} [Sun Aug 7 10:43:29 2022] GET /identity/v3/users/b7910a921d314003aad1ffd2c944d672 => generated 286 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 10:43:29.381185 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e08ab565-bee4-4cd2-bc4a-b51f0385f555 None admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 10:43:29.456442 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=100 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 10:43:29.479472 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e08ab565-bee4-4cd2-bc4a-b51f0385f555 None admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 10:43:29.512578 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89b508ab-a5e4-42b1-99ea-b55318a2e08b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 10:43:29.513188 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89b508ab-a5e4-42b1-99ea-b55318a2e08b None admin] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:43:29.513473 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89b508ab-a5e4-42b1-99ea-b55318a2e08b None admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:43:29.513750 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89b508ab-a5e4-42b1-99ea-b55318a2e08b None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:43:29.580616 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89b508ab-a5e4-42b1-99ea-b55318a2e08b None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 10:43:29.582033 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89b508ab-a5e4-42b1-99ea-b55318a2e08b None admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 10:43:29.655406 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=100 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 10:43:29.678045 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 590/1159] 10.222.0.25 () {58 vars in 1282 bytes} [Sun Aug 7 10:43:29 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 297 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 07 10:43:29.684166 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6788e6a1-d1bc-4dea-954a-eff30cf788b6 None admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 10:43:29.762270 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=100 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 10:43:29.780191 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6788e6a1-d1bc-4dea-954a-eff30cf788b6 None admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 10:43:29.811984 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d08da39f-4392-44af-bfca-ad800f104f7d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 10:43:29.812598 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d08da39f-4392-44af-bfca-ad800f104f7d None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:43:29.812874 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d08da39f-4392-44af-bfca-ad800f104f7d None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:43:29.813144 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d08da39f-4392-44af-bfca-ad800f104f7d None admin] PATH_INFO: `/v3/projects/f0ec96974da54a71801a169bbd28c43d` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:43:29.814225 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d08da39f-4392-44af-bfca-ad800f104f7d None admin] RBAC: Authorizing `identity:get_project(project_id=f0ec96974da54a71801a169bbd28c43d)` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 10:43:29.815667 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d08da39f-4392-44af-bfca-ad800f104f7d None admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 10:43:29.817253 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 570/1160] 10.222.0.25 () {56 vars in 1186 bytes} [Sun Aug 7 10:43:29 2022] GET /identity/v3/projects/f0ec96974da54a71801a169bbd28c43d => generated 311 bytes in 134 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 10:43:29.823917 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89b508ab-a5e4-42b1-99ea-b55318a2e08b None admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 10:43:29.919775 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89b508ab-a5e4-42b1-99ea-b55318a2e08b None admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 10:43:29.953316 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0324f1ae-616f-4c18-834d-87e386ee29c0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 10:43:29.953907 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0324f1ae-616f-4c18-834d-87e386ee29c0 None admin] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:43:29.954189 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0324f1ae-616f-4c18-834d-87e386ee29c0 None admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:43:29.954463 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0324f1ae-616f-4c18-834d-87e386ee29c0 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:43:29.955206 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0324f1ae-616f-4c18-834d-87e386ee29c0 None admin] RBAC: Adding query filter params (role.id=81b1f1bcee2a473088fbbd327c1c5b23, scope.project.id=f0ec96974da54a71801a169bbd28c43d, user.id=b7910a921d314003aad1ffd2c944d672) {{(pid=80144) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 10:43:29.955520 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0324f1ae-616f-4c18-834d-87e386ee29c0 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 10:43:29.956927 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0324f1ae-616f-4c18-834d-87e386ee29c0 None admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 10:43:29.962561 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 591/1161] 10.222.0.25 () {56 vars in 1374 bytes} [Sun Aug 7 10:43:29 2022] GET /identity/v3/role_assignments?user.id=b7910a921d314003aad1ffd2c944d672&scope.project.id=f0ec96974da54a71801a169bbd28c43d&role.id=81b1f1bcee2a473088fbbd327c1c5b23 => generated 608 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 10:43:31.968085 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-258be014-b2b1-4734-bc9a-d4d0c25a43a6 None None] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:43:31.968614 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-258be014-b2b1-4734-bc9a-d4d0c25a43a6 None None] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:43:31.968614 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-258be014-b2b1-4734-bc9a-d4d0c25a43a6 None None] PATH_INFO: `/` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:43:31.969971 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 571/1162] 10.222.0.25 () {54 vars in 927 bytes} [Sun Aug 7 10:43:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 07 10:43:31.975099 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aa76f82-797b-4dbc-9499-c1f898cc17b5 None None] REQUEST_METHOD: `POST` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:43:31.975493 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aa76f82-797b-4dbc-9499-c1f898cc17b5 None None] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:43:31.975645 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aa76f82-797b-4dbc-9499-c1f898cc17b5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:43:31.993427 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.auth.core [None req-4aa76f82-797b-4dbc-9499-c1f898cc17b5 None None] MFA Rules not processed for user `bac4e6b76fe64217ac8fa44e71cffe6a`. Rule list: `[]` (Enabled: `True`). {{(pid=80144) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 10:43:31.994580 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.fernet_utils [None req-4aa76f82-797b-4dbc-9499-c1f898cc17b5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80144) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 10:43:32.028092 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=100 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 10:43:32.060782 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 592/1163] 10.222.0.25 () {58 vars in 1024 bytes} [Sun Aug 7 10:43:31 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 07 10:43:32.083331 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-258be014-b2b1-4734-bc9a-d4d0c25a43a6 None None] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 10:43:32.182689 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-258be014-b2b1-4734-bc9a-d4d0c25a43a6 None None] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 10:43:32.215011 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e0b648c-2528-4f57-b89a-aafe88f25fab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 10:43:32.215610 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e0b648c-2528-4f57-b89a-aafe88f25fab None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:43:32.215894 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e0b648c-2528-4f57-b89a-aafe88f25fab None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:43:32.216171 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e0b648c-2528-4f57-b89a-aafe88f25fab None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:43:32.234102 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=100 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 10:43:32.282644 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e0b648c-2528-4f57-b89a-aafe88f25fab None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 10:43:32.284141 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e0b648c-2528-4f57-b89a-aafe88f25fab None admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 10:43:32.380428 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 572/1164] 10.222.0.25 () {58 vars in 1282 bytes} [Sun Aug 7 10:43:32 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 298 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 07 10:43:32.387052 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aa76f82-797b-4dbc-9499-c1f898cc17b5 None None] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 10:43:32.480832 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aa76f82-797b-4dbc-9499-c1f898cc17b5 None None] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 10:43:32.513455 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ba6c245-6d70-4f2c-801c-e059284d7a0d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 10:43:32.514067 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ba6c245-6d70-4f2c-801c-e059284d7a0d None admin] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:43:32.514347 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ba6c245-6d70-4f2c-801c-e059284d7a0d None admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:43:32.514618 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ba6c245-6d70-4f2c-801c-e059284d7a0d None admin] PATH_INFO: `/v3/domains/swift_test` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:43:32.520960 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ba6c245-6d70-4f2c-801c-e059284d7a0d None admin] RBAC: Authorizing `identity:get_domain(domain_id=swift_test)` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 10:43:32.522358 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ba6c245-6d70-4f2c-801c-e059284d7a0d None admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 10:43:32.533236 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application [None req-1ba6c245-6d70-4f2c-801c-e059284d7a0d None admin] Could not find domain: swift_test.: keystone.exception.DomainNotFound: Could not find domain: swift_test. Aug 07 10:43:32.533236 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 07 10:43:32.533236 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 712, in get_domain Aug 07 10:43:32.533236 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application project = self.driver.get_project(domain_id) Aug 07 10:43:32.533236 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 07 10:43:32.533236 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 07 10:43:32.533236 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 07 10:43:32.533236 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 07 10:43:32.533236 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: swift_test. Aug 07 10:43:32.533236 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application Aug 07 10:43:32.533236 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 07 10:43:32.533236 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application Aug 07 10:43:32.533236 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 07 10:43:32.533236 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 07 10:43:32.533236 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 07 10:43:32.533236 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 07 10:43:32.533236 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 07 10:43:32.534460 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 07 10:43:32.534460 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 07 10:43:32.534460 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 07 10:43:32.534460 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 07 10:43:32.534460 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 07 10:43:32.534460 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 07 10:43:32.534460 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 07 10:43:32.534460 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 07 10:43:32.534460 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 07 10:43:32.534460 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 07 10:43:32.534460 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 07 10:43:32.534460 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 07 10:43:32.534460 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 07 10:43:32.534460 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 07 10:43:32.534460 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 07 10:43:32.534460 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self.get_or_create( Aug 07 10:43:32.535362 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 07 10:43:32.535362 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application with Lock( Aug 07 10:43:32.535362 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 07 10:43:32.535362 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self._enter() Aug 07 10:43:32.535362 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 07 10:43:32.535362 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 07 10:43:32.535362 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 07 10:43:32.535362 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self.creator() Aug 07 10:43:32.535362 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 07 10:43:32.535362 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application created_value = creator( Aug 07 10:43:32.535362 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 718, in get_domain Aug 07 10:43:32.535362 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 07 10:43:32.535362 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: swift_test. Aug 07 10:43:32.535362 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application Aug 07 10:43:32.535362 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 593/1165] 10.222.0.25 () {56 vars in 1117 bytes} [Sun Aug 7 10:43:32 2022] GET /identity/v3/domains/swift_test => generated 90 bytes in 148 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 07 10:43:32.540793 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e0b648c-2528-4f57-b89a-aafe88f25fab None admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 10:43:32.589307 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=100 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 10:43:32.637131 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e0b648c-2528-4f57-b89a-aafe88f25fab None admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 10:43:32.669414 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-422a713a-ec15-475a-b52a-4dfb61f4d8b9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 10:43:32.670005 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-422a713a-ec15-475a-b52a-4dfb61f4d8b9 None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:43:32.670280 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-422a713a-ec15-475a-b52a-4dfb61f4d8b9 None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:43:32.670546 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-422a713a-ec15-475a-b52a-4dfb61f4d8b9 None admin] PATH_INFO: `/v3/domains` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:43:32.671203 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-422a713a-ec15-475a-b52a-4dfb61f4d8b9 None admin] RBAC: Adding query filter params (name=swift_test) {{(pid=80143) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 10:43:32.671500 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-422a713a-ec15-475a-b52a-4dfb61f4d8b9 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 10:43:32.672844 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-422a713a-ec15-475a-b52a-4dfb61f4d8b9 None admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 10:43:32.679659 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 573/1166] 10.222.0.25 () {56 vars in 1115 bytes} [Sun Aug 7 10:43:32 2022] GET /identity/v3/domains?name=swift_test => generated 125 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 10:43:34.622881 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89923152-3aad-4a3f-9fbf-19c8ced5ac37 None None] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:43:34.623576 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89923152-3aad-4a3f-9fbf-19c8ced5ac37 None None] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:43:34.623576 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89923152-3aad-4a3f-9fbf-19c8ced5ac37 None None] PATH_INFO: `/` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:43:34.624858 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 594/1167] 10.222.0.25 () {54 vars in 927 bytes} [Sun Aug 7 10:43:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 07 10:43:34.629837 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36bab07f-dd0d-4552-8379-64ed0bfa2499 None None] REQUEST_METHOD: `POST` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:43:34.630234 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36bab07f-dd0d-4552-8379-64ed0bfa2499 None None] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:43:34.630382 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36bab07f-dd0d-4552-8379-64ed0bfa2499 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:43:34.648463 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.auth.core [None req-36bab07f-dd0d-4552-8379-64ed0bfa2499 None None] MFA Rules not processed for user `bac4e6b76fe64217ac8fa44e71cffe6a`. Rule list: `[]` (Enabled: `True`). {{(pid=80143) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 10:43:34.649638 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-36bab07f-dd0d-4552-8379-64ed0bfa2499 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 10:43:34.714348 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 574/1168] 10.222.0.25 () {58 vars in 1024 bytes} [Sun Aug 7 10:43:34 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 07 10:43:34.736249 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89923152-3aad-4a3f-9fbf-19c8ced5ac37 None None] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 10:43:34.766758 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=100 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 10:43:34.835469 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89923152-3aad-4a3f-9fbf-19c8ced5ac37 None None] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 10:43:34.869838 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5a13132-7ed9-4682-8951-8b4aecc064c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 10:43:34.870429 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5a13132-7ed9-4682-8951-8b4aecc064c9 None admin] REQUEST_METHOD: `POST` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:43:34.870708 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5a13132-7ed9-4682-8951-8b4aecc064c9 None admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:43:34.871005 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5a13132-7ed9-4682-8951-8b4aecc064c9 None admin] PATH_INFO: `/v3/domains` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:43:34.871661 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5a13132-7ed9-4682-8951-8b4aecc064c9 None admin] RBAC: Authorizing `identity:create_domain()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 10:43:34.873000 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5a13132-7ed9-4682-8951-8b4aecc064c9 None admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 10:43:34.888074 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 595/1169] 10.222.0.25 () {60 vars in 1138 bytes} [Sun Aug 7 10:43:34 2022] POST /identity/v3/domains => generated 267 bytes in 152 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 07 10:43:36.887468 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba066698-71b1-4455-b88b-e161208b3c5b None None] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:43:36.888019 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba066698-71b1-4455-b88b-e161208b3c5b None None] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:43:36.888019 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba066698-71b1-4455-b88b-e161208b3c5b None None] PATH_INFO: `/` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:43:36.889326 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 575/1170] 10.222.0.25 () {54 vars in 927 bytes} [Sun Aug 7 10:43:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 07 10:43:36.894354 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fb6b0c3-4ca4-45ee-bd50-b627b714e2fe None None] REQUEST_METHOD: `POST` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:43:36.894759 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fb6b0c3-4ca4-45ee-bd50-b627b714e2fe None None] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:43:36.894912 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fb6b0c3-4ca4-45ee-bd50-b627b714e2fe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:43:36.912651 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.auth.core [None req-2fb6b0c3-4ca4-45ee-bd50-b627b714e2fe None None] MFA Rules not processed for user `bac4e6b76fe64217ac8fa44e71cffe6a`. Rule list: `[]` (Enabled: `True`). {{(pid=80144) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 10:43:36.914902 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.fernet_utils [None req-2fb6b0c3-4ca4-45ee-bd50-b627b714e2fe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80144) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 10:43:36.980134 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 596/1171] 10.222.0.25 () {58 vars in 1024 bytes} [Sun Aug 7 10:43:36 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 07 10:43:37.003484 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba066698-71b1-4455-b88b-e161208b3c5b None None] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 10:43:37.048559 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=100 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 10:43:37.101511 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba066698-71b1-4455-b88b-e161208b3c5b None None] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 10:43:37.133770 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-275a3f8d-5d22-4b9f-81b8-52932f4b6fab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 10:43:37.134378 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-275a3f8d-5d22-4b9f-81b8-52932f4b6fab None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:43:37.134654 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-275a3f8d-5d22-4b9f-81b8-52932f4b6fab None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:43:37.134949 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-275a3f8d-5d22-4b9f-81b8-52932f4b6fab None admin] PATH_INFO: `/v3/domains/12d36731a82c4503811322c87ebd6f29` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:43:37.144832 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-275a3f8d-5d22-4b9f-81b8-52932f4b6fab None admin] RBAC: Authorizing `identity:get_domain(domain_id=12d36731a82c4503811322c87ebd6f29)` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 10:43:37.146199 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-275a3f8d-5d22-4b9f-81b8-52932f4b6fab None admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 10:43:37.147805 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 576/1172] 10.222.0.25 () {56 vars in 1183 bytes} [Sun Aug 7 10:43:37 2022] GET /identity/v3/domains/12d36731a82c4503811322c87ebd6f29 => generated 267 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 10:43:37.153687 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fb6b0c3-4ca4-45ee-bd50-b627b714e2fe None None] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 10:43:37.174411 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 10:43:37.250388 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fb6b0c3-4ca4-45ee-bd50-b627b714e2fe None None] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 10:43:37.283726 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3119c8b-68f8-459a-aa5d-a34b91056716 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 10:43:37.284329 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3119c8b-68f8-459a-aa5d-a34b91056716 None admin] REQUEST_METHOD: `POST` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:43:37.284609 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3119c8b-68f8-459a-aa5d-a34b91056716 None admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:43:37.284882 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3119c8b-68f8-459a-aa5d-a34b91056716 None admin] PATH_INFO: `/v3/projects` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:43:37.285526 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3119c8b-68f8-459a-aa5d-a34b91056716 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 10:43:37.286975 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3119c8b-68f8-459a-aa5d-a34b91056716 None admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 10:43:37.312496 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 597/1173] 10.222.0.25 () {60 vars in 1141 bytes} [Sun Aug 7 10:43:37 2022] POST /identity/v3/projects => generated 361 bytes in 159 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 07 10:43:39.309505 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3ec5bbc-e7ad-4303-817b-e2dd047ac116 None None] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:43:39.310063 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3ec5bbc-e7ad-4303-817b-e2dd047ac116 None None] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:43:39.310063 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3ec5bbc-e7ad-4303-817b-e2dd047ac116 None None] PATH_INFO: `/` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:43:39.311448 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 577/1174] 10.222.0.25 () {54 vars in 927 bytes} [Sun Aug 7 10:43:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 07 10:43:39.316543 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bcf81e4-b29c-4f80-8f4c-ab90f9a07858 None None] REQUEST_METHOD: `POST` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:43:39.316943 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bcf81e4-b29c-4f80-8f4c-ab90f9a07858 None None] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:43:39.317094 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bcf81e4-b29c-4f80-8f4c-ab90f9a07858 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:43:39.336042 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.auth.core [None req-7bcf81e4-b29c-4f80-8f4c-ab90f9a07858 None None] MFA Rules not processed for user `bac4e6b76fe64217ac8fa44e71cffe6a`. Rule list: `[]` (Enabled: `True`). {{(pid=80144) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 10:43:39.337208 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.fernet_utils [None req-7bcf81e4-b29c-4f80-8f4c-ab90f9a07858 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80144) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 10:43:39.412659 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 598/1175] 10.222.0.25 () {58 vars in 1024 bytes} [Sun Aug 7 10:43:39 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 07 10:43:39.437510 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3ec5bbc-e7ad-4303-817b-e2dd047ac116 None None] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 10:43:39.537145 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3ec5bbc-e7ad-4303-817b-e2dd047ac116 None None] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 10:43:39.539584 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=100 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 10:43:39.570629 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1201e5f4-1c09-48a3-93d4-bfc05645edda None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 10:43:39.571222 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1201e5f4-1c09-48a3-93d4-bfc05645edda None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:43:39.571499 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1201e5f4-1c09-48a3-93d4-bfc05645edda None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:43:39.571767 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1201e5f4-1c09-48a3-93d4-bfc05645edda None admin] PATH_INFO: `/v3/domains/12d36731a82c4503811322c87ebd6f29` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:43:39.572863 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1201e5f4-1c09-48a3-93d4-bfc05645edda None admin] RBAC: Authorizing `identity:get_domain(domain_id=12d36731a82c4503811322c87ebd6f29)` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 10:43:39.574268 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1201e5f4-1c09-48a3-93d4-bfc05645edda None admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 10:43:39.575832 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 578/1176] 10.222.0.25 () {56 vars in 1183 bytes} [Sun Aug 7 10:43:39 2022] GET /identity/v3/domains/12d36731a82c4503811322c87ebd6f29 => generated 267 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 10:43:39.581562 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bcf81e4-b29c-4f80-8f4c-ab90f9a07858 None None] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 10:43:39.583741 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 10:43:39.678489 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bcf81e4-b29c-4f80-8f4c-ab90f9a07858 None None] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 10:43:39.712029 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32d82688-ff85-4bd7-b738-f08bb6242641 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 10:43:39.712630 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32d82688-ff85-4bd7-b738-f08bb6242641 None admin] REQUEST_METHOD: `POST` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:43:39.712907 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32d82688-ff85-4bd7-b738-f08bb6242641 None admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:43:39.713178 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32d82688-ff85-4bd7-b738-f08bb6242641 None admin] PATH_INFO: `/v3/users` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:43:39.713818 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32d82688-ff85-4bd7-b738-f08bb6242641 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 10:43:39.715244 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32d82688-ff85-4bd7-b738-f08bb6242641 None admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 10:43:39.735038 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 599/1177] 10.222.0.25 () {60 vars in 1132 bytes} [Sun Aug 7 10:43:39 2022] POST /identity/v3/users => generated 311 bytes in 154 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 07 10:43:41.759794 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1492bef2-b5ec-43c9-bafc-2556ffe78a8d None None] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:43:41.760301 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1492bef2-b5ec-43c9-bafc-2556ffe78a8d None None] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:43:41.760395 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1492bef2-b5ec-43c9-bafc-2556ffe78a8d None None] PATH_INFO: `/` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:43:41.761675 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 579/1178] 10.222.0.25 () {54 vars in 927 bytes} [Sun Aug 7 10:43:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 07 10:43:41.767151 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f70c5703-e381-4a55-aa03-e3ed5f381157 None None] REQUEST_METHOD: `POST` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:43:41.767519 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f70c5703-e381-4a55-aa03-e3ed5f381157 None None] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:43:41.767710 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f70c5703-e381-4a55-aa03-e3ed5f381157 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:43:41.785869 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.auth.core [None req-f70c5703-e381-4a55-aa03-e3ed5f381157 None None] MFA Rules not processed for user `bac4e6b76fe64217ac8fa44e71cffe6a`. Rule list: `[]` (Enabled: `True`). {{(pid=80144) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 10:43:41.787088 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.fernet_utils [None req-f70c5703-e381-4a55-aa03-e3ed5f381157 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80144) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 10:43:41.838763 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 10:43:41.852873 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 600/1179] 10.222.0.25 () {58 vars in 1024 bytes} [Sun Aug 7 10:43:41 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 07 10:43:41.876884 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1492bef2-b5ec-43c9-bafc-2556ffe78a8d None None] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 10:43:41.974512 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1492bef2-b5ec-43c9-bafc-2556ffe78a8d None None] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 10:43:42.006598 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d20b9931-041f-454a-b08b-1aa9e942f996 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 10:43:42.007782 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d20b9931-041f-454a-b08b-1aa9e942f996 None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:43:42.008213 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d20b9931-041f-454a-b08b-1aa9e942f996 None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:43:42.008488 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d20b9931-041f-454a-b08b-1aa9e942f996 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:43:42.018564 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d20b9931-041f-454a-b08b-1aa9e942f996 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 07 10:43:42.023298 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application [None req-d20b9931-041f-454a-b08b-1aa9e942f996 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 07 10:43:42.023298 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 07 10:43:42.023298 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 07 10:43:42.023298 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 07 10:43:42.023298 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 07 10:43:42.023298 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 07 10:43:42.023298 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 07 10:43:42.023298 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 07 10:43:42.023298 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 07 10:43:42.023298 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 07 10:43:42.023298 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 07 10:43:42.023298 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 07 10:43:42.023298 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 07 10:43:42.023298 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 07 10:43:42.023298 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 07 10:43:42.023298 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 07 10:43:42.024416 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 07 10:43:42.024416 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 07 10:43:42.024416 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 07 10:43:42.024416 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 07 10:43:42.024416 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 07 10:43:42.024416 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 07 10:43:42.024416 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 07 10:43:42.024416 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 07 10:43:42.024416 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 07 10:43:42.024416 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.get_or_create( Aug 07 10:43:42.024416 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 07 10:43:42.024416 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application with Lock( Aug 07 10:43:42.024416 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 07 10:43:42.024416 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self._enter() Aug 07 10:43:42.024416 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 07 10:43:42.024416 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 07 10:43:42.025460 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 07 10:43:42.025460 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.creator() Aug 07 10:43:42.025460 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 07 10:43:42.025460 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application created_value = creator( Aug 07 10:43:42.025460 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 07 10:43:42.025460 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 07 10:43:42.025460 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 07 10:43:42.025460 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 07 10:43:42.025460 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 07 10:43:42.025460 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 07 10:43:42.025460 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 07 10:43:42.025460 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application Aug 07 10:43:42.025460 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 580/1180] 10.222.0.25 () {56 vars in 1096 bytes} [Sun Aug 7 10:43:41 2022] GET /identity/v3/roles/admin => generated 83 bytes in 148 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 07 10:43:42.030448 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f70c5703-e381-4a55-aa03-e3ed5f381157 None None] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 10:43:42.125125 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f70c5703-e381-4a55-aa03-e3ed5f381157 None None] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 10:43:42.157969 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1251e11-9c57-4d24-9afa-de0d9f9ec979 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 10:43:42.158558 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1251e11-9c57-4d24-9afa-de0d9f9ec979 None admin] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:43:42.158839 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1251e11-9c57-4d24-9afa-de0d9f9ec979 None admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:43:42.159135 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1251e11-9c57-4d24-9afa-de0d9f9ec979 None admin] PATH_INFO: `/v3/roles` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:43:42.159789 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1251e11-9c57-4d24-9afa-de0d9f9ec979 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80144) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 10:43:42.160091 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1251e11-9c57-4d24-9afa-de0d9f9ec979 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 10:43:42.161423 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1251e11-9c57-4d24-9afa-de0d9f9ec979 None admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 10:43:42.169737 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 601/1181] 10.222.0.25 () {56 vars in 1099 bytes} [Sun Aug 7 10:43:42 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 10:43:42.175572 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d20b9931-041f-454a-b08b-1aa9e942f996 None admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 10:43:42.207459 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=100 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 10:43:42.275452 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d20b9931-041f-454a-b08b-1aa9e942f996 None admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 10:43:42.308934 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-596ef3ec-3c56-48d1-bfe0-3beb008e2281 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 10:43:42.309553 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-596ef3ec-3c56-48d1-bfe0-3beb008e2281 None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:43:42.309834 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-596ef3ec-3c56-48d1-bfe0-3beb008e2281 None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:43:42.310108 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-596ef3ec-3c56-48d1-bfe0-3beb008e2281 None admin] PATH_INFO: `/v3/users/9e1213a0e9504176b25a7b6dfb79a4a5` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:43:42.322817 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-596ef3ec-3c56-48d1-bfe0-3beb008e2281 None admin] RBAC: Authorizing `identity:get_user(user_id=9e1213a0e9504176b25a7b6dfb79a4a5)` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 10:43:42.324287 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-596ef3ec-3c56-48d1-bfe0-3beb008e2281 None admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 10:43:42.325864 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 581/1182] 10.222.0.25 () {56 vars in 1177 bytes} [Sun Aug 7 10:43:42 2022] GET /identity/v3/users/9e1213a0e9504176b25a7b6dfb79a4a5 => generated 311 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 10:43:42.331621 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1251e11-9c57-4d24-9afa-de0d9f9ec979 None admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 10:43:42.381139 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=100 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 10:43:42.429948 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1251e11-9c57-4d24-9afa-de0d9f9ec979 None admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 10:43:42.462107 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f23c256-870c-4b9a-aa3a-ca3f72f69a54 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 10:43:42.462708 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f23c256-870c-4b9a-aa3a-ca3f72f69a54 None admin] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:43:42.463018 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f23c256-870c-4b9a-aa3a-ca3f72f69a54 None admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:43:42.463297 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f23c256-870c-4b9a-aa3a-ca3f72f69a54 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:43:42.528110 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f23c256-870c-4b9a-aa3a-ca3f72f69a54 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 10:43:42.529468 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f23c256-870c-4b9a-aa3a-ca3f72f69a54 None admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 10:43:42.575235 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=100 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 10:43:42.623928 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 602/1183] 10.222.0.25 () {58 vars in 1282 bytes} [Sun Aug 7 10:43:42 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 293 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 07 10:43:42.629811 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-596ef3ec-3c56-48d1-bfe0-3beb008e2281 None admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 10:43:42.715954 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=100 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 10:43:42.725632 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-596ef3ec-3c56-48d1-bfe0-3beb008e2281 None admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 10:43:42.758732 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0a881a7-baba-4c5e-9cb2-f1d6b899d3de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 10:43:42.759378 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0a881a7-baba-4c5e-9cb2-f1d6b899d3de None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:43:42.759663 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0a881a7-baba-4c5e-9cb2-f1d6b899d3de None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:43:42.759932 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0a881a7-baba-4c5e-9cb2-f1d6b899d3de None admin] PATH_INFO: `/v3/projects/45014b9a456c4f5c8f51f2a25b15746e` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:43:42.761021 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0a881a7-baba-4c5e-9cb2-f1d6b899d3de None admin] RBAC: Authorizing `identity:get_project(project_id=45014b9a456c4f5c8f51f2a25b15746e)` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 10:43:42.762432 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0a881a7-baba-4c5e-9cb2-f1d6b899d3de None admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 10:43:42.764052 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 582/1184] 10.222.0.25 () {56 vars in 1186 bytes} [Sun Aug 7 10:43:42 2022] GET /identity/v3/projects/45014b9a456c4f5c8f51f2a25b15746e => generated 361 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 10:43:42.770254 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f23c256-870c-4b9a-aa3a-ca3f72f69a54 None admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 10:43:42.866621 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f23c256-870c-4b9a-aa3a-ca3f72f69a54 None admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 10:43:42.898910 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d9a15a4-52ac-4386-bcdb-11174b35844a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 10:43:42.899512 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d9a15a4-52ac-4386-bcdb-11174b35844a None admin] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:43:42.899789 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d9a15a4-52ac-4386-bcdb-11174b35844a None admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:43:42.900059 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d9a15a4-52ac-4386-bcdb-11174b35844a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:43:42.900766 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d9a15a4-52ac-4386-bcdb-11174b35844a None admin] RBAC: Adding query filter params (role.id=81b1f1bcee2a473088fbbd327c1c5b23, scope.project.id=45014b9a456c4f5c8f51f2a25b15746e, user.id=9e1213a0e9504176b25a7b6dfb79a4a5) {{(pid=80144) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 10:43:42.901071 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d9a15a4-52ac-4386-bcdb-11174b35844a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 10:43:42.902448 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d9a15a4-52ac-4386-bcdb-11174b35844a None admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 10:43:42.907635 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 603/1185] 10.222.0.25 () {56 vars in 1374 bytes} [Sun Aug 7 10:43:42 2022] GET /identity/v3/role_assignments?user.id=9e1213a0e9504176b25a7b6dfb79a4a5&scope.project.id=45014b9a456c4f5c8f51f2a25b15746e&role.id=81b1f1bcee2a473088fbbd327c1c5b23 => generated 259 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 10:43:44.899429 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86528ba3-6249-4a39-bd45-f2f207db6c18 None None] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:43:44.899957 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86528ba3-6249-4a39-bd45-f2f207db6c18 None None] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:43:44.899957 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86528ba3-6249-4a39-bd45-f2f207db6c18 None None] PATH_INFO: `/` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:43:44.901293 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 583/1186] 10.222.0.25 () {54 vars in 927 bytes} [Sun Aug 7 10:43:44 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 07 10:43:44.906345 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96bd7ebe-6c61-4c29-bce9-a8d1eae9b28e None None] REQUEST_METHOD: `POST` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:43:44.906717 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96bd7ebe-6c61-4c29-bce9-a8d1eae9b28e None None] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:43:44.906898 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96bd7ebe-6c61-4c29-bce9-a8d1eae9b28e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:43:44.924869 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.auth.core [None req-96bd7ebe-6c61-4c29-bce9-a8d1eae9b28e None None] MFA Rules not processed for user `bac4e6b76fe64217ac8fa44e71cffe6a`. Rule list: `[]` (Enabled: `True`). {{(pid=80144) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 10:43:44.926044 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.fernet_utils [None req-96bd7ebe-6c61-4c29-bce9-a8d1eae9b28e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80144) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 10:43:44.935858 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=100 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 10:43:44.990438 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 604/1187] 10.222.0.25 () {58 vars in 1024 bytes} [Sun Aug 7 10:43:44 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 07 10:43:45.013363 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86528ba3-6249-4a39-bd45-f2f207db6c18 None None] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 10:43:45.110462 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86528ba3-6249-4a39-bd45-f2f207db6c18 None None] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 10:43:45.143491 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfb00d0c-81e9-41c6-86b4-8419c60dc6f3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 10:43:45.144095 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfb00d0c-81e9-41c6-86b4-8419c60dc6f3 None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:43:45.144371 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfb00d0c-81e9-41c6-86b4-8419c60dc6f3 None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:43:45.144639 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfb00d0c-81e9-41c6-86b4-8419c60dc6f3 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:43:45.154257 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfb00d0c-81e9-41c6-86b4-8419c60dc6f3 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 07 10:43:45.158767 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application [None req-bfb00d0c-81e9-41c6-86b4-8419c60dc6f3 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 07 10:43:45.158767 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 07 10:43:45.158767 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 07 10:43:45.158767 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 07 10:43:45.158767 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 07 10:43:45.158767 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 07 10:43:45.158767 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 07 10:43:45.158767 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 07 10:43:45.158767 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 07 10:43:45.158767 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 07 10:43:45.158767 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 07 10:43:45.158767 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 07 10:43:45.158767 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 07 10:43:45.158767 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 07 10:43:45.158767 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 07 10:43:45.158767 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 07 10:43:45.159903 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 07 10:43:45.159903 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 07 10:43:45.159903 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 07 10:43:45.159903 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 07 10:43:45.159903 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 07 10:43:45.159903 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 07 10:43:45.159903 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 07 10:43:45.159903 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 07 10:43:45.159903 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 07 10:43:45.159903 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.get_or_create( Aug 07 10:43:45.159903 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 07 10:43:45.159903 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application with Lock( Aug 07 10:43:45.159903 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 07 10:43:45.159903 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self._enter() Aug 07 10:43:45.159903 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 07 10:43:45.159903 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 07 10:43:45.161106 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 07 10:43:45.161106 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.creator() Aug 07 10:43:45.161106 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 07 10:43:45.161106 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application created_value = creator( Aug 07 10:43:45.161106 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 07 10:43:45.161106 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 07 10:43:45.161106 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 07 10:43:45.161106 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 07 10:43:45.161106 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 07 10:43:45.161106 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 07 10:43:45.161106 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 07 10:43:45.161106 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application Aug 07 10:43:45.161106 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 584/1188] 10.222.0.25 () {56 vars in 1096 bytes} [Sun Aug 7 10:43:45 2022] GET /identity/v3/roles/admin => generated 83 bytes in 147 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 07 10:43:45.166216 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96bd7ebe-6c61-4c29-bce9-a8d1eae9b28e None None] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 10:43:45.261781 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96bd7ebe-6c61-4c29-bce9-a8d1eae9b28e None None] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 10:43:45.294491 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f3dfe43-1bc7-4139-b8b5-2d36e636930f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 10:43:45.295109 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f3dfe43-1bc7-4139-b8b5-2d36e636930f None admin] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:43:45.295387 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f3dfe43-1bc7-4139-b8b5-2d36e636930f None admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:43:45.295655 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f3dfe43-1bc7-4139-b8b5-2d36e636930f None admin] PATH_INFO: `/v3/roles` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:43:45.296304 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f3dfe43-1bc7-4139-b8b5-2d36e636930f None admin] RBAC: Adding query filter params (name=admin) {{(pid=80144) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 10:43:45.296605 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f3dfe43-1bc7-4139-b8b5-2d36e636930f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 10:43:45.297939 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f3dfe43-1bc7-4139-b8b5-2d36e636930f None admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 10:43:45.306327 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 605/1189] 10.222.0.25 () {56 vars in 1099 bytes} [Sun Aug 7 10:43:45 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 10:43:45.312088 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfb00d0c-81e9-41c6-86b4-8419c60dc6f3 None admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 10:43:45.330872 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=100 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 10:43:45.408301 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfb00d0c-81e9-41c6-86b4-8419c60dc6f3 None admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 10:43:45.442095 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b77b2ea-a6f9-4fdb-8f92-37feeb66fbee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 10:43:45.442720 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b77b2ea-a6f9-4fdb-8f92-37feeb66fbee None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:43:45.443023 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b77b2ea-a6f9-4fdb-8f92-37feeb66fbee None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:43:45.443297 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b77b2ea-a6f9-4fdb-8f92-37feeb66fbee None admin] PATH_INFO: `/v3/users/9e1213a0e9504176b25a7b6dfb79a4a5` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:43:45.444437 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b77b2ea-a6f9-4fdb-8f92-37feeb66fbee None admin] RBAC: Authorizing `identity:get_user(user_id=9e1213a0e9504176b25a7b6dfb79a4a5)` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 10:43:45.445851 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b77b2ea-a6f9-4fdb-8f92-37feeb66fbee None admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 10:43:45.447442 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 585/1190] 10.222.0.25 () {56 vars in 1177 bytes} [Sun Aug 7 10:43:45 2022] GET /identity/v3/users/9e1213a0e9504176b25a7b6dfb79a4a5 => generated 311 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 10:43:45.453086 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f3dfe43-1bc7-4139-b8b5-2d36e636930f None admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 10:43:45.462765 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=100 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 10:43:45.549431 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f3dfe43-1bc7-4139-b8b5-2d36e636930f None admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 10:43:45.580452 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e24c3cbd-90a9-40f7-9bc1-044d5b5df15a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 10:43:45.581071 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e24c3cbd-90a9-40f7-9bc1-044d5b5df15a None admin] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:43:45.581349 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e24c3cbd-90a9-40f7-9bc1-044d5b5df15a None admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:43:45.581620 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e24c3cbd-90a9-40f7-9bc1-044d5b5df15a None admin] PATH_INFO: `/v3/projects/45014b9a456c4f5c8f51f2a25b15746e` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:43:45.582706 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e24c3cbd-90a9-40f7-9bc1-044d5b5df15a None admin] RBAC: Authorizing `identity:get_project(project_id=45014b9a456c4f5c8f51f2a25b15746e)` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 10:43:45.584147 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e24c3cbd-90a9-40f7-9bc1-044d5b5df15a None admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 10:43:45.586359 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 606/1191] 10.222.0.25 () {56 vars in 1186 bytes} [Sun Aug 7 10:43:45 2022] GET /identity/v3/projects/45014b9a456c4f5c8f51f2a25b15746e => generated 361 bytes in 134 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 10:43:45.592341 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b77b2ea-a6f9-4fdb-8f92-37feeb66fbee None admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 10:43:45.674957 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=100 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 10:43:45.689516 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b77b2ea-a6f9-4fdb-8f92-37feeb66fbee None admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 10:43:45.721927 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f92f65a-fec8-43c1-a327-45aec870c8bd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 10:43:45.722603 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f92f65a-fec8-43c1-a327-45aec870c8bd None admin] REQUEST_METHOD: `PUT` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:43:45.722877 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f92f65a-fec8-43c1-a327-45aec870c8bd None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:43:45.723176 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f92f65a-fec8-43c1-a327-45aec870c8bd None admin] PATH_INFO: `/v3/projects/45014b9a456c4f5c8f51f2a25b15746e/users/9e1213a0e9504176b25a7b6dfb79a4a5/roles/81b1f1bcee2a473088fbbd327c1c5b23` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:43:45.725037 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f92f65a-fec8-43c1-a327-45aec870c8bd None admin] RBAC: Authorizing `identity:create_grant(project_id=45014b9a456c4f5c8f51f2a25b15746e, user_id=9e1213a0e9504176b25a7b6dfb79a4a5, role_id=81b1f1bcee2a473088fbbd327c1c5b23)` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 10:43:45.727130 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f92f65a-fec8-43c1-a327-45aec870c8bd None admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 10:43:45.739678 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 586/1192] 10.222.0.25 () {58 vars in 1439 bytes} [Sun Aug 7 10:43:45 2022] PUT /identity/v3/projects/45014b9a456c4f5c8f51f2a25b15746e/users/9e1213a0e9504176b25a7b6dfb79a4a5/roles/81b1f1bcee2a473088fbbd327c1c5b23 => generated 0 bytes in 148 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 07 10:43:47.687445 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f8154bb-3722-4eaa-a715-18d536fe5ac9 None None] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:43:47.687967 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f8154bb-3722-4eaa-a715-18d536fe5ac9 None None] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:43:47.687967 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f8154bb-3722-4eaa-a715-18d536fe5ac9 None None] PATH_INFO: `/` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:43:47.689255 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 607/1193] 10.222.0.25 () {54 vars in 927 bytes} [Sun Aug 7 10:43:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 07 10:43:47.694218 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45f88563-cf00-49b6-8853-b9880957dd6f None None] REQUEST_METHOD: `POST` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:43:47.695309 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45f88563-cf00-49b6-8853-b9880957dd6f None None] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:43:47.695309 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45f88563-cf00-49b6-8853-b9880957dd6f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:43:47.715619 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.auth.core [None req-45f88563-cf00-49b6-8853-b9880957dd6f None None] MFA Rules not processed for user `bac4e6b76fe64217ac8fa44e71cffe6a`. Rule list: `[]` (Enabled: `True`). {{(pid=80143) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 10:43:47.716786 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-45f88563-cf00-49b6-8853-b9880957dd6f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 10:43:47.781124 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 587/1194] 10.222.0.25 () {58 vars in 1024 bytes} [Sun Aug 7 10:43:47 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 07 10:43:47.804280 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f8154bb-3722-4eaa-a715-18d536fe5ac9 None None] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 10:43:47.881517 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=100 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 10:43:47.907492 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f8154bb-3722-4eaa-a715-18d536fe5ac9 None None] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 10:43:47.940698 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2059d034-c017-43d8-bc23-52753994faab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 10:43:47.941258 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2059d034-c017-43d8-bc23-52753994faab None admin] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:43:47.941536 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2059d034-c017-43d8-bc23-52753994faab None admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:43:47.941808 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2059d034-c017-43d8-bc23-52753994faab None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:43:47.952179 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2059d034-c017-43d8-bc23-52753994faab None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 07 10:43:47.956824 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application [None req-2059d034-c017-43d8-bc23-52753994faab None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 07 10:43:47.956824 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 07 10:43:47.956824 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 07 10:43:47.956824 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 07 10:43:47.956824 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 07 10:43:47.956824 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 07 10:43:47.956824 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 07 10:43:47.956824 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 07 10:43:47.956824 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 07 10:43:47.956824 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 07 10:43:47.956824 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 07 10:43:47.956824 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 07 10:43:47.956824 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 07 10:43:47.956824 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 07 10:43:47.956824 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 07 10:43:47.956824 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 07 10:43:47.958004 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 07 10:43:47.958004 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 07 10:43:47.958004 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 07 10:43:47.958004 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 07 10:43:47.958004 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 07 10:43:47.958004 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 07 10:43:47.958004 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 07 10:43:47.958004 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 07 10:43:47.958004 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 07 10:43:47.958004 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self.get_or_create( Aug 07 10:43:47.958004 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 07 10:43:47.958004 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application with Lock( Aug 07 10:43:47.958004 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 07 10:43:47.958004 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self._enter() Aug 07 10:43:47.958004 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 07 10:43:47.958004 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 07 10:43:47.959163 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 07 10:43:47.959163 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self.creator() Aug 07 10:43:47.959163 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 07 10:43:47.959163 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application created_value = creator( Aug 07 10:43:47.959163 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 07 10:43:47.959163 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 07 10:43:47.959163 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 07 10:43:47.959163 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 07 10:43:47.959163 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 07 10:43:47.959163 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 07 10:43:47.959163 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 07 10:43:47.959163 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application Aug 07 10:43:47.959163 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 608/1195] 10.222.0.25 () {56 vars in 1096 bytes} [Sun Aug 7 10:43:47 2022] GET /identity/v3/roles/admin => generated 83 bytes in 155 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 07 10:43:47.964382 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45f88563-cf00-49b6-8853-b9880957dd6f None None] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 10:43:48.037199 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 10:43:48.061017 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45f88563-cf00-49b6-8853-b9880957dd6f None None] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 10:43:48.094707 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-436e72d1-4bde-49e8-98b0-eca330887126 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 10:43:48.095335 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-436e72d1-4bde-49e8-98b0-eca330887126 None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:43:48.095612 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-436e72d1-4bde-49e8-98b0-eca330887126 None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:43:48.095885 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-436e72d1-4bde-49e8-98b0-eca330887126 None admin] PATH_INFO: `/v3/roles` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:43:48.096540 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-436e72d1-4bde-49e8-98b0-eca330887126 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80143) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 10:43:48.096842 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-436e72d1-4bde-49e8-98b0-eca330887126 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 10:43:48.098186 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-436e72d1-4bde-49e8-98b0-eca330887126 None admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 10:43:48.106511 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 588/1196] 10.222.0.25 () {56 vars in 1099 bytes} [Sun Aug 7 10:43:47 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 10:43:48.112177 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2059d034-c017-43d8-bc23-52753994faab None admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 10:43:48.208238 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2059d034-c017-43d8-bc23-52753994faab None admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 10:43:48.240995 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b773002-0c8f-4ffc-ab30-9e7497e99e2d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 10:43:48.241645 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b773002-0c8f-4ffc-ab30-9e7497e99e2d None admin] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:43:48.241917 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b773002-0c8f-4ffc-ab30-9e7497e99e2d None admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:43:48.242186 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b773002-0c8f-4ffc-ab30-9e7497e99e2d None admin] PATH_INFO: `/v3/users/9e1213a0e9504176b25a7b6dfb79a4a5` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:43:48.243314 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b773002-0c8f-4ffc-ab30-9e7497e99e2d None admin] RBAC: Authorizing `identity:get_user(user_id=9e1213a0e9504176b25a7b6dfb79a4a5)` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 10:43:48.244705 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b773002-0c8f-4ffc-ab30-9e7497e99e2d None admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 10:43:48.246229 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 609/1197] 10.222.0.25 () {56 vars in 1177 bytes} [Sun Aug 7 10:43:48 2022] GET /identity/v3/users/9e1213a0e9504176b25a7b6dfb79a4a5 => generated 311 bytes in 134 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 10:43:48.251888 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-436e72d1-4bde-49e8-98b0-eca330887126 None admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 10:43:48.347751 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-436e72d1-4bde-49e8-98b0-eca330887126 None admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 10:43:48.379898 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e79981bc-b7d6-4760-a9a5-cb7332965e95 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 10:43:48.380491 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e79981bc-b7d6-4760-a9a5-cb7332965e95 None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:43:48.380768 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e79981bc-b7d6-4760-a9a5-cb7332965e95 None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:43:48.381041 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e79981bc-b7d6-4760-a9a5-cb7332965e95 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:43:48.389723 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=100 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 10:43:48.447574 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e79981bc-b7d6-4760-a9a5-cb7332965e95 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 10:43:48.449015 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e79981bc-b7d6-4760-a9a5-cb7332965e95 None admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 10:43:48.543562 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 589/1198] 10.222.0.25 () {58 vars in 1282 bytes} [Sun Aug 7 10:43:48 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 292 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 07 10:43:48.549586 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b773002-0c8f-4ffc-ab30-9e7497e99e2d None admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 10:43:48.552463 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=100 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 10:43:48.647292 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b773002-0c8f-4ffc-ab30-9e7497e99e2d None admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 10:43:48.679986 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8241097a-03b1-4afd-887b-17636e19449c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 10:43:48.680604 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8241097a-03b1-4afd-887b-17636e19449c None admin] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:43:48.680889 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8241097a-03b1-4afd-887b-17636e19449c None admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:43:48.681167 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8241097a-03b1-4afd-887b-17636e19449c None admin] PATH_INFO: `/v3/projects/45014b9a456c4f5c8f51f2a25b15746e` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:43:48.682281 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8241097a-03b1-4afd-887b-17636e19449c None admin] RBAC: Authorizing `identity:get_project(project_id=45014b9a456c4f5c8f51f2a25b15746e)` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 10:43:48.683722 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8241097a-03b1-4afd-887b-17636e19449c None admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 10:43:48.685319 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 610/1199] 10.222.0.25 () {56 vars in 1186 bytes} [Sun Aug 7 10:43:48 2022] GET /identity/v3/projects/45014b9a456c4f5c8f51f2a25b15746e => generated 361 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 10:43:48.691311 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e79981bc-b7d6-4760-a9a5-cb7332965e95 None admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 10:43:48.729462 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=100 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 10:43:48.788372 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e79981bc-b7d6-4760-a9a5-cb7332965e95 None admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 10:43:48.821184 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb1972d4-bfed-4501-9ff2-1de4f054eb58 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 10:43:48.821719 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb1972d4-bfed-4501-9ff2-1de4f054eb58 None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:43:48.821998 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb1972d4-bfed-4501-9ff2-1de4f054eb58 None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:43:48.822267 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb1972d4-bfed-4501-9ff2-1de4f054eb58 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:43:48.823015 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb1972d4-bfed-4501-9ff2-1de4f054eb58 None admin] RBAC: Adding query filter params (role.id=81b1f1bcee2a473088fbbd327c1c5b23, scope.project.id=45014b9a456c4f5c8f51f2a25b15746e, user.id=9e1213a0e9504176b25a7b6dfb79a4a5) {{(pid=80143) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 10:43:48.823323 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb1972d4-bfed-4501-9ff2-1de4f054eb58 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 10:43:48.824726 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb1972d4-bfed-4501-9ff2-1de4f054eb58 None admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 10:43:48.830214 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 590/1200] 10.222.0.25 () {56 vars in 1374 bytes} [Sun Aug 7 10:43:48 2022] GET /identity/v3/role_assignments?user.id=9e1213a0e9504176b25a7b6dfb79a4a5&scope.project.id=45014b9a456c4f5c8f51f2a25b15746e&role.id=81b1f1bcee2a473088fbbd327c1c5b23 => generated 608 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 10:43:58.675336 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=65 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 10:43:58.827730 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=49 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 10:44:43.681106 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d07408ba-9ccb-4a30-960c-7311f214aeb1 None None] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:44:43.681672 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d07408ba-9ccb-4a30-960c-7311f214aeb1 None None] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:44:43.681672 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d07408ba-9ccb-4a30-960c-7311f214aeb1 None None] PATH_INFO: `/` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:44:43.682911 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 611/1201] 10.222.0.25 () {54 vars in 927 bytes} [Sun Aug 7 10:44:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 07 10:44:43.688398 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d35bab9f-57d6-49d2-917a-47719264ad67 None None] REQUEST_METHOD: `POST` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:44:43.688894 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d35bab9f-57d6-49d2-917a-47719264ad67 None None] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:44:43.688989 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d35bab9f-57d6-49d2-917a-47719264ad67 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:44:43.720926 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.auth.core [None req-d35bab9f-57d6-49d2-917a-47719264ad67 None None] MFA Rules not processed for user `bac4e6b76fe64217ac8fa44e71cffe6a`. Rule list: `[]` (Enabled: `True`). {{(pid=80143) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 10:44:43.722465 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-d35bab9f-57d6-49d2-917a-47719264ad67 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 10:44:43.815223 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 591/1202] 10.222.0.25 () {58 vars in 1024 bytes} [Sun Aug 7 10:44:43 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 07 10:44:43.842031 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d07408ba-9ccb-4a30-960c-7311f214aeb1 None None] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 10:44:43.979081 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d07408ba-9ccb-4a30-960c-7311f214aeb1 None None] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 10:44:44.032720 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bf7a9f2-af5b-4fdf-a74b-0c8a6581506a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 10:44:44.033372 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bf7a9f2-af5b-4fdf-a74b-0c8a6581506a None admin] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:44:44.033657 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bf7a9f2-af5b-4fdf-a74b-0c8a6581506a None admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:44:44.033990 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bf7a9f2-af5b-4fdf-a74b-0c8a6581506a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:44:44.047834 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bf7a9f2-af5b-4fdf-a74b-0c8a6581506a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 10:44:44.049317 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bf7a9f2-af5b-4fdf-a74b-0c8a6581506a None admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 10:44:44.071008 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application [None req-3bf7a9f2-af5b-4fdf-a74b-0c8a6581506a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 07 10:44:44.071008 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 07 10:44:44.071008 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 07 10:44:44.071008 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 07 10:44:44.071008 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 07 10:44:44.071008 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 07 10:44:44.071008 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 07 10:44:44.071008 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 07 10:44:44.071008 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 07 10:44:44.071008 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 07 10:44:44.071008 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 07 10:44:44.071008 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 07 10:44:44.071008 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 07 10:44:44.071008 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 07 10:44:44.071008 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 07 10:44:44.071008 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 07 10:44:44.072523 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 07 10:44:44.072523 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 07 10:44:44.072523 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 07 10:44:44.072523 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 07 10:44:44.072523 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 07 10:44:44.072523 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self.get_or_create( Aug 07 10:44:44.072523 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 07 10:44:44.072523 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application with Lock( Aug 07 10:44:44.072523 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 07 10:44:44.072523 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self._enter() Aug 07 10:44:44.072523 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 07 10:44:44.072523 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 07 10:44:44.072523 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 07 10:44:44.072523 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self.creator() Aug 07 10:44:44.072523 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 07 10:44:44.072523 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application created_value = creator( Aug 07 10:44:44.072523 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 07 10:44:44.073855 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 07 10:44:44.073855 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 07 10:44:44.073855 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application Aug 07 10:44:44.074695 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 612/1203] 10.222.0.25 () {56 vars in 1108 bytes} [Sun Aug 7 10:44:43 2022] GET /identity/v3/domains/Default => generated 87 bytes in 233 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 07 10:44:44.080734 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d35bab9f-57d6-49d2-917a-47719264ad67 None None] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 10:44:44.213331 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d35bab9f-57d6-49d2-917a-47719264ad67 None None] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 10:44:44.253184 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=100 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 10:44:44.260823 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e975b154-af7a-45aa-9ab8-620e5aeb4d6f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 10:44:44.261563 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e975b154-af7a-45aa-9ab8-620e5aeb4d6f None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:44:44.261942 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e975b154-af7a-45aa-9ab8-620e5aeb4d6f None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:44:44.262301 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e975b154-af7a-45aa-9ab8-620e5aeb4d6f None admin] PATH_INFO: `/v3/domains` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:44:44.263078 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e975b154-af7a-45aa-9ab8-620e5aeb4d6f None admin] RBAC: Adding query filter params (name=Default) {{(pid=80143) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 10:44:44.263440 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e975b154-af7a-45aa-9ab8-620e5aeb4d6f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 10:44:44.264865 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e975b154-af7a-45aa-9ab8-620e5aeb4d6f None admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 10:44:44.277005 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 592/1204] 10.222.0.25 () {56 vars in 1109 bytes} [Sun Aug 7 10:44:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 10:44:44.282690 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bf7a9f2-af5b-4fdf-a74b-0c8a6581506a None admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 10:44:44.356741 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=100 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 10:44:44.431754 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bf7a9f2-af5b-4fdf-a74b-0c8a6581506a None admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 10:44:44.477732 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-797a41fd-8ec6-47ec-95dd-31b11f34710d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 10:44:44.478888 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-797a41fd-8ec6-47ec-95dd-31b11f34710d None admin] REQUEST_METHOD: `POST` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:44:44.479519 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-797a41fd-8ec6-47ec-95dd-31b11f34710d None admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:44:44.480028 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-797a41fd-8ec6-47ec-95dd-31b11f34710d None admin] PATH_INFO: `/v3/users` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:44:44.481021 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-797a41fd-8ec6-47ec-95dd-31b11f34710d None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 10:44:44.483027 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-797a41fd-8ec6-47ec-95dd-31b11f34710d None admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 10:44:44.512818 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 613/1205] 10.222.0.25 () {60 vars in 1132 bytes} [Sun Aug 7 10:44:44 2022] POST /identity/v3/users => generated 251 bytes in 231 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 07 10:44:46.687030 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63af61bb-5184-4a47-8e27-5c87ceed3ee9 None None] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:44:46.687030 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63af61bb-5184-4a47-8e27-5c87ceed3ee9 None None] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:44:46.687030 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63af61bb-5184-4a47-8e27-5c87ceed3ee9 None None] PATH_INFO: `/` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:44:46.688098 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 593/1206] 10.222.0.25 () {54 vars in 927 bytes} [Sun Aug 7 10:44:46 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 07 10:44:46.693832 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77812297-ed71-43ec-a077-b09c67e2fc0a None None] REQUEST_METHOD: `POST` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:44:46.694632 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77812297-ed71-43ec-a077-b09c67e2fc0a None None] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:44:46.694632 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77812297-ed71-43ec-a077-b09c67e2fc0a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:44:46.713067 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.auth.core [None req-77812297-ed71-43ec-a077-b09c67e2fc0a None None] MFA Rules not processed for user `bac4e6b76fe64217ac8fa44e71cffe6a`. Rule list: `[]` (Enabled: `True`). {{(pid=80144) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 10:44:46.714263 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.fernet_utils [None req-77812297-ed71-43ec-a077-b09c67e2fc0a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80144) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 10:44:46.791342 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 614/1207] 10.222.0.25 () {58 vars in 1024 bytes} [Sun Aug 7 10:44:46 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 07 10:44:46.815579 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63af61bb-5184-4a47-8e27-5c87ceed3ee9 None None] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 10:44:46.928546 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63af61bb-5184-4a47-8e27-5c87ceed3ee9 None None] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 10:44:46.962626 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8292cde9-0ca8-4f16-ac14-e92dc10cbb21 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 10:44:46.963207 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8292cde9-0ca8-4f16-ac14-e92dc10cbb21 None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:44:46.963488 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8292cde9-0ca8-4f16-ac14-e92dc10cbb21 None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:44:46.963761 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8292cde9-0ca8-4f16-ac14-e92dc10cbb21 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:44:46.975083 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8292cde9-0ca8-4f16-ac14-e92dc10cbb21 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 07 10:44:46.979966 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application [None req-8292cde9-0ca8-4f16-ac14-e92dc10cbb21 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 07 10:44:46.979966 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 07 10:44:46.979966 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 07 10:44:46.979966 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 07 10:44:46.979966 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 07 10:44:46.979966 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 07 10:44:46.979966 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 07 10:44:46.979966 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 07 10:44:46.979966 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 07 10:44:46.979966 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 07 10:44:46.979966 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 07 10:44:46.979966 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 07 10:44:46.979966 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 07 10:44:46.979966 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 07 10:44:46.979966 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 07 10:44:46.979966 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 07 10:44:46.981142 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 07 10:44:46.981142 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 07 10:44:46.981142 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 07 10:44:46.981142 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 07 10:44:46.981142 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 07 10:44:46.981142 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 07 10:44:46.981142 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 07 10:44:46.981142 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 07 10:44:46.981142 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 07 10:44:46.981142 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.get_or_create( Aug 07 10:44:46.981142 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 07 10:44:46.981142 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application with Lock( Aug 07 10:44:46.981142 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 07 10:44:46.981142 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self._enter() Aug 07 10:44:46.981142 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 07 10:44:46.981142 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 07 10:44:46.982219 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 07 10:44:46.982219 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.creator() Aug 07 10:44:46.982219 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 07 10:44:46.982219 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application created_value = creator( Aug 07 10:44:46.982219 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 07 10:44:46.982219 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 07 10:44:46.982219 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 07 10:44:46.982219 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 07 10:44:46.982219 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 07 10:44:46.982219 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 07 10:44:46.982219 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 07 10:44:46.982219 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application Aug 07 10:44:46.984224 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 594/1208] 10.222.0.25 () {56 vars in 1102 bytes} [Sun Aug 7 10:44:46 2022] GET /identity/v3/roles/service => generated 85 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 07 10:44:46.990575 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77812297-ed71-43ec-a077-b09c67e2fc0a None None] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 10:44:47.034268 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 10:44:47.108652 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77812297-ed71-43ec-a077-b09c67e2fc0a None None] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 10:44:47.148467 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53a2b184-f049-4067-a29b-ff142365919a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 10:44:47.149097 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53a2b184-f049-4067-a29b-ff142365919a None admin] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:44:47.149387 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53a2b184-f049-4067-a29b-ff142365919a None admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:44:47.149658 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53a2b184-f049-4067-a29b-ff142365919a None admin] PATH_INFO: `/v3/roles` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:44:47.150331 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53a2b184-f049-4067-a29b-ff142365919a None admin] RBAC: Adding query filter params (name=service) {{(pid=80144) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 10:44:47.150643 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53a2b184-f049-4067-a29b-ff142365919a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 10:44:47.152048 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53a2b184-f049-4067-a29b-ff142365919a None admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 10:44:47.160978 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 615/1209] 10.222.0.25 () {56 vars in 1103 bytes} [Sun Aug 7 10:44:46 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 10:44:47.166949 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8292cde9-0ca8-4f16-ac14-e92dc10cbb21 None admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 10:44:47.224099 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=100 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 10:44:47.280845 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8292cde9-0ca8-4f16-ac14-e92dc10cbb21 None admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 10:44:47.320279 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ece0bbb3-c4e4-4eca-84a6-6a2f0a6d5494 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 10:44:47.320917 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ece0bbb3-c4e4-4eca-84a6-6a2f0a6d5494 None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:44:47.321203 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ece0bbb3-c4e4-4eca-84a6-6a2f0a6d5494 None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:44:47.321462 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ece0bbb3-c4e4-4eca-84a6-6a2f0a6d5494 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:44:47.334986 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ece0bbb3-c4e4-4eca-84a6-6a2f0a6d5494 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 10:44:47.336757 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ece0bbb3-c4e4-4eca-84a6-6a2f0a6d5494 None admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 10:44:47.353064 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application [None req-ece0bbb3-c4e4-4eca-84a6-6a2f0a6d5494 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 07 10:44:47.353064 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 07 10:44:47.353064 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 07 10:44:47.353064 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 07 10:44:47.353064 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 07 10:44:47.353064 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 07 10:44:47.353064 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 07 10:44:47.353064 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 07 10:44:47.353064 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 07 10:44:47.353064 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 07 10:44:47.353064 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 07 10:44:47.353064 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 07 10:44:47.353064 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 07 10:44:47.353064 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 07 10:44:47.353064 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 07 10:44:47.353064 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 07 10:44:47.354508 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 07 10:44:47.354508 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 07 10:44:47.354508 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 07 10:44:47.354508 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 07 10:44:47.354508 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 07 10:44:47.354508 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.get_or_create( Aug 07 10:44:47.354508 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 07 10:44:47.354508 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application with Lock( Aug 07 10:44:47.354508 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 07 10:44:47.354508 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self._enter() Aug 07 10:44:47.354508 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 07 10:44:47.354508 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 07 10:44:47.354508 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 07 10:44:47.354508 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.creator() Aug 07 10:44:47.354508 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 07 10:44:47.354508 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application created_value = creator( Aug 07 10:44:47.354508 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 07 10:44:47.355601 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 07 10:44:47.355601 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 07 10:44:47.355601 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application Aug 07 10:44:47.355601 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 595/1210] 10.222.0.25 () {56 vars in 1108 bytes} [Sun Aug 7 10:44:47 2022] GET /identity/v3/domains/Default => generated 87 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 07 10:44:47.360059 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53a2b184-f049-4067-a29b-ff142365919a None admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 10:44:47.476138 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=100 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 10:44:47.480498 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53a2b184-f049-4067-a29b-ff142365919a None admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 10:44:47.521064 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20efbc92-bb15-48b0-babe-4941137d1a9e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 10:44:47.522168 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20efbc92-bb15-48b0-babe-4941137d1a9e None admin] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:44:47.522687 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20efbc92-bb15-48b0-babe-4941137d1a9e None admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:44:47.523130 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20efbc92-bb15-48b0-babe-4941137d1a9e None admin] PATH_INFO: `/v3/domains` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:44:47.523797 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20efbc92-bb15-48b0-babe-4941137d1a9e None admin] RBAC: Adding query filter params (name=Default) {{(pid=80144) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 10:44:47.524111 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20efbc92-bb15-48b0-babe-4941137d1a9e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 10:44:47.525500 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20efbc92-bb15-48b0-babe-4941137d1a9e None admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 10:44:47.541312 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 616/1211] 10.222.0.25 () {56 vars in 1109 bytes} [Sun Aug 7 10:44:47 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 10:44:47.547369 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ece0bbb3-c4e4-4eca-84a6-6a2f0a6d5494 None admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 10:44:47.683591 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ece0bbb3-c4e4-4eca-84a6-6a2f0a6d5494 None admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 10:44:47.685388 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=100 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 10:44:47.731665 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3242d056-62a0-43ed-a52d-62473ae1bef9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 10:44:47.732401 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3242d056-62a0-43ed-a52d-62473ae1bef9 None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:44:47.732545 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3242d056-62a0-43ed-a52d-62473ae1bef9 None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:44:47.732962 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3242d056-62a0-43ed-a52d-62473ae1bef9 None admin] PATH_INFO: `/v3/users/placement` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:44:47.742537 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3242d056-62a0-43ed-a52d-62473ae1bef9 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 10:44:47.744058 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3242d056-62a0-43ed-a52d-62473ae1bef9 None admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 10:44:47.755528 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application [None req-3242d056-62a0-43ed-a52d-62473ae1bef9 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Aug 07 10:44:47.755528 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 07 10:44:47.755528 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 07 10:44:47.755528 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 07 10:44:47.755528 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 07 10:44:47.755528 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 07 10:44:47.755528 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 07 10:44:47.755528 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 07 10:44:47.755528 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 07 10:44:47.755528 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 07 10:44:47.755528 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 07 10:44:47.755528 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 07 10:44:47.755528 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 07 10:44:47.755528 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 07 10:44:47.755528 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 07 10:44:47.755528 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 07 10:44:47.756593 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 07 10:44:47.756593 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 07 10:44:47.756593 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 07 10:44:47.756593 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 07 10:44:47.756593 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 07 10:44:47.756593 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 07 10:44:47.756593 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 07 10:44:47.756593 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 07 10:44:47.756593 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 07 10:44:47.756593 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.get_or_create( Aug 07 10:44:47.756593 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 07 10:44:47.756593 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application with Lock( Aug 07 10:44:47.756593 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 07 10:44:47.756593 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self._enter() Aug 07 10:44:47.756593 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 07 10:44:47.756593 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 07 10:44:47.756593 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 07 10:44:47.756593 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.creator() Aug 07 10:44:47.757943 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 07 10:44:47.757943 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application created_value = creator( Aug 07 10:44:47.757943 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 07 10:44:47.757943 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 07 10:44:47.757943 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 07 10:44:47.757943 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 07 10:44:47.757943 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 07 10:44:47.757943 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 07 10:44:47.757943 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Aug 07 10:44:47.757943 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application Aug 07 10:44:47.757943 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 596/1212] 10.222.0.25 () {56 vars in 1108 bytes} [Sun Aug 7 10:44:47 2022] GET /identity/v3/users/placement => generated 87 bytes in 210 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 07 10:44:47.762754 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20efbc92-bb15-48b0-babe-4941137d1a9e None admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 10:44:47.881212 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20efbc92-bb15-48b0-babe-4941137d1a9e None admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 10:44:47.923301 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-199a0781-8512-49d3-b274-091b0ffab6af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 10:44:47.925166 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-199a0781-8512-49d3-b274-091b0ffab6af None admin] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:44:47.925166 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-199a0781-8512-49d3-b274-091b0ffab6af None admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:44:47.925166 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-199a0781-8512-49d3-b274-091b0ffab6af None admin] PATH_INFO: `/v3/users` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:44:47.925453 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-199a0781-8512-49d3-b274-091b0ffab6af None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80144) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 10:44:47.925570 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-199a0781-8512-49d3-b274-091b0ffab6af None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 10:44:47.926987 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-199a0781-8512-49d3-b274-091b0ffab6af None admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 10:44:47.949393 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 617/1213] 10.222.0.25 () {56 vars in 1143 bytes} [Sun Aug 7 10:44:47 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 07 10:44:47.956707 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3242d056-62a0-43ed-a52d-62473ae1bef9 None admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 10:44:48.079479 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3242d056-62a0-43ed-a52d-62473ae1bef9 None admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 10:44:48.122377 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7575c52a-43a6-4257-94c6-e0e15ad57da9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 10:44:48.123391 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7575c52a-43a6-4257-94c6-e0e15ad57da9 None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:44:48.123648 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7575c52a-43a6-4257-94c6-e0e15ad57da9 None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:44:48.123915 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7575c52a-43a6-4257-94c6-e0e15ad57da9 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:44:48.158348 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=100 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 10:44:48.202376 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7575c52a-43a6-4257-94c6-e0e15ad57da9 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 10:44:48.203845 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7575c52a-43a6-4257-94c6-e0e15ad57da9 None admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 10:44:48.319326 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 597/1214] 10.222.0.25 () {58 vars in 1282 bytes} [Sun Aug 7 10:44:47 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 363 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 07 10:44:48.325462 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-199a0781-8512-49d3-b274-091b0ffab6af None admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 10:44:48.345621 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=100 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 10:44:48.440636 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-199a0781-8512-49d3-b274-091b0ffab6af None admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 10:44:48.482285 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b8abe93-b72b-4a4f-9949-ee328381cf4d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 10:44:48.483238 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b8abe93-b72b-4a4f-9949-ee328381cf4d None admin] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:44:48.483511 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b8abe93-b72b-4a4f-9949-ee328381cf4d None admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:44:48.483767 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b8abe93-b72b-4a4f-9949-ee328381cf4d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:44:48.497095 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b8abe93-b72b-4a4f-9949-ee328381cf4d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 10:44:48.499098 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b8abe93-b72b-4a4f-9949-ee328381cf4d None admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 10:44:48.518758 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application [None req-2b8abe93-b72b-4a4f-9949-ee328381cf4d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 07 10:44:48.518758 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 07 10:44:48.518758 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 07 10:44:48.518758 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 07 10:44:48.518758 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 07 10:44:48.518758 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 07 10:44:48.518758 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 07 10:44:48.518758 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 07 10:44:48.518758 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 07 10:44:48.518758 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 07 10:44:48.518758 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 07 10:44:48.518758 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 07 10:44:48.518758 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 07 10:44:48.518758 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 07 10:44:48.518758 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 07 10:44:48.518758 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 07 10:44:48.520052 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 07 10:44:48.520052 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 07 10:44:48.520052 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 07 10:44:48.520052 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 07 10:44:48.520052 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 07 10:44:48.520052 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self.get_or_create( Aug 07 10:44:48.520052 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 07 10:44:48.520052 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application with Lock( Aug 07 10:44:48.520052 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 07 10:44:48.520052 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self._enter() Aug 07 10:44:48.520052 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 07 10:44:48.520052 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 07 10:44:48.520052 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 07 10:44:48.520052 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self.creator() Aug 07 10:44:48.520052 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 07 10:44:48.520052 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application created_value = creator( Aug 07 10:44:48.520052 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 07 10:44:48.521342 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 07 10:44:48.521342 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 07 10:44:48.521342 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application Aug 07 10:44:48.521342 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 618/1215] 10.222.0.25 () {56 vars in 1108 bytes} [Sun Aug 7 10:44:48 2022] GET /identity/v3/domains/Default => generated 87 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 07 10:44:48.529069 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7575c52a-43a6-4257-94c6-e0e15ad57da9 None admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 10:44:48.603071 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=100 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 10:44:48.645482 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7575c52a-43a6-4257-94c6-e0e15ad57da9 None admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 10:44:48.684027 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c5b956e-5a25-45bd-a980-f1726ac5cafc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 10:44:48.684796 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c5b956e-5a25-45bd-a980-f1726ac5cafc None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:44:48.685185 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c5b956e-5a25-45bd-a980-f1726ac5cafc None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:44:48.685576 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c5b956e-5a25-45bd-a980-f1726ac5cafc None admin] PATH_INFO: `/v3/domains` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:44:48.686352 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c5b956e-5a25-45bd-a980-f1726ac5cafc None admin] RBAC: Adding query filter params (name=Default) {{(pid=80143) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 10:44:48.686739 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c5b956e-5a25-45bd-a980-f1726ac5cafc None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 10:44:48.688189 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c5b956e-5a25-45bd-a980-f1726ac5cafc None admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 10:44:48.700033 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 598/1216] 10.222.0.25 () {56 vars in 1109 bytes} [Sun Aug 7 10:44:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 10:44:48.705563 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b8abe93-b72b-4a4f-9949-ee328381cf4d None admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 10:44:48.791685 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=100 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 10:44:48.824257 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b8abe93-b72b-4a4f-9949-ee328381cf4d None admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 10:44:48.869664 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d668716-9db4-4967-83fe-364ff4030a24 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 10:44:48.870244 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d668716-9db4-4967-83fe-364ff4030a24 None admin] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:44:48.870533 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d668716-9db4-4967-83fe-364ff4030a24 None admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:44:48.870834 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d668716-9db4-4967-83fe-364ff4030a24 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:44:48.878914 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d668716-9db4-4967-83fe-364ff4030a24 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 10:44:48.880626 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d668716-9db4-4967-83fe-364ff4030a24 None admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 10:44:48.892645 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application [None req-8d668716-9db4-4967-83fe-364ff4030a24 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 07 10:44:48.892645 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 07 10:44:48.892645 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 07 10:44:48.892645 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 07 10:44:48.892645 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 07 10:44:48.892645 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 07 10:44:48.892645 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 07 10:44:48.892645 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 07 10:44:48.892645 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 07 10:44:48.892645 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 07 10:44:48.892645 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 07 10:44:48.892645 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 07 10:44:48.892645 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 07 10:44:48.892645 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 07 10:44:48.892645 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 07 10:44:48.894026 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 07 10:44:48.894026 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 07 10:44:48.894026 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 07 10:44:48.894026 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 07 10:44:48.894026 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 07 10:44:48.894026 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 07 10:44:48.894026 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self.get_or_create( Aug 07 10:44:48.894026 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 07 10:44:48.894026 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application with Lock( Aug 07 10:44:48.894026 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 07 10:44:48.894026 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self._enter() Aug 07 10:44:48.894026 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 07 10:44:48.894026 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 07 10:44:48.894026 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 07 10:44:48.894026 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self.creator() Aug 07 10:44:48.894026 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 07 10:44:48.894026 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application created_value = creator( Aug 07 10:44:48.894964 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 07 10:44:48.894964 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 07 10:44:48.894964 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 07 10:44:48.894964 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 07 10:44:48.894964 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 07 10:44:48.894964 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 07 10:44:48.894964 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 07 10:44:48.894964 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application Aug 07 10:44:48.894964 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 619/1217] 10.222.0.25 () {56 vars in 1111 bytes} [Sun Aug 7 10:44:48 2022] GET /identity/v3/projects/service => generated 88 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 07 10:44:48.900928 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c5b956e-5a25-45bd-a980-f1726ac5cafc None admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 10:44:49.021717 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c5b956e-5a25-45bd-a980-f1726ac5cafc None admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 10:44:49.053342 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=100 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 10:44:49.070119 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b40c792-6616-4d1a-8732-f4fd86a48aba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 10:44:49.070119 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b40c792-6616-4d1a-8732-f4fd86a48aba None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:44:49.070119 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b40c792-6616-4d1a-8732-f4fd86a48aba None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:44:49.070560 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b40c792-6616-4d1a-8732-f4fd86a48aba None admin] PATH_INFO: `/v3/projects` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:44:49.070948 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b40c792-6616-4d1a-8732-f4fd86a48aba None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80143) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 10:44:49.071256 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b40c792-6616-4d1a-8732-f4fd86a48aba None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 10:44:49.072663 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b40c792-6616-4d1a-8732-f4fd86a48aba None admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 10:44:49.084868 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 599/1218] 10.222.0.25 () {56 vars in 1148 bytes} [Sun Aug 7 10:44:48 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 10:44:49.090659 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d668716-9db4-4967-83fe-364ff4030a24 None admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 10:44:49.212279 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d668716-9db4-4967-83fe-364ff4030a24 None admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 10:44:49.243859 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=100 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 10:44:49.251745 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97596a3c-58fe-46e7-97ac-091eccf6e745 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 10:44:49.252415 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97596a3c-58fe-46e7-97ac-091eccf6e745 None admin] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:44:49.252760 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97596a3c-58fe-46e7-97ac-091eccf6e745 None admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:44:49.253099 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97596a3c-58fe-46e7-97ac-091eccf6e745 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:44:49.253897 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97596a3c-58fe-46e7-97ac-091eccf6e745 None admin] RBAC: Adding query filter params (role.id=fbf375e9760340c9aa9529333eeb25f9, scope.project.id=6de1f3cac02c4bcdb05dd2337176033d, user.id=6ee2e4c08af04c219ed395d7ca7085a6) {{(pid=80144) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 10:44:49.254283 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97596a3c-58fe-46e7-97ac-091eccf6e745 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 10:44:49.256324 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97596a3c-58fe-46e7-97ac-091eccf6e745 None admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 10:44:49.271444 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 620/1219] 10.222.0.25 () {56 vars in 1374 bytes} [Sun Aug 7 10:44:49 2022] GET /identity/v3/role_assignments?user.id=6ee2e4c08af04c219ed395d7ca7085a6&scope.project.id=6de1f3cac02c4bcdb05dd2337176033d&role.id=fbf375e9760340c9aa9529333eeb25f9 => generated 259 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 10:44:51.260440 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75ecd9b5-8248-4a45-9291-efa53cbbb6cf None None] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:44:51.261139 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75ecd9b5-8248-4a45-9291-efa53cbbb6cf None None] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:44:51.261139 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75ecd9b5-8248-4a45-9291-efa53cbbb6cf None None] PATH_INFO: `/` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:44:51.262292 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 600/1220] 10.222.0.25 () {54 vars in 927 bytes} [Sun Aug 7 10:44:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 07 10:44:51.267820 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62f0f1c5-ec4f-4ed5-8bcd-570752c52524 None None] REQUEST_METHOD: `POST` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:44:51.268277 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62f0f1c5-ec4f-4ed5-8bcd-570752c52524 None None] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:44:51.268453 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62f0f1c5-ec4f-4ed5-8bcd-570752c52524 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:44:51.291113 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.auth.core [None req-62f0f1c5-ec4f-4ed5-8bcd-570752c52524 None None] MFA Rules not processed for user `bac4e6b76fe64217ac8fa44e71cffe6a`. Rule list: `[]` (Enabled: `True`). {{(pid=80144) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 10:44:51.292175 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.fernet_utils [None req-62f0f1c5-ec4f-4ed5-8bcd-570752c52524 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80144) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 10:44:51.371450 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 621/1221] 10.222.0.25 () {58 vars in 1024 bytes} [Sun Aug 7 10:44:51 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 07 10:44:51.394481 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75ecd9b5-8248-4a45-9291-efa53cbbb6cf None None] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 10:44:51.511454 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75ecd9b5-8248-4a45-9291-efa53cbbb6cf None None] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 10:44:51.549764 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0f850de-b6eb-4000-bb21-766b30100c74 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 10:44:51.551208 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0f850de-b6eb-4000-bb21-766b30100c74 None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:44:51.551490 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0f850de-b6eb-4000-bb21-766b30100c74 None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:44:51.551797 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0f850de-b6eb-4000-bb21-766b30100c74 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:44:51.563958 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0f850de-b6eb-4000-bb21-766b30100c74 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 07 10:44:51.569107 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application [None req-e0f850de-b6eb-4000-bb21-766b30100c74 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 07 10:44:51.569107 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 07 10:44:51.569107 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 07 10:44:51.569107 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 07 10:44:51.569107 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 07 10:44:51.569107 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 07 10:44:51.569107 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 07 10:44:51.569107 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 07 10:44:51.569107 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 07 10:44:51.569107 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 07 10:44:51.569107 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 07 10:44:51.569107 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 07 10:44:51.569107 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 07 10:44:51.569107 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 07 10:44:51.569107 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 07 10:44:51.569107 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 07 10:44:51.570267 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 07 10:44:51.570267 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 07 10:44:51.570267 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 07 10:44:51.570267 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 07 10:44:51.570267 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 07 10:44:51.570267 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 07 10:44:51.570267 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 07 10:44:51.570267 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 07 10:44:51.570267 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 07 10:44:51.570267 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.get_or_create( Aug 07 10:44:51.570267 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 07 10:44:51.570267 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application with Lock( Aug 07 10:44:51.570267 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 07 10:44:51.570267 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self._enter() Aug 07 10:44:51.570267 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 07 10:44:51.570267 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 07 10:44:51.571407 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 07 10:44:51.571407 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.creator() Aug 07 10:44:51.571407 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 07 10:44:51.571407 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application created_value = creator( Aug 07 10:44:51.571407 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 07 10:44:51.571407 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 07 10:44:51.571407 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 07 10:44:51.571407 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 07 10:44:51.571407 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 07 10:44:51.571407 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 07 10:44:51.571407 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 07 10:44:51.571407 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application Aug 07 10:44:51.571407 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 601/1222] 10.222.0.25 () {56 vars in 1102 bytes} [Sun Aug 7 10:44:51 2022] GET /identity/v3/roles/service => generated 85 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 07 10:44:51.576472 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62f0f1c5-ec4f-4ed5-8bcd-570752c52524 None None] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 10:44:51.696081 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62f0f1c5-ec4f-4ed5-8bcd-570752c52524 None None] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 10:44:51.721970 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=100 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 10:44:51.733813 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5845d05-6c2f-415c-bf78-a13a6e32bf3c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 10:44:51.734678 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5845d05-6c2f-415c-bf78-a13a6e32bf3c None admin] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:44:51.734979 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5845d05-6c2f-415c-bf78-a13a6e32bf3c None admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:44:51.735256 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5845d05-6c2f-415c-bf78-a13a6e32bf3c None admin] PATH_INFO: `/v3/roles` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:44:51.735932 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5845d05-6c2f-415c-bf78-a13a6e32bf3c None admin] RBAC: Adding query filter params (name=service) {{(pid=80144) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 10:44:51.736239 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5845d05-6c2f-415c-bf78-a13a6e32bf3c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 10:44:51.737607 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5845d05-6c2f-415c-bf78-a13a6e32bf3c None admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 10:44:51.747481 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 622/1223] 10.222.0.25 () {56 vars in 1103 bytes} [Sun Aug 7 10:44:51 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 10:44:51.753638 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0f850de-b6eb-4000-bb21-766b30100c74 None admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 10:44:51.802331 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=100 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 10:44:51.862904 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0f850de-b6eb-4000-bb21-766b30100c74 None admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 10:44:51.898339 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7962931-e98c-44f3-aa74-35b685a02a07 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 10:44:51.899025 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7962931-e98c-44f3-aa74-35b685a02a07 None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:44:51.899313 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7962931-e98c-44f3-aa74-35b685a02a07 None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:44:51.899588 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7962931-e98c-44f3-aa74-35b685a02a07 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:44:51.910247 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7962931-e98c-44f3-aa74-35b685a02a07 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 10:44:51.911827 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7962931-e98c-44f3-aa74-35b685a02a07 None admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 10:44:51.926059 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application [None req-a7962931-e98c-44f3-aa74-35b685a02a07 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 07 10:44:51.926059 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 07 10:44:51.926059 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 07 10:44:51.926059 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 07 10:44:51.926059 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 07 10:44:51.926059 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 07 10:44:51.926059 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 07 10:44:51.926059 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 07 10:44:51.926059 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 07 10:44:51.926059 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 07 10:44:51.926059 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 07 10:44:51.926059 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 07 10:44:51.926059 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 07 10:44:51.926059 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 07 10:44:51.926059 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 07 10:44:51.926059 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 07 10:44:51.927429 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 07 10:44:51.927429 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 07 10:44:51.927429 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 07 10:44:51.927429 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 07 10:44:51.927429 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 07 10:44:51.927429 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.get_or_create( Aug 07 10:44:51.927429 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 07 10:44:51.927429 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application with Lock( Aug 07 10:44:51.927429 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 07 10:44:51.927429 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self._enter() Aug 07 10:44:51.927429 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 07 10:44:51.927429 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 07 10:44:51.927429 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 07 10:44:51.927429 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.creator() Aug 07 10:44:51.927429 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 07 10:44:51.927429 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application created_value = creator( Aug 07 10:44:51.927429 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 07 10:44:51.928685 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 07 10:44:51.928685 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 07 10:44:51.928685 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application Aug 07 10:44:51.928685 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 602/1224] 10.222.0.25 () {56 vars in 1108 bytes} [Sun Aug 7 10:44:51 2022] GET /identity/v3/domains/Default => generated 87 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 07 10:44:51.933535 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5845d05-6c2f-415c-bf78-a13a6e32bf3c None admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 10:44:52.044019 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5845d05-6c2f-415c-bf78-a13a6e32bf3c None admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 10:44:52.080215 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-363ff338-17d1-46b2-b928-1389f9583462 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 10:44:52.081183 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-363ff338-17d1-46b2-b928-1389f9583462 None admin] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:44:52.081692 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-363ff338-17d1-46b2-b928-1389f9583462 None admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:44:52.082201 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-363ff338-17d1-46b2-b928-1389f9583462 None admin] PATH_INFO: `/v3/domains` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:44:52.084040 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-363ff338-17d1-46b2-b928-1389f9583462 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80144) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 10:44:52.084562 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-363ff338-17d1-46b2-b928-1389f9583462 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 10:44:52.087582 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-363ff338-17d1-46b2-b928-1389f9583462 None admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 10:44:52.100126 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 623/1225] 10.222.0.25 () {56 vars in 1109 bytes} [Sun Aug 7 10:44:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 10:44:52.106217 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7962931-e98c-44f3-aa74-35b685a02a07 None admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 10:44:52.205594 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=100 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 10:44:52.213366 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7962931-e98c-44f3-aa74-35b685a02a07 None admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 10:44:52.249778 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e69f54df-2d18-46ff-8e16-3f38d9e25fc4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 10:44:52.250409 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e69f54df-2d18-46ff-8e16-3f38d9e25fc4 None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:44:52.250689 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e69f54df-2d18-46ff-8e16-3f38d9e25fc4 None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:44:52.250989 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e69f54df-2d18-46ff-8e16-3f38d9e25fc4 None admin] PATH_INFO: `/v3/users/placement` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:44:52.259040 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e69f54df-2d18-46ff-8e16-3f38d9e25fc4 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 10:44:52.260467 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e69f54df-2d18-46ff-8e16-3f38d9e25fc4 None admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 10:44:52.271752 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application [None req-e69f54df-2d18-46ff-8e16-3f38d9e25fc4 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Aug 07 10:44:52.271752 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 07 10:44:52.271752 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 07 10:44:52.271752 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 07 10:44:52.271752 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 07 10:44:52.271752 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 07 10:44:52.271752 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 07 10:44:52.271752 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 07 10:44:52.271752 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 07 10:44:52.271752 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 07 10:44:52.271752 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 07 10:44:52.271752 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 07 10:44:52.271752 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 07 10:44:52.271752 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 07 10:44:52.271752 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 07 10:44:52.271752 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 07 10:44:52.278446 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 07 10:44:52.278446 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 07 10:44:52.278446 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 07 10:44:52.278446 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 07 10:44:52.278446 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 07 10:44:52.278446 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 07 10:44:52.278446 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 07 10:44:52.278446 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 07 10:44:52.278446 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 07 10:44:52.278446 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.get_or_create( Aug 07 10:44:52.278446 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 07 10:44:52.278446 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application with Lock( Aug 07 10:44:52.278446 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 07 10:44:52.278446 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self._enter() Aug 07 10:44:52.278446 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 07 10:44:52.278446 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 07 10:44:52.278446 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 07 10:44:52.278446 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.creator() Aug 07 10:44:52.279959 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 07 10:44:52.279959 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application created_value = creator( Aug 07 10:44:52.279959 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 07 10:44:52.279959 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 07 10:44:52.279959 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 07 10:44:52.279959 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 07 10:44:52.279959 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 07 10:44:52.279959 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 07 10:44:52.279959 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Aug 07 10:44:52.279959 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application Aug 07 10:44:52.279959 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 603/1226] 10.222.0.25 () {56 vars in 1108 bytes} [Sun Aug 7 10:44:52 2022] GET /identity/v3/users/placement => generated 87 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 07 10:44:52.280701 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-363ff338-17d1-46b2-b928-1389f9583462 None admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 10:44:52.327227 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=100 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 10:44:52.383262 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-363ff338-17d1-46b2-b928-1389f9583462 None admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 10:44:52.418362 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d855a70-0bc0-4a08-8540-27eadaf0e87b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 10:44:52.419492 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d855a70-0bc0-4a08-8540-27eadaf0e87b None admin] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:44:52.419581 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d855a70-0bc0-4a08-8540-27eadaf0e87b None admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:44:52.419826 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d855a70-0bc0-4a08-8540-27eadaf0e87b None admin] PATH_INFO: `/v3/users` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:44:52.420554 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d855a70-0bc0-4a08-8540-27eadaf0e87b None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80144) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 10:44:52.420867 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d855a70-0bc0-4a08-8540-27eadaf0e87b None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 10:44:52.422405 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d855a70-0bc0-4a08-8540-27eadaf0e87b None admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 10:44:52.448139 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 624/1227] 10.222.0.25 () {56 vars in 1143 bytes} [Sun Aug 7 10:44:52 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 10:44:52.449317 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e69f54df-2d18-46ff-8e16-3f38d9e25fc4 None admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 10:44:52.553946 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e69f54df-2d18-46ff-8e16-3f38d9e25fc4 None admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 10:44:52.588098 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab0a5b94-22ee-4e37-933e-07422ce1fae5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 10:44:52.588731 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab0a5b94-22ee-4e37-933e-07422ce1fae5 None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:44:52.589014 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab0a5b94-22ee-4e37-933e-07422ce1fae5 None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:44:52.589290 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab0a5b94-22ee-4e37-933e-07422ce1fae5 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:44:52.600436 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab0a5b94-22ee-4e37-933e-07422ce1fae5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 10:44:52.602013 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab0a5b94-22ee-4e37-933e-07422ce1fae5 None admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 10:44:52.617430 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application [None req-ab0a5b94-22ee-4e37-933e-07422ce1fae5 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 07 10:44:52.617430 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 07 10:44:52.617430 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 07 10:44:52.617430 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 07 10:44:52.617430 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 07 10:44:52.617430 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 07 10:44:52.617430 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 07 10:44:52.617430 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 07 10:44:52.617430 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 07 10:44:52.617430 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 07 10:44:52.617430 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 07 10:44:52.617430 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 07 10:44:52.617430 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 07 10:44:52.617430 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 07 10:44:52.617430 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 07 10:44:52.617430 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 07 10:44:52.623368 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 07 10:44:52.623368 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 07 10:44:52.623368 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 07 10:44:52.623368 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 07 10:44:52.623368 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 07 10:44:52.623368 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.get_or_create( Aug 07 10:44:52.623368 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 07 10:44:52.623368 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application with Lock( Aug 07 10:44:52.623368 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 07 10:44:52.623368 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self._enter() Aug 07 10:44:52.623368 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 07 10:44:52.623368 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 07 10:44:52.623368 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 07 10:44:52.623368 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.creator() Aug 07 10:44:52.623368 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 07 10:44:52.623368 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application created_value = creator( Aug 07 10:44:52.623368 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 07 10:44:52.624501 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 07 10:44:52.624501 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 07 10:44:52.624501 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application Aug 07 10:44:52.624501 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 604/1228] 10.222.0.25 () {56 vars in 1108 bytes} [Sun Aug 7 10:44:52 2022] GET /identity/v3/domains/Default => generated 87 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 07 10:44:52.624773 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d855a70-0bc0-4a08-8540-27eadaf0e87b None admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 10:44:52.734982 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d855a70-0bc0-4a08-8540-27eadaf0e87b None admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 10:44:52.737436 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=100 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 10:44:52.776038 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-220946e2-4a24-46ba-847a-adbb346012b9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 10:44:52.776739 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-220946e2-4a24-46ba-847a-adbb346012b9 None admin] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:44:52.777024 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-220946e2-4a24-46ba-847a-adbb346012b9 None admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:44:52.777336 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-220946e2-4a24-46ba-847a-adbb346012b9 None admin] PATH_INFO: `/v3/domains` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:44:52.778016 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-220946e2-4a24-46ba-847a-adbb346012b9 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80144) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 10:44:52.778359 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-220946e2-4a24-46ba-847a-adbb346012b9 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 10:44:52.779764 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-220946e2-4a24-46ba-847a-adbb346012b9 None admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 10:44:52.792069 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 625/1229] 10.222.0.25 () {56 vars in 1109 bytes} [Sun Aug 7 10:44:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 07 10:44:52.798373 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab0a5b94-22ee-4e37-933e-07422ce1fae5 None admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 10:44:52.808809 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=100 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 10:44:52.912320 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab0a5b94-22ee-4e37-933e-07422ce1fae5 None admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 10:44:52.949581 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8007d13b-8ee6-4719-98b2-5b1e4a651627 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 10:44:52.950219 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8007d13b-8ee6-4719-98b2-5b1e4a651627 None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:44:52.950484 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8007d13b-8ee6-4719-98b2-5b1e4a651627 None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:44:52.950764 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8007d13b-8ee6-4719-98b2-5b1e4a651627 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:44:52.957688 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8007d13b-8ee6-4719-98b2-5b1e4a651627 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 10:44:52.959217 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8007d13b-8ee6-4719-98b2-5b1e4a651627 None admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 10:44:52.970315 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application [None req-8007d13b-8ee6-4719-98b2-5b1e4a651627 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 07 10:44:52.970315 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 07 10:44:52.970315 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 07 10:44:52.970315 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 07 10:44:52.970315 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 07 10:44:52.970315 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 07 10:44:52.970315 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 07 10:44:52.970315 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 07 10:44:52.970315 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 07 10:44:52.970315 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 07 10:44:52.970315 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 07 10:44:52.970315 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 07 10:44:52.970315 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 07 10:44:52.970315 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 07 10:44:52.970315 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 07 10:44:52.971403 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 07 10:44:52.971403 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 07 10:44:52.971403 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 07 10:44:52.971403 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 07 10:44:52.971403 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 07 10:44:52.971403 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 07 10:44:52.971403 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.get_or_create( Aug 07 10:44:52.971403 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 07 10:44:52.971403 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application with Lock( Aug 07 10:44:52.971403 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 07 10:44:52.971403 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self._enter() Aug 07 10:44:52.971403 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 07 10:44:52.971403 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 07 10:44:52.971403 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 07 10:44:52.971403 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.creator() Aug 07 10:44:52.971403 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 07 10:44:52.971403 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application created_value = creator( Aug 07 10:44:52.972313 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 07 10:44:52.972313 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 07 10:44:52.972313 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 07 10:44:52.972313 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 07 10:44:52.972313 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 07 10:44:52.972313 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 07 10:44:52.972313 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 07 10:44:52.972313 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application Aug 07 10:44:52.972313 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 605/1230] 10.222.0.25 () {56 vars in 1111 bytes} [Sun Aug 7 10:44:52 2022] GET /identity/v3/projects/service => generated 88 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 07 10:44:52.978605 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-220946e2-4a24-46ba-847a-adbb346012b9 None admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 10:44:53.097950 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-220946e2-4a24-46ba-847a-adbb346012b9 None admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 10:44:53.135652 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5d5190f-e90d-48ac-bb22-3305156739f7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 10:44:53.136387 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5d5190f-e90d-48ac-bb22-3305156739f7 None admin] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:44:53.136763 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5d5190f-e90d-48ac-bb22-3305156739f7 None admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:44:53.137288 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5d5190f-e90d-48ac-bb22-3305156739f7 None admin] PATH_INFO: `/v3/projects` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:44:53.138222 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5d5190f-e90d-48ac-bb22-3305156739f7 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80144) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 10:44:53.138814 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5d5190f-e90d-48ac-bb22-3305156739f7 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 10:44:53.140604 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5d5190f-e90d-48ac-bb22-3305156739f7 None admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 10:44:53.155130 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 626/1231] 10.222.0.25 () {56 vars in 1148 bytes} [Sun Aug 7 10:44:52 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 10:44:53.163383 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8007d13b-8ee6-4719-98b2-5b1e4a651627 None admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 10:44:53.237030 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=100 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 10:44:53.277093 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8007d13b-8ee6-4719-98b2-5b1e4a651627 None admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 10:44:53.314461 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03333da1-eaad-4d29-a03d-5ab60f1d48da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 10:44:53.315176 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03333da1-eaad-4d29-a03d-5ab60f1d48da None admin] REQUEST_METHOD: `PUT` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:44:53.315456 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03333da1-eaad-4d29-a03d-5ab60f1d48da None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:44:53.315759 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03333da1-eaad-4d29-a03d-5ab60f1d48da None admin] PATH_INFO: `/v3/projects/6de1f3cac02c4bcdb05dd2337176033d/users/6ee2e4c08af04c219ed395d7ca7085a6/roles/fbf375e9760340c9aa9529333eeb25f9` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:44:53.331488 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03333da1-eaad-4d29-a03d-5ab60f1d48da None admin] RBAC: Authorizing `identity:create_grant(project_id=6de1f3cac02c4bcdb05dd2337176033d, user_id=6ee2e4c08af04c219ed395d7ca7085a6, role_id=fbf375e9760340c9aa9529333eeb25f9)` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 10:44:53.332929 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03333da1-eaad-4d29-a03d-5ab60f1d48da None admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 10:44:53.349900 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 606/1232] 10.222.0.25 () {58 vars in 1439 bytes} [Sun Aug 7 10:44:53 2022] PUT /identity/v3/projects/6de1f3cac02c4bcdb05dd2337176033d/users/6ee2e4c08af04c219ed395d7ca7085a6/roles/fbf375e9760340c9aa9529333eeb25f9 => generated 0 bytes in 187 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 07 10:44:55.335837 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5652ca4d-4088-4ed1-b431-ef85c40697fd None None] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:44:55.336391 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5652ca4d-4088-4ed1-b431-ef85c40697fd None None] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:44:55.336391 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5652ca4d-4088-4ed1-b431-ef85c40697fd None None] PATH_INFO: `/` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:44:55.337732 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 627/1233] 10.222.0.25 () {54 vars in 927 bytes} [Sun Aug 7 10:44:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 07 10:44:55.343176 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63cd311f-6c1f-4650-9674-7581ea00f8b1 None None] REQUEST_METHOD: `POST` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:44:55.343633 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63cd311f-6c1f-4650-9674-7581ea00f8b1 None None] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:44:55.343734 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63cd311f-6c1f-4650-9674-7581ea00f8b1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:44:55.361644 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.auth.core [None req-63cd311f-6c1f-4650-9674-7581ea00f8b1 None None] MFA Rules not processed for user `bac4e6b76fe64217ac8fa44e71cffe6a`. Rule list: `[]` (Enabled: `True`). {{(pid=80143) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 10:44:55.362839 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-63cd311f-6c1f-4650-9674-7581ea00f8b1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 10:44:55.444708 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 607/1234] 10.222.0.25 () {58 vars in 1024 bytes} [Sun Aug 7 10:44:55 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 07 10:44:55.468424 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5652ca4d-4088-4ed1-b431-ef85c40697fd None None] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 10:44:55.494234 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=100 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 10:44:55.573536 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5652ca4d-4088-4ed1-b431-ef85c40697fd None None] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 10:44:55.609173 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2008cecd-06a3-465a-b8cd-110f49d3d43e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 10:44:55.609992 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2008cecd-06a3-465a-b8cd-110f49d3d43e None admin] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:44:55.610494 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2008cecd-06a3-465a-b8cd-110f49d3d43e None admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:44:55.610899 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2008cecd-06a3-465a-b8cd-110f49d3d43e None admin] PATH_INFO: `/v3/roles/service` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:44:55.621978 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2008cecd-06a3-465a-b8cd-110f49d3d43e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 07 10:44:55.626745 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application [None req-2008cecd-06a3-465a-b8cd-110f49d3d43e None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 07 10:44:55.626745 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 07 10:44:55.626745 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 07 10:44:55.626745 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 07 10:44:55.626745 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 07 10:44:55.626745 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 07 10:44:55.626745 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 07 10:44:55.626745 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 07 10:44:55.626745 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 07 10:44:55.626745 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 07 10:44:55.626745 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 07 10:44:55.626745 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 07 10:44:55.626745 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 07 10:44:55.626745 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 07 10:44:55.626745 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 07 10:44:55.626745 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 07 10:44:55.627765 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 07 10:44:55.627765 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 07 10:44:55.627765 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 07 10:44:55.627765 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 07 10:44:55.627765 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 07 10:44:55.627765 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 07 10:44:55.627765 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 07 10:44:55.627765 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 07 10:44:55.627765 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 07 10:44:55.627765 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self.get_or_create( Aug 07 10:44:55.627765 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 07 10:44:55.627765 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application with Lock( Aug 07 10:44:55.627765 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 07 10:44:55.627765 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self._enter() Aug 07 10:44:55.627765 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 07 10:44:55.627765 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 07 10:44:55.628629 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 07 10:44:55.628629 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self.creator() Aug 07 10:44:55.628629 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 07 10:44:55.628629 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application created_value = creator( Aug 07 10:44:55.628629 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 07 10:44:55.628629 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 07 10:44:55.628629 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 07 10:44:55.628629 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 07 10:44:55.628629 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 07 10:44:55.628629 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 07 10:44:55.628629 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 07 10:44:55.628629 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application Aug 07 10:44:55.628629 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 628/1235] 10.222.0.25 () {56 vars in 1102 bytes} [Sun Aug 7 10:44:55 2022] GET /identity/v3/roles/service => generated 85 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 07 10:44:55.635479 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63cd311f-6c1f-4650-9674-7581ea00f8b1 None None] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 10:44:55.688880 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 10:44:55.751069 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63cd311f-6c1f-4650-9674-7581ea00f8b1 None None] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 10:44:55.790202 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c203eb50-7482-4077-a46b-b69102db2ad7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 10:44:55.790816 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c203eb50-7482-4077-a46b-b69102db2ad7 None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:44:55.791129 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c203eb50-7482-4077-a46b-b69102db2ad7 None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:44:55.791402 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c203eb50-7482-4077-a46b-b69102db2ad7 None admin] PATH_INFO: `/v3/roles` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:44:55.792096 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c203eb50-7482-4077-a46b-b69102db2ad7 None admin] RBAC: Adding query filter params (name=service) {{(pid=80143) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 10:44:55.792401 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c203eb50-7482-4077-a46b-b69102db2ad7 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 10:44:55.793761 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c203eb50-7482-4077-a46b-b69102db2ad7 None admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 10:44:55.802076 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 608/1236] 10.222.0.25 () {56 vars in 1103 bytes} [Sun Aug 7 10:44:55 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 10:44:55.807668 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2008cecd-06a3-465a-b8cd-110f49d3d43e None admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 10:44:55.901909 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=100 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 10:44:55.918113 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2008cecd-06a3-465a-b8cd-110f49d3d43e None admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 10:44:55.952590 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caae9f94-a439-4e6d-a85e-ab848a0c15d8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 10:44:55.953214 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caae9f94-a439-4e6d-a85e-ab848a0c15d8 None admin] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:44:55.953491 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caae9f94-a439-4e6d-a85e-ab848a0c15d8 None admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:44:55.953758 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caae9f94-a439-4e6d-a85e-ab848a0c15d8 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:44:55.963479 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caae9f94-a439-4e6d-a85e-ab848a0c15d8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 10:44:55.964902 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caae9f94-a439-4e6d-a85e-ab848a0c15d8 None admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 10:44:55.981837 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application [None req-caae9f94-a439-4e6d-a85e-ab848a0c15d8 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 07 10:44:55.981837 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 07 10:44:55.981837 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 07 10:44:55.981837 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 07 10:44:55.981837 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 07 10:44:55.981837 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 07 10:44:55.981837 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 07 10:44:55.981837 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 07 10:44:55.981837 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 07 10:44:55.981837 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 07 10:44:55.981837 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 07 10:44:55.981837 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 07 10:44:55.981837 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 07 10:44:55.981837 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 07 10:44:55.981837 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 07 10:44:55.981837 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 07 10:44:55.982850 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 07 10:44:55.982850 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 07 10:44:55.982850 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 07 10:44:55.982850 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 07 10:44:55.982850 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 07 10:44:55.982850 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self.get_or_create( Aug 07 10:44:55.982850 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 07 10:44:55.982850 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application with Lock( Aug 07 10:44:55.982850 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 07 10:44:55.982850 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self._enter() Aug 07 10:44:55.982850 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 07 10:44:55.982850 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 07 10:44:55.982850 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 07 10:44:55.982850 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self.creator() Aug 07 10:44:55.982850 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 07 10:44:55.982850 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application created_value = creator( Aug 07 10:44:55.982850 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 07 10:44:55.983817 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 07 10:44:55.983817 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 07 10:44:55.983817 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application Aug 07 10:44:55.983817 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 629/1237] 10.222.0.25 () {56 vars in 1108 bytes} [Sun Aug 7 10:44:55 2022] GET /identity/v3/domains/Default => generated 87 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 07 10:44:55.988740 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c203eb50-7482-4077-a46b-b69102db2ad7 None admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 10:44:56.091800 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c203eb50-7482-4077-a46b-b69102db2ad7 None admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 10:44:56.101371 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=100 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 10:44:56.140645 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf2777fb-d598-491f-9bd0-9e7ac735f24d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 10:44:56.141253 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf2777fb-d598-491f-9bd0-9e7ac735f24d None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:44:56.141531 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf2777fb-d598-491f-9bd0-9e7ac735f24d None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:44:56.141802 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf2777fb-d598-491f-9bd0-9e7ac735f24d None admin] PATH_INFO: `/v3/domains` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:44:56.142442 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf2777fb-d598-491f-9bd0-9e7ac735f24d None admin] RBAC: Adding query filter params (name=Default) {{(pid=80143) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 10:44:56.142748 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf2777fb-d598-491f-9bd0-9e7ac735f24d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 10:44:56.144335 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf2777fb-d598-491f-9bd0-9e7ac735f24d None admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 10:44:56.157522 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 609/1238] 10.222.0.25 () {56 vars in 1109 bytes} [Sun Aug 7 10:44:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 10:44:56.163645 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caae9f94-a439-4e6d-a85e-ab848a0c15d8 None admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 10:44:56.271322 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caae9f94-a439-4e6d-a85e-ab848a0c15d8 None admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 10:44:56.308451 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72626513-a158-444b-9ca7-3554fde69a57 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 10:44:56.309121 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72626513-a158-444b-9ca7-3554fde69a57 None admin] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:44:56.309631 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72626513-a158-444b-9ca7-3554fde69a57 None admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:44:56.309631 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72626513-a158-444b-9ca7-3554fde69a57 None admin] PATH_INFO: `/v3/users/placement` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:44:56.312565 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=100 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 10:44:56.319057 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72626513-a158-444b-9ca7-3554fde69a57 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 10:44:56.320755 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72626513-a158-444b-9ca7-3554fde69a57 None admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 10:44:56.334743 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application [None req-72626513-a158-444b-9ca7-3554fde69a57 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Aug 07 10:44:56.334743 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 07 10:44:56.334743 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 07 10:44:56.334743 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 07 10:44:56.334743 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 07 10:44:56.334743 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 07 10:44:56.334743 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 07 10:44:56.334743 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 07 10:44:56.334743 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 07 10:44:56.334743 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 07 10:44:56.334743 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 07 10:44:56.334743 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 07 10:44:56.334743 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 07 10:44:56.334743 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 07 10:44:56.334743 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 07 10:44:56.334743 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 07 10:44:56.336388 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 07 10:44:56.336388 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 07 10:44:56.336388 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 07 10:44:56.336388 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 07 10:44:56.336388 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 07 10:44:56.336388 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 07 10:44:56.336388 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 07 10:44:56.336388 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 07 10:44:56.336388 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 07 10:44:56.336388 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self.get_or_create( Aug 07 10:44:56.336388 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 07 10:44:56.336388 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application with Lock( Aug 07 10:44:56.336388 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 07 10:44:56.336388 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self._enter() Aug 07 10:44:56.336388 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 07 10:44:56.336388 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 07 10:44:56.336388 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 07 10:44:56.336388 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self.creator() Aug 07 10:44:56.337571 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 07 10:44:56.337571 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application created_value = creator( Aug 07 10:44:56.337571 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 07 10:44:56.337571 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 07 10:44:56.337571 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 07 10:44:56.337571 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 07 10:44:56.337571 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 07 10:44:56.337571 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 07 10:44:56.337571 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Aug 07 10:44:56.337571 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application Aug 07 10:44:56.337571 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 630/1239] 10.222.0.25 () {56 vars in 1108 bytes} [Sun Aug 7 10:44:56 2022] GET /identity/v3/users/placement => generated 87 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 07 10:44:56.342074 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf2777fb-d598-491f-9bd0-9e7ac735f24d None admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 10:44:56.456324 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf2777fb-d598-491f-9bd0-9e7ac735f24d None admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 10:44:56.493327 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d20cd8d7-556c-4636-9ba3-f6faed540b13 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 10:44:56.494039 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d20cd8d7-556c-4636-9ba3-f6faed540b13 None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:44:56.494353 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d20cd8d7-556c-4636-9ba3-f6faed540b13 None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:44:56.494629 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d20cd8d7-556c-4636-9ba3-f6faed540b13 None admin] PATH_INFO: `/v3/users` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:44:56.495459 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d20cd8d7-556c-4636-9ba3-f6faed540b13 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80143) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 10:44:56.495829 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d20cd8d7-556c-4636-9ba3-f6faed540b13 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 10:44:56.497513 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d20cd8d7-556c-4636-9ba3-f6faed540b13 None admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 10:44:56.518026 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 610/1240] 10.222.0.25 () {56 vars in 1143 bytes} [Sun Aug 7 10:44:56 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 10:44:56.524957 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72626513-a158-444b-9ca7-3554fde69a57 None admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 10:44:56.629879 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72626513-a158-444b-9ca7-3554fde69a57 None admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 10:44:56.666878 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37bce572-e3a3-4283-bbc4-1aba3fe038a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 10:44:56.667507 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37bce572-e3a3-4283-bbc4-1aba3fe038a9 None admin] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:44:56.667788 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37bce572-e3a3-4283-bbc4-1aba3fe038a9 None admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:44:56.668060 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37bce572-e3a3-4283-bbc4-1aba3fe038a9 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:44:56.733258 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=100 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 10:44:56.740471 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37bce572-e3a3-4283-bbc4-1aba3fe038a9 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 10:44:56.741906 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37bce572-e3a3-4283-bbc4-1aba3fe038a9 None admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 10:44:56.850645 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 631/1241] 10.222.0.25 () {58 vars in 1282 bytes} [Sun Aug 7 10:44:56 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 327 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 07 10:44:56.856767 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d20cd8d7-556c-4636-9ba3-f6faed540b13 None admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 10:44:56.887456 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=100 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 10:44:56.975616 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d20cd8d7-556c-4636-9ba3-f6faed540b13 None admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 10:44:57.018169 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81d2141f-999d-4854-bc6b-d9f148ab69aa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 10:44:57.018798 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81d2141f-999d-4854-bc6b-d9f148ab69aa None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:44:57.019118 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81d2141f-999d-4854-bc6b-d9f148ab69aa None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:44:57.019390 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81d2141f-999d-4854-bc6b-d9f148ab69aa None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:44:57.032146 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81d2141f-999d-4854-bc6b-d9f148ab69aa None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 10:44:57.033605 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81d2141f-999d-4854-bc6b-d9f148ab69aa None admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 10:44:57.052695 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application [None req-81d2141f-999d-4854-bc6b-d9f148ab69aa None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 07 10:44:57.052695 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 07 10:44:57.052695 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 07 10:44:57.052695 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 07 10:44:57.052695 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 07 10:44:57.052695 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 07 10:44:57.052695 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 07 10:44:57.052695 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 07 10:44:57.052695 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 07 10:44:57.052695 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 07 10:44:57.052695 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 07 10:44:57.052695 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 07 10:44:57.052695 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 07 10:44:57.052695 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 07 10:44:57.052695 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 07 10:44:57.052695 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 07 10:44:57.053808 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 07 10:44:57.053808 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 07 10:44:57.053808 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 07 10:44:57.053808 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 07 10:44:57.053808 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 07 10:44:57.053808 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.get_or_create( Aug 07 10:44:57.053808 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 07 10:44:57.053808 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application with Lock( Aug 07 10:44:57.053808 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 07 10:44:57.053808 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self._enter() Aug 07 10:44:57.053808 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 07 10:44:57.053808 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 07 10:44:57.053808 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 07 10:44:57.053808 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.creator() Aug 07 10:44:57.053808 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 07 10:44:57.053808 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application created_value = creator( Aug 07 10:44:57.053808 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 07 10:44:57.054893 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 07 10:44:57.054893 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 07 10:44:57.054893 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application Aug 07 10:44:57.054893 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 611/1242] 10.222.0.25 () {56 vars in 1108 bytes} [Sun Aug 7 10:44:56 2022] GET /identity/v3/domains/Default => generated 87 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 07 10:44:57.060695 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37bce572-e3a3-4283-bbc4-1aba3fe038a9 None admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 10:44:57.174358 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=100 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 10:44:57.181281 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37bce572-e3a3-4283-bbc4-1aba3fe038a9 None admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 10:44:57.218598 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae6e721a-71d0-478e-ac47-850ebff3dfda None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 10:44:57.219239 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae6e721a-71d0-478e-ac47-850ebff3dfda None admin] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:44:57.219519 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae6e721a-71d0-478e-ac47-850ebff3dfda None admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:44:57.219823 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae6e721a-71d0-478e-ac47-850ebff3dfda None admin] PATH_INFO: `/v3/domains` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:44:57.220467 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae6e721a-71d0-478e-ac47-850ebff3dfda None admin] RBAC: Adding query filter params (name=Default) {{(pid=80144) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 10:44:57.220779 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae6e721a-71d0-478e-ac47-850ebff3dfda None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 10:44:57.222124 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae6e721a-71d0-478e-ac47-850ebff3dfda None admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 10:44:57.235579 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 632/1243] 10.222.0.25 () {56 vars in 1109 bytes} [Sun Aug 7 10:44:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 10:44:57.241697 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81d2141f-999d-4854-bc6b-d9f148ab69aa None admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 10:44:57.331740 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=100 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 10:44:57.357911 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81d2141f-999d-4854-bc6b-d9f148ab69aa None admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 10:44:57.393065 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17d75d55-af63-4df4-bdf5-50b514b4c7dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 10:44:57.393752 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17d75d55-af63-4df4-bdf5-50b514b4c7dd None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:44:57.394071 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17d75d55-af63-4df4-bdf5-50b514b4c7dd None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:44:57.394408 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17d75d55-af63-4df4-bdf5-50b514b4c7dd None admin] PATH_INFO: `/v3/projects/service` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:44:57.402451 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17d75d55-af63-4df4-bdf5-50b514b4c7dd None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 10:44:57.403955 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17d75d55-af63-4df4-bdf5-50b514b4c7dd None admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 10:44:57.415208 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application [None req-17d75d55-af63-4df4-bdf5-50b514b4c7dd None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 07 10:44:57.415208 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 07 10:44:57.415208 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 07 10:44:57.415208 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 07 10:44:57.415208 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 07 10:44:57.415208 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 07 10:44:57.415208 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 07 10:44:57.415208 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 07 10:44:57.415208 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 07 10:44:57.415208 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 07 10:44:57.415208 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 07 10:44:57.415208 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 07 10:44:57.415208 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 07 10:44:57.415208 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 07 10:44:57.415208 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 07 10:44:57.416339 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 07 10:44:57.416339 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 07 10:44:57.416339 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 07 10:44:57.416339 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 07 10:44:57.416339 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 07 10:44:57.416339 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 07 10:44:57.416339 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.get_or_create( Aug 07 10:44:57.416339 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 07 10:44:57.416339 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application with Lock( Aug 07 10:44:57.416339 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 07 10:44:57.416339 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self._enter() Aug 07 10:44:57.416339 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 07 10:44:57.416339 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 07 10:44:57.416339 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 07 10:44:57.416339 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.creator() Aug 07 10:44:57.416339 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 07 10:44:57.416339 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application created_value = creator( Aug 07 10:44:57.420041 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 07 10:44:57.420041 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 07 10:44:57.420041 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 07 10:44:57.420041 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 07 10:44:57.420041 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 07 10:44:57.420041 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 07 10:44:57.420041 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 07 10:44:57.420041 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application Aug 07 10:44:57.420041 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 612/1244] 10.222.0.25 () {56 vars in 1111 bytes} [Sun Aug 7 10:44:57 2022] GET /identity/v3/projects/service => generated 88 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 07 10:44:57.422177 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae6e721a-71d0-478e-ac47-850ebff3dfda None admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 10:44:57.530014 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae6e721a-71d0-478e-ac47-850ebff3dfda None admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 10:44:57.566878 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e01ccb56-34d1-43da-9ea7-7e31c4cedcaa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 10:44:57.568075 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e01ccb56-34d1-43da-9ea7-7e31c4cedcaa None admin] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:44:57.568574 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e01ccb56-34d1-43da-9ea7-7e31c4cedcaa None admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:44:57.568851 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e01ccb56-34d1-43da-9ea7-7e31c4cedcaa None admin] PATH_INFO: `/v3/projects` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:44:57.569598 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e01ccb56-34d1-43da-9ea7-7e31c4cedcaa None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80144) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 10:44:57.569945 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e01ccb56-34d1-43da-9ea7-7e31c4cedcaa None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 10:44:57.571610 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e01ccb56-34d1-43da-9ea7-7e31c4cedcaa None admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 10:44:57.585224 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 633/1245] 10.222.0.25 () {56 vars in 1148 bytes} [Sun Aug 7 10:44:57 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 10:44:57.591597 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17d75d55-af63-4df4-bdf5-50b514b4c7dd None admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 10:44:57.704340 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17d75d55-af63-4df4-bdf5-50b514b4c7dd None admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 10:44:57.742608 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e89ab95c-8e7a-4a29-a378-f2497cd84105 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 10:44:57.743245 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e89ab95c-8e7a-4a29-a378-f2497cd84105 None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:44:57.743514 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e89ab95c-8e7a-4a29-a378-f2497cd84105 None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:44:57.744348 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e89ab95c-8e7a-4a29-a378-f2497cd84105 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:44:57.745073 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e89ab95c-8e7a-4a29-a378-f2497cd84105 None admin] RBAC: Adding query filter params (role.id=fbf375e9760340c9aa9529333eeb25f9, scope.project.id=6de1f3cac02c4bcdb05dd2337176033d, user.id=6ee2e4c08af04c219ed395d7ca7085a6) {{(pid=80143) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 10:44:57.745383 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e89ab95c-8e7a-4a29-a378-f2497cd84105 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 10:44:57.746872 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e89ab95c-8e7a-4a29-a378-f2497cd84105 None admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 10:44:57.753191 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 613/1246] 10.222.0.25 () {56 vars in 1374 bytes} [Sun Aug 7 10:44:57 2022] GET /identity/v3/role_assignments?user.id=6ee2e4c08af04c219ed395d7ca7085a6&scope.project.id=6de1f3cac02c4bcdb05dd2337176033d&role.id=fbf375e9760340c9aa9529333eeb25f9 => generated 608 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 10:45:00.307289 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38249622-ec34-4463-ac7b-ae8df4559e2d None None] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:45:00.307856 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38249622-ec34-4463-ac7b-ae8df4559e2d None None] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:45:00.307856 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38249622-ec34-4463-ac7b-ae8df4559e2d None None] PATH_INFO: `/` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:45:00.309346 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 634/1247] 10.222.0.25 () {54 vars in 927 bytes} [Sun Aug 7 10:45:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 07 10:45:00.315403 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0baa53a-061b-4478-b024-e24db9f3cf73 None None] REQUEST_METHOD: `POST` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:45:00.315804 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0baa53a-061b-4478-b024-e24db9f3cf73 None None] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:45:00.315948 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0baa53a-061b-4478-b024-e24db9f3cf73 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:45:00.322940 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=100 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 10:45:00.339646 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.auth.core [None req-a0baa53a-061b-4478-b024-e24db9f3cf73 None None] MFA Rules not processed for user `bac4e6b76fe64217ac8fa44e71cffe6a`. Rule list: `[]` (Enabled: `True`). {{(pid=80143) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 10:45:00.340826 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-a0baa53a-061b-4478-b024-e24db9f3cf73 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 10:45:00.422397 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 614/1248] 10.222.0.25 () {58 vars in 1024 bytes} [Sun Aug 7 10:45:00 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 07 10:45:00.450200 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38249622-ec34-4463-ac7b-ae8df4559e2d None None] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 10:45:00.470124 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=100 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 10:45:00.569913 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38249622-ec34-4463-ac7b-ae8df4559e2d None None] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 10:45:00.609185 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a664cca-cb32-488c-bc0c-3e2cfc9ee548 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 10:45:00.609870 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a664cca-cb32-488c-bc0c-3e2cfc9ee548 None admin] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:45:00.610164 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a664cca-cb32-488c-bc0c-3e2cfc9ee548 None admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:45:00.610442 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a664cca-cb32-488c-bc0c-3e2cfc9ee548 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:45:00.624074 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a664cca-cb32-488c-bc0c-3e2cfc9ee548 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 07 10:45:00.629042 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application [None req-8a664cca-cb32-488c-bc0c-3e2cfc9ee548 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 07 10:45:00.629042 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 07 10:45:00.629042 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 07 10:45:00.629042 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 07 10:45:00.629042 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 07 10:45:00.629042 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 07 10:45:00.629042 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 07 10:45:00.629042 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 07 10:45:00.629042 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 07 10:45:00.629042 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 07 10:45:00.629042 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 07 10:45:00.629042 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 07 10:45:00.629042 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 07 10:45:00.629042 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 07 10:45:00.629042 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 07 10:45:00.629042 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 07 10:45:00.630483 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 07 10:45:00.630483 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 07 10:45:00.630483 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 07 10:45:00.630483 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 07 10:45:00.630483 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 07 10:45:00.630483 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 07 10:45:00.630483 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 07 10:45:00.630483 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 07 10:45:00.630483 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 07 10:45:00.630483 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self.get_or_create( Aug 07 10:45:00.630483 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 07 10:45:00.630483 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application with Lock( Aug 07 10:45:00.630483 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 07 10:45:00.630483 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self._enter() Aug 07 10:45:00.630483 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 07 10:45:00.630483 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 07 10:45:00.631579 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 07 10:45:00.631579 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self.creator() Aug 07 10:45:00.631579 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 07 10:45:00.631579 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application created_value = creator( Aug 07 10:45:00.631579 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 07 10:45:00.631579 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 07 10:45:00.631579 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 07 10:45:00.631579 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 07 10:45:00.631579 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 07 10:45:00.631579 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 07 10:45:00.631579 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 07 10:45:00.631579 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application Aug 07 10:45:00.631579 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 635/1249] 10.222.0.25 () {56 vars in 1096 bytes} [Sun Aug 7 10:45:00 2022] GET /identity/v3/roles/admin => generated 83 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 07 10:45:00.637243 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0baa53a-061b-4478-b024-e24db9f3cf73 None None] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 10:45:00.754085 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0baa53a-061b-4478-b024-e24db9f3cf73 None None] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 10:45:00.787235 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=100 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 10:45:00.793264 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deb4c05d-c08f-45fc-b1ac-454290ebc547 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 10:45:00.793905 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deb4c05d-c08f-45fc-b1ac-454290ebc547 None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:45:00.794244 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deb4c05d-c08f-45fc-b1ac-454290ebc547 None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:45:00.794523 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deb4c05d-c08f-45fc-b1ac-454290ebc547 None admin] PATH_INFO: `/v3/roles` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:45:00.795296 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-deb4c05d-c08f-45fc-b1ac-454290ebc547 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80143) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 10:45:00.795599 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-deb4c05d-c08f-45fc-b1ac-454290ebc547 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 10:45:00.796964 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-deb4c05d-c08f-45fc-b1ac-454290ebc547 None admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 10:45:00.806249 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 615/1250] 10.222.0.25 () {56 vars in 1099 bytes} [Sun Aug 7 10:45:00 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 07 10:45:00.812413 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a664cca-cb32-488c-bc0c-3e2cfc9ee548 None admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 10:45:00.902886 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=100 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 10:45:00.927807 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a664cca-cb32-488c-bc0c-3e2cfc9ee548 None admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 10:45:00.966800 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f483a87b-59c1-48e7-83cf-b852bf14f8cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 10:45:00.967480 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f483a87b-59c1-48e7-83cf-b852bf14f8cf None admin] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:45:00.967765 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f483a87b-59c1-48e7-83cf-b852bf14f8cf None admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:45:00.968042 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f483a87b-59c1-48e7-83cf-b852bf14f8cf None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:45:00.980291 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f483a87b-59c1-48e7-83cf-b852bf14f8cf None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 10:45:00.981831 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f483a87b-59c1-48e7-83cf-b852bf14f8cf None admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 10:45:00.997103 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application [None req-f483a87b-59c1-48e7-83cf-b852bf14f8cf None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 07 10:45:00.997103 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 07 10:45:00.997103 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 07 10:45:00.997103 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 07 10:45:00.997103 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 07 10:45:00.997103 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 07 10:45:00.997103 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 07 10:45:00.997103 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 07 10:45:00.997103 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 07 10:45:00.997103 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 07 10:45:00.997103 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 07 10:45:00.997103 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 07 10:45:00.997103 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 07 10:45:00.997103 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 07 10:45:00.997103 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 07 10:45:00.997103 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 07 10:45:00.998297 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 07 10:45:00.998297 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 07 10:45:00.998297 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 07 10:45:00.998297 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 07 10:45:00.998297 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 07 10:45:00.998297 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self.get_or_create( Aug 07 10:45:00.998297 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 07 10:45:00.998297 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application with Lock( Aug 07 10:45:00.998297 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 07 10:45:00.998297 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self._enter() Aug 07 10:45:00.998297 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 07 10:45:00.998297 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 07 10:45:00.998297 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 07 10:45:00.998297 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self.creator() Aug 07 10:45:00.998297 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 07 10:45:00.998297 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application created_value = creator( Aug 07 10:45:00.998297 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 07 10:45:00.999506 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 07 10:45:00.999506 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 07 10:45:00.999506 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application Aug 07 10:45:00.999506 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 636/1251] 10.222.0.25 () {56 vars in 1108 bytes} [Sun Aug 7 10:45:00 2022] GET /identity/v3/domains/Default => generated 87 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 07 10:45:01.004671 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deb4c05d-c08f-45fc-b1ac-454290ebc547 None admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 10:45:01.128738 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deb4c05d-c08f-45fc-b1ac-454290ebc547 None admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 10:45:01.170771 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02454ac2-aee2-4acd-8926-03cbdccc94f6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 10:45:01.171387 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02454ac2-aee2-4acd-8926-03cbdccc94f6 None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:45:01.171732 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02454ac2-aee2-4acd-8926-03cbdccc94f6 None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:45:01.172018 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02454ac2-aee2-4acd-8926-03cbdccc94f6 None admin] PATH_INFO: `/v3/domains` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:45:01.172779 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02454ac2-aee2-4acd-8926-03cbdccc94f6 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80143) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 10:45:01.173089 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02454ac2-aee2-4acd-8926-03cbdccc94f6 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 10:45:01.174677 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02454ac2-aee2-4acd-8926-03cbdccc94f6 None admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 10:45:01.192432 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 616/1252] 10.222.0.25 () {56 vars in 1109 bytes} [Sun Aug 7 10:45:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 10:45:01.193980 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f483a87b-59c1-48e7-83cf-b852bf14f8cf None admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 10:45:01.316768 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f483a87b-59c1-48e7-83cf-b852bf14f8cf None admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 10:45:01.348333 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=100 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 10:45:01.359302 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cedfcc1e-34da-4a76-8e43-1a322632ed2f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 10:45:01.359950 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cedfcc1e-34da-4a76-8e43-1a322632ed2f None admin] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:45:01.360358 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cedfcc1e-34da-4a76-8e43-1a322632ed2f None admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:45:01.360687 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cedfcc1e-34da-4a76-8e43-1a322632ed2f None admin] PATH_INFO: `/v3/users/placement` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:45:01.370536 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cedfcc1e-34da-4a76-8e43-1a322632ed2f None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 10:45:01.372208 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cedfcc1e-34da-4a76-8e43-1a322632ed2f None admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 10:45:01.384644 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application [None req-cedfcc1e-34da-4a76-8e43-1a322632ed2f None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Aug 07 10:45:01.384644 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 07 10:45:01.384644 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 07 10:45:01.384644 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 07 10:45:01.384644 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 07 10:45:01.384644 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 07 10:45:01.384644 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 07 10:45:01.384644 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 07 10:45:01.384644 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 07 10:45:01.384644 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 07 10:45:01.384644 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 07 10:45:01.384644 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 07 10:45:01.384644 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 07 10:45:01.384644 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 07 10:45:01.384644 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 07 10:45:01.384644 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 07 10:45:01.385749 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 07 10:45:01.385749 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 07 10:45:01.385749 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 07 10:45:01.385749 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 07 10:45:01.385749 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 07 10:45:01.385749 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 07 10:45:01.385749 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 07 10:45:01.385749 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 07 10:45:01.385749 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 07 10:45:01.385749 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self.get_or_create( Aug 07 10:45:01.385749 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 07 10:45:01.385749 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application with Lock( Aug 07 10:45:01.385749 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 07 10:45:01.385749 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self._enter() Aug 07 10:45:01.385749 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 07 10:45:01.385749 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 07 10:45:01.385749 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 07 10:45:01.385749 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self.creator() Aug 07 10:45:01.386698 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 07 10:45:01.386698 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application created_value = creator( Aug 07 10:45:01.386698 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 07 10:45:01.386698 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 07 10:45:01.386698 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 07 10:45:01.386698 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 07 10:45:01.386698 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 07 10:45:01.386698 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 07 10:45:01.386698 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Aug 07 10:45:01.386698 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application Aug 07 10:45:01.386698 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 637/1253] 10.222.0.25 () {56 vars in 1108 bytes} [Sun Aug 7 10:45:01 2022] GET /identity/v3/users/placement => generated 87 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 07 10:45:01.392218 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02454ac2-aee2-4acd-8926-03cbdccc94f6 None admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 10:45:01.459789 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=100 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 10:45:01.511762 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02454ac2-aee2-4acd-8926-03cbdccc94f6 None admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 10:45:01.554282 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91a86cf0-f93e-4ccc-9a6b-a3d51b746a9b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 10:45:01.554912 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91a86cf0-f93e-4ccc-9a6b-a3d51b746a9b None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:45:01.555285 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91a86cf0-f93e-4ccc-9a6b-a3d51b746a9b None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:45:01.555577 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91a86cf0-f93e-4ccc-9a6b-a3d51b746a9b None admin] PATH_INFO: `/v3/users` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:45:01.556349 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91a86cf0-f93e-4ccc-9a6b-a3d51b746a9b None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80143) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 10:45:01.556626 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91a86cf0-f93e-4ccc-9a6b-a3d51b746a9b None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 10:45:01.558264 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91a86cf0-f93e-4ccc-9a6b-a3d51b746a9b None admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 10:45:01.577578 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 617/1254] 10.222.0.25 () {56 vars in 1143 bytes} [Sun Aug 7 10:45:01 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 10:45:01.584403 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cedfcc1e-34da-4a76-8e43-1a322632ed2f None admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 10:45:01.697266 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cedfcc1e-34da-4a76-8e43-1a322632ed2f None admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 10:45:01.740685 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a724967c-4a9c-4198-8beb-4996a788f52d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 10:45:01.741311 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a724967c-4a9c-4198-8beb-4996a788f52d None admin] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:45:01.741517 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a724967c-4a9c-4198-8beb-4996a788f52d None admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:45:01.741844 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a724967c-4a9c-4198-8beb-4996a788f52d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:45:01.801976 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=100 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 10:45:01.815526 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a724967c-4a9c-4198-8beb-4996a788f52d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 10:45:01.817065 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a724967c-4a9c-4198-8beb-4996a788f52d None admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 10:45:01.923459 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 638/1255] 10.222.0.25 () {58 vars in 1282 bytes} [Sun Aug 7 10:45:01 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 340 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 07 10:45:01.929586 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91a86cf0-f93e-4ccc-9a6b-a3d51b746a9b None admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 10:45:02.039637 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91a86cf0-f93e-4ccc-9a6b-a3d51b746a9b None admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 10:45:02.054817 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=100 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 10:45:02.078467 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6168d35-b120-481a-baf6-5475d9324f00 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 10:45:02.079137 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6168d35-b120-481a-baf6-5475d9324f00 None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:45:02.079420 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6168d35-b120-481a-baf6-5475d9324f00 None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:45:02.079696 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6168d35-b120-481a-baf6-5475d9324f00 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:45:02.092217 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6168d35-b120-481a-baf6-5475d9324f00 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 10:45:02.094070 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6168d35-b120-481a-baf6-5475d9324f00 None admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 10:45:02.110656 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application [None req-e6168d35-b120-481a-baf6-5475d9324f00 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 07 10:45:02.110656 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 07 10:45:02.110656 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 07 10:45:02.110656 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 07 10:45:02.110656 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 07 10:45:02.110656 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 07 10:45:02.110656 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 07 10:45:02.110656 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 07 10:45:02.110656 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 07 10:45:02.110656 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 07 10:45:02.110656 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 07 10:45:02.110656 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 07 10:45:02.110656 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 07 10:45:02.110656 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 07 10:45:02.110656 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 07 10:45:02.110656 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 07 10:45:02.112165 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 07 10:45:02.112165 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 07 10:45:02.112165 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 07 10:45:02.112165 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 07 10:45:02.112165 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 07 10:45:02.112165 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.get_or_create( Aug 07 10:45:02.112165 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 07 10:45:02.112165 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application with Lock( Aug 07 10:45:02.112165 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 07 10:45:02.112165 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self._enter() Aug 07 10:45:02.112165 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 07 10:45:02.112165 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 07 10:45:02.112165 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 07 10:45:02.112165 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.creator() Aug 07 10:45:02.112165 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 07 10:45:02.112165 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application created_value = creator( Aug 07 10:45:02.112165 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 07 10:45:02.113511 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 07 10:45:02.113511 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 07 10:45:02.113511 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application Aug 07 10:45:02.113511 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 618/1256] 10.222.0.25 () {56 vars in 1108 bytes} [Sun Aug 7 10:45:01 2022] GET /identity/v3/domains/Default => generated 87 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 07 10:45:02.118149 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a724967c-4a9c-4198-8beb-4996a788f52d None admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 10:45:02.227703 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=100 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 10:45:02.245968 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a724967c-4a9c-4198-8beb-4996a788f52d None admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 10:45:02.284592 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ebc8771-ea4a-4cf0-91af-0f9d77ab2bf6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 10:45:02.285235 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ebc8771-ea4a-4cf0-91af-0f9d77ab2bf6 None admin] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:45:02.285549 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ebc8771-ea4a-4cf0-91af-0f9d77ab2bf6 None admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:45:02.285882 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ebc8771-ea4a-4cf0-91af-0f9d77ab2bf6 None admin] PATH_INFO: `/v3/domains` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:45:02.286605 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ebc8771-ea4a-4cf0-91af-0f9d77ab2bf6 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80144) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 10:45:02.287115 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ebc8771-ea4a-4cf0-91af-0f9d77ab2bf6 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 10:45:02.288629 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ebc8771-ea4a-4cf0-91af-0f9d77ab2bf6 None admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 10:45:02.301172 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 639/1257] 10.222.0.25 () {56 vars in 1109 bytes} [Sun Aug 7 10:45:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 10:45:02.308517 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6168d35-b120-481a-baf6-5475d9324f00 None admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 10:45:02.424328 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6168d35-b120-481a-baf6-5475d9324f00 None admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 10:45:02.463960 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc5e1944-15cc-437a-865b-22b9b5908dec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 10:45:02.464602 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc5e1944-15cc-437a-865b-22b9b5908dec None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:45:02.464999 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc5e1944-15cc-437a-865b-22b9b5908dec None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:45:02.465350 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc5e1944-15cc-437a-865b-22b9b5908dec None admin] PATH_INFO: `/v3/projects/service` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:45:02.472017 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc5e1944-15cc-437a-865b-22b9b5908dec None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 10:45:02.473454 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc5e1944-15cc-437a-865b-22b9b5908dec None admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 10:45:02.486972 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application [None req-fc5e1944-15cc-437a-865b-22b9b5908dec None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 07 10:45:02.486972 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 07 10:45:02.486972 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 07 10:45:02.486972 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 07 10:45:02.486972 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 07 10:45:02.486972 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 07 10:45:02.486972 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 07 10:45:02.486972 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 07 10:45:02.486972 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 07 10:45:02.486972 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 07 10:45:02.486972 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 07 10:45:02.486972 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 07 10:45:02.486972 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 07 10:45:02.486972 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 07 10:45:02.486972 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 07 10:45:02.488105 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 07 10:45:02.488105 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 07 10:45:02.488105 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 07 10:45:02.488105 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 07 10:45:02.488105 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 07 10:45:02.488105 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 07 10:45:02.488105 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.get_or_create( Aug 07 10:45:02.488105 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 07 10:45:02.488105 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application with Lock( Aug 07 10:45:02.488105 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 07 10:45:02.488105 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self._enter() Aug 07 10:45:02.488105 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 07 10:45:02.488105 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 07 10:45:02.488105 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 07 10:45:02.488105 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.creator() Aug 07 10:45:02.488105 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 07 10:45:02.488105 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application created_value = creator( Aug 07 10:45:02.489880 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 07 10:45:02.489880 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 07 10:45:02.489880 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 07 10:45:02.489880 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 07 10:45:02.489880 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 07 10:45:02.489880 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 07 10:45:02.489880 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 07 10:45:02.489880 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application Aug 07 10:45:02.489880 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 619/1258] 10.222.0.25 () {56 vars in 1111 bytes} [Sun Aug 7 10:45:02 2022] GET /identity/v3/projects/service => generated 88 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 07 10:45:02.495111 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ebc8771-ea4a-4cf0-91af-0f9d77ab2bf6 None admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 10:45:02.597821 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ebc8771-ea4a-4cf0-91af-0f9d77ab2bf6 None admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 10:45:02.637878 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ec22761-3ac3-4a49-b40c-a6672f0586f8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 10:45:02.638681 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ec22761-3ac3-4a49-b40c-a6672f0586f8 None admin] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:45:02.638995 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ec22761-3ac3-4a49-b40c-a6672f0586f8 None admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:45:02.639272 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ec22761-3ac3-4a49-b40c-a6672f0586f8 None admin] PATH_INFO: `/v3/projects` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:45:02.640167 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ec22761-3ac3-4a49-b40c-a6672f0586f8 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80144) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 10:45:02.640580 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ec22761-3ac3-4a49-b40c-a6672f0586f8 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 10:45:02.642043 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ec22761-3ac3-4a49-b40c-a6672f0586f8 None admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 10:45:02.656846 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 640/1259] 10.222.0.25 () {56 vars in 1148 bytes} [Sun Aug 7 10:45:02 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 10:45:02.663329 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc5e1944-15cc-437a-865b-22b9b5908dec None admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 10:45:02.693735 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=100 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 10:45:02.781500 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc5e1944-15cc-437a-865b-22b9b5908dec None admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 10:45:02.818237 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c87411f7-9443-4ba2-acaa-3482f2b62002 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 10:45:02.818881 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c87411f7-9443-4ba2-acaa-3482f2b62002 None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:45:02.819198 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c87411f7-9443-4ba2-acaa-3482f2b62002 None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:45:02.819473 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c87411f7-9443-4ba2-acaa-3482f2b62002 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:45:02.820203 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c87411f7-9443-4ba2-acaa-3482f2b62002 None admin] RBAC: Adding query filter params (role.id=81b1f1bcee2a473088fbbd327c1c5b23, scope.project.id=6de1f3cac02c4bcdb05dd2337176033d, user.id=6ee2e4c08af04c219ed395d7ca7085a6) {{(pid=80143) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 10:45:02.820513 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c87411f7-9443-4ba2-acaa-3482f2b62002 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 10:45:02.821937 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c87411f7-9443-4ba2-acaa-3482f2b62002 None admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 10:45:02.827462 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 620/1260] 10.222.0.25 () {56 vars in 1374 bytes} [Sun Aug 7 10:45:02 2022] GET /identity/v3/role_assignments?user.id=6ee2e4c08af04c219ed395d7ca7085a6&scope.project.id=6de1f3cac02c4bcdb05dd2337176033d&role.id=81b1f1bcee2a473088fbbd327c1c5b23 => generated 259 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 10:45:04.850512 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baef9cb2-7607-4eed-b633-15003d5bbb0c None None] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:45:04.851058 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baef9cb2-7607-4eed-b633-15003d5bbb0c None None] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:45:04.851172 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baef9cb2-7607-4eed-b633-15003d5bbb0c None None] PATH_INFO: `/` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:45:04.852429 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 641/1261] 10.222.0.25 () {54 vars in 927 bytes} [Sun Aug 7 10:45:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 07 10:45:04.857536 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b53f5865-243f-4e25-859f-cea686ff5c07 None None] REQUEST_METHOD: `POST` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:45:04.858001 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b53f5865-243f-4e25-859f-cea686ff5c07 None None] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:45:04.858103 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b53f5865-243f-4e25-859f-cea686ff5c07 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:45:04.876975 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.auth.core [None req-b53f5865-243f-4e25-859f-cea686ff5c07 None None] MFA Rules not processed for user `bac4e6b76fe64217ac8fa44e71cffe6a`. Rule list: `[]` (Enabled: `True`). {{(pid=80143) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 10:45:04.878154 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-b53f5865-243f-4e25-859f-cea686ff5c07 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 10:45:04.942293 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 621/1262] 10.222.0.25 () {58 vars in 1024 bytes} [Sun Aug 7 10:45:04 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 07 10:45:04.964653 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baef9cb2-7607-4eed-b633-15003d5bbb0c None None] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 10:45:04.975961 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=100 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 10:45:05.064042 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baef9cb2-7607-4eed-b633-15003d5bbb0c None None] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 10:45:05.096595 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9afa5ecb-1b46-4a02-aa0e-1aa71b92e081 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 10:45:05.097221 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9afa5ecb-1b46-4a02-aa0e-1aa71b92e081 None admin] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:45:05.097621 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9afa5ecb-1b46-4a02-aa0e-1aa71b92e081 None admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:45:05.097836 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9afa5ecb-1b46-4a02-aa0e-1aa71b92e081 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:45:05.109176 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9afa5ecb-1b46-4a02-aa0e-1aa71b92e081 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 07 10:45:05.114552 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application [None req-9afa5ecb-1b46-4a02-aa0e-1aa71b92e081 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 07 10:45:05.114552 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 07 10:45:05.114552 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 07 10:45:05.114552 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 07 10:45:05.114552 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 07 10:45:05.114552 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 07 10:45:05.114552 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 07 10:45:05.114552 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 07 10:45:05.114552 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 07 10:45:05.114552 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 07 10:45:05.114552 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 07 10:45:05.114552 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 07 10:45:05.114552 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 07 10:45:05.114552 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 07 10:45:05.114552 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 07 10:45:05.114552 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 07 10:45:05.115621 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 07 10:45:05.115621 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 07 10:45:05.115621 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 07 10:45:05.115621 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 07 10:45:05.115621 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 07 10:45:05.115621 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 07 10:45:05.115621 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 07 10:45:05.115621 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 07 10:45:05.115621 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 07 10:45:05.115621 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self.get_or_create( Aug 07 10:45:05.115621 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 07 10:45:05.115621 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application with Lock( Aug 07 10:45:05.115621 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 07 10:45:05.115621 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self._enter() Aug 07 10:45:05.115621 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 07 10:45:05.115621 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 07 10:45:05.116617 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 07 10:45:05.116617 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self.creator() Aug 07 10:45:05.116617 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 07 10:45:05.116617 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application created_value = creator( Aug 07 10:45:05.116617 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 07 10:45:05.116617 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 07 10:45:05.116617 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 07 10:45:05.116617 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 07 10:45:05.116617 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 07 10:45:05.116617 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 07 10:45:05.116617 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 07 10:45:05.116617 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application Aug 07 10:45:05.121397 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 642/1263] 10.222.0.25 () {56 vars in 1096 bytes} [Sun Aug 7 10:45:04 2022] GET /identity/v3/roles/admin => generated 83 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 07 10:45:05.122372 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b53f5865-243f-4e25-859f-cea686ff5c07 None None] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 10:45:05.141826 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=100 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 10:45:05.219742 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b53f5865-243f-4e25-859f-cea686ff5c07 None None] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 10:45:05.252421 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-183c1405-5813-4cd2-966d-128cb90d1fad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 10:45:05.253019 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-183c1405-5813-4cd2-966d-128cb90d1fad None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:45:05.253300 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-183c1405-5813-4cd2-966d-128cb90d1fad None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:45:05.253570 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-183c1405-5813-4cd2-966d-128cb90d1fad None admin] PATH_INFO: `/v3/roles` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:45:05.254226 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-183c1405-5813-4cd2-966d-128cb90d1fad None admin] RBAC: Adding query filter params (name=admin) {{(pid=80143) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 10:45:05.254528 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-183c1405-5813-4cd2-966d-128cb90d1fad None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 10:45:05.255976 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-183c1405-5813-4cd2-966d-128cb90d1fad None admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 10:45:05.263710 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 622/1264] 10.222.0.25 () {56 vars in 1099 bytes} [Sun Aug 7 10:45:05 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 07 10:45:05.269568 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9afa5ecb-1b46-4a02-aa0e-1aa71b92e081 None admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 10:45:05.335424 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=100 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 10:45:05.367919 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9afa5ecb-1b46-4a02-aa0e-1aa71b92e081 None admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 10:45:05.402078 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea3d67d8-5664-407c-89c6-3f47c0caacc0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 10:45:05.402700 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea3d67d8-5664-407c-89c6-3f47c0caacc0 None admin] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:45:05.403010 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea3d67d8-5664-407c-89c6-3f47c0caacc0 None admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:45:05.403288 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea3d67d8-5664-407c-89c6-3f47c0caacc0 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:45:05.413493 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea3d67d8-5664-407c-89c6-3f47c0caacc0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 10:45:05.414942 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea3d67d8-5664-407c-89c6-3f47c0caacc0 None admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 10:45:05.429401 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application [None req-ea3d67d8-5664-407c-89c6-3f47c0caacc0 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 07 10:45:05.429401 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 07 10:45:05.429401 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 07 10:45:05.429401 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 07 10:45:05.429401 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 07 10:45:05.429401 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 07 10:45:05.429401 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 07 10:45:05.429401 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 07 10:45:05.429401 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 07 10:45:05.429401 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 07 10:45:05.429401 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 07 10:45:05.429401 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 07 10:45:05.429401 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 07 10:45:05.429401 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 07 10:45:05.429401 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 07 10:45:05.429401 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 07 10:45:05.434277 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 07 10:45:05.434277 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 07 10:45:05.434277 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 07 10:45:05.434277 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 07 10:45:05.434277 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 07 10:45:05.434277 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self.get_or_create( Aug 07 10:45:05.434277 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 07 10:45:05.434277 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application with Lock( Aug 07 10:45:05.434277 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 07 10:45:05.434277 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self._enter() Aug 07 10:45:05.434277 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 07 10:45:05.434277 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 07 10:45:05.434277 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 07 10:45:05.434277 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self.creator() Aug 07 10:45:05.434277 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 07 10:45:05.434277 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application created_value = creator( Aug 07 10:45:05.434277 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 07 10:45:05.435399 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 07 10:45:05.435399 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 07 10:45:05.435399 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application Aug 07 10:45:05.435399 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 643/1265] 10.222.0.25 () {56 vars in 1108 bytes} [Sun Aug 7 10:45:05 2022] GET /identity/v3/domains/Default => generated 87 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 07 10:45:05.436238 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-183c1405-5813-4cd2-966d-128cb90d1fad None admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 10:45:05.517197 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=100 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 10:45:05.536657 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-183c1405-5813-4cd2-966d-128cb90d1fad None admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 10:45:05.568968 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60378175-e9da-49be-afa2-f806f35aa7d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 10:45:05.569561 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60378175-e9da-49be-afa2-f806f35aa7d4 None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:45:05.569841 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60378175-e9da-49be-afa2-f806f35aa7d4 None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:45:05.570115 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60378175-e9da-49be-afa2-f806f35aa7d4 None admin] PATH_INFO: `/v3/domains` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:45:05.570748 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60378175-e9da-49be-afa2-f806f35aa7d4 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80143) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 10:45:05.571084 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60378175-e9da-49be-afa2-f806f35aa7d4 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 10:45:05.572436 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60378175-e9da-49be-afa2-f806f35aa7d4 None admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 10:45:05.584201 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 623/1266] 10.222.0.25 () {56 vars in 1109 bytes} [Sun Aug 7 10:45:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 10:45:05.589887 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea3d67d8-5664-407c-89c6-3f47c0caacc0 None admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 10:45:05.685803 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea3d67d8-5664-407c-89c6-3f47c0caacc0 None admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 10:45:05.705033 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=100 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 10:45:05.719522 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed6942a9-011f-4324-b4ea-af4b82b26dbf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 10:45:05.720135 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed6942a9-011f-4324-b4ea-af4b82b26dbf None admin] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:45:05.720420 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed6942a9-011f-4324-b4ea-af4b82b26dbf None admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:45:05.720696 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed6942a9-011f-4324-b4ea-af4b82b26dbf None admin] PATH_INFO: `/v3/users/placement` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:45:05.727942 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed6942a9-011f-4324-b4ea-af4b82b26dbf None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 10:45:05.729372 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed6942a9-011f-4324-b4ea-af4b82b26dbf None admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 10:45:05.740473 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application [None req-ed6942a9-011f-4324-b4ea-af4b82b26dbf None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Aug 07 10:45:05.740473 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 07 10:45:05.740473 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 07 10:45:05.740473 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 07 10:45:05.740473 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 07 10:45:05.740473 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 07 10:45:05.740473 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 07 10:45:05.740473 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 07 10:45:05.740473 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 07 10:45:05.740473 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 07 10:45:05.740473 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 07 10:45:05.740473 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 07 10:45:05.740473 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 07 10:45:05.740473 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 07 10:45:05.740473 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 07 10:45:05.740473 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 07 10:45:05.742369 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 07 10:45:05.742369 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 07 10:45:05.742369 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 07 10:45:05.742369 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 07 10:45:05.742369 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 07 10:45:05.742369 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 07 10:45:05.742369 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 07 10:45:05.742369 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 07 10:45:05.742369 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 07 10:45:05.742369 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self.get_or_create( Aug 07 10:45:05.742369 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 07 10:45:05.742369 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application with Lock( Aug 07 10:45:05.742369 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 07 10:45:05.742369 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self._enter() Aug 07 10:45:05.742369 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 07 10:45:05.742369 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 07 10:45:05.742369 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 07 10:45:05.742369 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self.creator() Aug 07 10:45:05.743700 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 07 10:45:05.743700 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application created_value = creator( Aug 07 10:45:05.743700 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 07 10:45:05.743700 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 07 10:45:05.743700 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 07 10:45:05.743700 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 07 10:45:05.743700 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 07 10:45:05.743700 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 07 10:45:05.743700 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Aug 07 10:45:05.743700 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application Aug 07 10:45:05.743700 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 644/1267] 10.222.0.25 () {56 vars in 1108 bytes} [Sun Aug 7 10:45:05 2022] GET /identity/v3/users/placement => generated 87 bytes in 152 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 07 10:45:05.747433 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60378175-e9da-49be-afa2-f806f35aa7d4 None admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 10:45:05.845478 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60378175-e9da-49be-afa2-f806f35aa7d4 None admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 10:45:05.882010 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0a3399b-ac17-4cc6-9ce1-698e5c61a028 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 10:45:05.882845 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0a3399b-ac17-4cc6-9ce1-698e5c61a028 None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:45:05.883384 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0a3399b-ac17-4cc6-9ce1-698e5c61a028 None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:45:05.883886 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0a3399b-ac17-4cc6-9ce1-698e5c61a028 None admin] PATH_INFO: `/v3/users` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:45:05.885211 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0a3399b-ac17-4cc6-9ce1-698e5c61a028 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80143) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 10:45:05.885528 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0a3399b-ac17-4cc6-9ce1-698e5c61a028 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 10:45:05.886969 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0a3399b-ac17-4cc6-9ce1-698e5c61a028 None admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 10:45:05.895416 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=100 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 10:45:05.905660 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 624/1268] 10.222.0.25 () {56 vars in 1143 bytes} [Sun Aug 7 10:45:05 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 10:45:05.911410 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed6942a9-011f-4324-b4ea-af4b82b26dbf None admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 10:45:06.008827 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed6942a9-011f-4324-b4ea-af4b82b26dbf None admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 10:45:06.040234 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ac13be1-c4ae-46cc-90a2-f3dd102eac71 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 10:45:06.040837 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ac13be1-c4ae-46cc-90a2-f3dd102eac71 None admin] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:45:06.041114 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ac13be1-c4ae-46cc-90a2-f3dd102eac71 None admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:45:06.041386 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ac13be1-c4ae-46cc-90a2-f3dd102eac71 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:45:06.051388 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ac13be1-c4ae-46cc-90a2-f3dd102eac71 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 10:45:06.053440 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ac13be1-c4ae-46cc-90a2-f3dd102eac71 None admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 10:45:06.066751 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application [None req-7ac13be1-c4ae-46cc-90a2-f3dd102eac71 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 07 10:45:06.066751 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 07 10:45:06.066751 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 07 10:45:06.066751 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 07 10:45:06.066751 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 07 10:45:06.066751 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 07 10:45:06.066751 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 07 10:45:06.066751 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 07 10:45:06.066751 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 07 10:45:06.066751 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 07 10:45:06.066751 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 07 10:45:06.066751 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 07 10:45:06.066751 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 07 10:45:06.066751 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 07 10:45:06.066751 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 07 10:45:06.066751 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 07 10:45:06.068676 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 07 10:45:06.068676 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 07 10:45:06.068676 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 07 10:45:06.068676 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 07 10:45:06.068676 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 07 10:45:06.068676 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self.get_or_create( Aug 07 10:45:06.068676 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 07 10:45:06.068676 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application with Lock( Aug 07 10:45:06.068676 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 07 10:45:06.068676 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self._enter() Aug 07 10:45:06.068676 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 07 10:45:06.068676 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 07 10:45:06.068676 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 07 10:45:06.068676 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self.creator() Aug 07 10:45:06.068676 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 07 10:45:06.068676 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application created_value = creator( Aug 07 10:45:06.068676 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 07 10:45:06.069576 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 07 10:45:06.069576 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 07 10:45:06.069576 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application Aug 07 10:45:06.069576 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 645/1269] 10.222.0.25 () {56 vars in 1108 bytes} [Sun Aug 7 10:45:05 2022] GET /identity/v3/domains/Default => generated 87 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 07 10:45:06.074264 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0a3399b-ac17-4cc6-9ce1-698e5c61a028 None admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 10:45:06.181371 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0a3399b-ac17-4cc6-9ce1-698e5c61a028 None admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 10:45:06.213268 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-763a6e0a-0e90-403b-afdd-c70065641581 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 10:45:06.213865 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-763a6e0a-0e90-403b-afdd-c70065641581 None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:45:06.214141 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-763a6e0a-0e90-403b-afdd-c70065641581 None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:45:06.214410 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-763a6e0a-0e90-403b-afdd-c70065641581 None admin] PATH_INFO: `/v3/domains` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:45:06.215074 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-763a6e0a-0e90-403b-afdd-c70065641581 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80143) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 10:45:06.215375 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-763a6e0a-0e90-403b-afdd-c70065641581 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 10:45:06.216724 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-763a6e0a-0e90-403b-afdd-c70065641581 None admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 10:45:06.228853 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 625/1270] 10.222.0.25 () {56 vars in 1109 bytes} [Sun Aug 7 10:45:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 10:45:06.234534 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ac13be1-c4ae-46cc-90a2-f3dd102eac71 None admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 10:45:06.265906 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=100 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 10:45:06.332006 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ac13be1-c4ae-46cc-90a2-f3dd102eac71 None admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 10:45:06.365068 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f08189f0-800c-432c-943d-20a2dca8c89a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 10:45:06.365681 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f08189f0-800c-432c-943d-20a2dca8c89a None admin] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:45:06.365965 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f08189f0-800c-432c-943d-20a2dca8c89a None admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:45:06.366241 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f08189f0-800c-432c-943d-20a2dca8c89a None admin] PATH_INFO: `/v3/projects/service` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:45:06.372660 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f08189f0-800c-432c-943d-20a2dca8c89a None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 10:45:06.374075 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f08189f0-800c-432c-943d-20a2dca8c89a None admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 10:45:06.385009 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application [None req-f08189f0-800c-432c-943d-20a2dca8c89a None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 07 10:45:06.385009 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 07 10:45:06.385009 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 07 10:45:06.385009 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 07 10:45:06.385009 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 07 10:45:06.385009 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 07 10:45:06.385009 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 07 10:45:06.385009 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 07 10:45:06.385009 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 07 10:45:06.385009 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 07 10:45:06.385009 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 07 10:45:06.385009 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 07 10:45:06.385009 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 07 10:45:06.385009 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 07 10:45:06.385009 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 07 10:45:06.386054 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 07 10:45:06.386054 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 07 10:45:06.386054 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 07 10:45:06.386054 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 07 10:45:06.386054 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 07 10:45:06.386054 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 07 10:45:06.386054 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self.get_or_create( Aug 07 10:45:06.386054 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 07 10:45:06.386054 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application with Lock( Aug 07 10:45:06.386054 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 07 10:45:06.386054 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self._enter() Aug 07 10:45:06.386054 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 07 10:45:06.386054 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 07 10:45:06.386054 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 07 10:45:06.386054 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self.creator() Aug 07 10:45:06.386054 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 07 10:45:06.386054 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application created_value = creator( Aug 07 10:45:06.387176 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 07 10:45:06.387176 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 07 10:45:06.387176 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 07 10:45:06.387176 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 07 10:45:06.387176 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 07 10:45:06.387176 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 07 10:45:06.387176 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 07 10:45:06.387176 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application Aug 07 10:45:06.387176 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 646/1271] 10.222.0.25 () {56 vars in 1111 bytes} [Sun Aug 7 10:45:06 2022] GET /identity/v3/projects/service => generated 88 bytes in 152 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 07 10:45:06.391857 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-763a6e0a-0e90-403b-afdd-c70065641581 None admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 10:45:06.448215 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=100 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 10:45:06.490394 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-763a6e0a-0e90-403b-afdd-c70065641581 None admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 10:45:06.523239 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69147747-6989-4456-851f-bcbc0b15e5b8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 10:45:06.523793 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69147747-6989-4456-851f-bcbc0b15e5b8 None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:45:06.524075 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69147747-6989-4456-851f-bcbc0b15e5b8 None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:45:06.524349 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69147747-6989-4456-851f-bcbc0b15e5b8 None admin] PATH_INFO: `/v3/projects` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:45:06.525031 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69147747-6989-4456-851f-bcbc0b15e5b8 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80143) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 10:45:06.525336 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69147747-6989-4456-851f-bcbc0b15e5b8 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 10:45:06.526757 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69147747-6989-4456-851f-bcbc0b15e5b8 None admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 10:45:06.538706 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 626/1272] 10.222.0.25 () {56 vars in 1148 bytes} [Sun Aug 7 10:45:06 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 10:45:06.544604 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f08189f0-800c-432c-943d-20a2dca8c89a None admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 10:45:06.632468 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=100 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 10:45:06.641661 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f08189f0-800c-432c-943d-20a2dca8c89a None admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 10:45:06.674444 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf39d833-3c28-4b09-97aa-87364744cd09 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 10:45:06.675116 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf39d833-3c28-4b09-97aa-87364744cd09 None admin] REQUEST_METHOD: `PUT` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:45:06.675396 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf39d833-3c28-4b09-97aa-87364744cd09 None admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:45:06.675667 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf39d833-3c28-4b09-97aa-87364744cd09 None admin] PATH_INFO: `/v3/projects/6de1f3cac02c4bcdb05dd2337176033d/users/6ee2e4c08af04c219ed395d7ca7085a6/roles/81b1f1bcee2a473088fbbd327c1c5b23` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:45:06.677740 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf39d833-3c28-4b09-97aa-87364744cd09 None admin] RBAC: Authorizing `identity:create_grant(project_id=6de1f3cac02c4bcdb05dd2337176033d, user_id=6ee2e4c08af04c219ed395d7ca7085a6, role_id=81b1f1bcee2a473088fbbd327c1c5b23)` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 10:45:06.679162 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf39d833-3c28-4b09-97aa-87364744cd09 None admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 10:45:06.690825 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 647/1273] 10.222.0.25 () {58 vars in 1439 bytes} [Sun Aug 7 10:45:06 2022] PUT /identity/v3/projects/6de1f3cac02c4bcdb05dd2337176033d/users/6ee2e4c08af04c219ed395d7ca7085a6/roles/81b1f1bcee2a473088fbbd327c1c5b23 => generated 0 bytes in 147 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 07 10:45:08.721759 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ea69bc1-7c67-4693-863e-ab12458e5ce4 None None] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:45:08.722265 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ea69bc1-7c67-4693-863e-ab12458e5ce4 None None] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:45:08.722265 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ea69bc1-7c67-4693-863e-ab12458e5ce4 None None] PATH_INFO: `/` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:45:08.724167 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 627/1274] 10.222.0.25 () {54 vars in 927 bytes} [Sun Aug 7 10:45:08 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 07 10:45:08.729339 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1b982b9-3132-448d-8553-e69df37be3bb None None] REQUEST_METHOD: `POST` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:45:08.729715 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1b982b9-3132-448d-8553-e69df37be3bb None None] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:45:08.729890 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1b982b9-3132-448d-8553-e69df37be3bb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:45:08.748416 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.auth.core [None req-d1b982b9-3132-448d-8553-e69df37be3bb None None] MFA Rules not processed for user `bac4e6b76fe64217ac8fa44e71cffe6a`. Rule list: `[]` (Enabled: `True`). {{(pid=80144) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 10:45:08.749608 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.fernet_utils [None req-d1b982b9-3132-448d-8553-e69df37be3bb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80144) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 10:45:08.813828 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 648/1275] 10.222.0.25 () {58 vars in 1024 bytes} [Sun Aug 7 10:45:08 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 07 10:45:08.837986 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ea69bc1-7c67-4693-863e-ab12458e5ce4 None None] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 10:45:08.936535 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ea69bc1-7c67-4693-863e-ab12458e5ce4 None None] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 10:45:08.953050 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=100 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 10:45:08.969754 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-547c0863-7368-48aa-8b95-115a0c04fada None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 10:45:08.970371 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-547c0863-7368-48aa-8b95-115a0c04fada None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:45:08.970650 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-547c0863-7368-48aa-8b95-115a0c04fada None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:45:08.970947 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-547c0863-7368-48aa-8b95-115a0c04fada None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:45:08.981312 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-547c0863-7368-48aa-8b95-115a0c04fada None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 07 10:45:08.985880 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application [None req-547c0863-7368-48aa-8b95-115a0c04fada None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 07 10:45:08.985880 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 07 10:45:08.985880 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 07 10:45:08.985880 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 07 10:45:08.985880 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 07 10:45:08.985880 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 07 10:45:08.985880 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 07 10:45:08.985880 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 07 10:45:08.985880 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 07 10:45:08.985880 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 07 10:45:08.985880 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 07 10:45:08.985880 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 07 10:45:08.985880 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 07 10:45:08.985880 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 07 10:45:08.985880 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 07 10:45:08.985880 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 07 10:45:08.986986 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 07 10:45:08.986986 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 07 10:45:08.986986 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 07 10:45:08.986986 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 07 10:45:08.986986 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 07 10:45:08.986986 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 07 10:45:08.986986 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 07 10:45:08.986986 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 07 10:45:08.986986 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 07 10:45:08.986986 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.get_or_create( Aug 07 10:45:08.986986 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 07 10:45:08.986986 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application with Lock( Aug 07 10:45:08.986986 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 07 10:45:08.986986 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self._enter() Aug 07 10:45:08.986986 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 07 10:45:08.986986 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 07 10:45:08.988048 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 07 10:45:08.988048 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.creator() Aug 07 10:45:08.988048 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 07 10:45:08.988048 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application created_value = creator( Aug 07 10:45:08.988048 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 07 10:45:08.988048 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 07 10:45:08.988048 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 07 10:45:08.988048 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 07 10:45:08.988048 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 07 10:45:08.988048 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 07 10:45:08.988048 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 07 10:45:08.988048 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application Aug 07 10:45:08.988048 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 628/1276] 10.222.0.25 () {56 vars in 1096 bytes} [Sun Aug 7 10:45:08 2022] GET /identity/v3/roles/admin => generated 83 bytes in 149 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 07 10:45:08.993337 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1b982b9-3132-448d-8553-e69df37be3bb None None] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 10:45:09.071393 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 10:45:09.089937 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1b982b9-3132-448d-8553-e69df37be3bb None None] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 10:45:09.124949 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f94b53a9-a913-432b-8df3-fa8398496cee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 10:45:09.125549 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f94b53a9-a913-432b-8df3-fa8398496cee None admin] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:45:09.125831 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f94b53a9-a913-432b-8df3-fa8398496cee None admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:45:09.126100 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f94b53a9-a913-432b-8df3-fa8398496cee None admin] PATH_INFO: `/v3/roles` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:45:09.126781 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f94b53a9-a913-432b-8df3-fa8398496cee None admin] RBAC: Adding query filter params (name=admin) {{(pid=80144) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 10:45:09.127099 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f94b53a9-a913-432b-8df3-fa8398496cee None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 10:45:09.128439 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f94b53a9-a913-432b-8df3-fa8398496cee None admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 10:45:09.137360 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 649/1277] 10.222.0.25 () {56 vars in 1099 bytes} [Sun Aug 7 10:45:08 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 10:45:09.143244 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-547c0863-7368-48aa-8b95-115a0c04fada None admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 10:45:09.240562 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-547c0863-7368-48aa-8b95-115a0c04fada None admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 10:45:09.273340 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf7da78d-a48d-4fe9-a43d-8a2e83e2e038 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 10:45:09.273950 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf7da78d-a48d-4fe9-a43d-8a2e83e2e038 None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:45:09.274225 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf7da78d-a48d-4fe9-a43d-8a2e83e2e038 None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:45:09.274492 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf7da78d-a48d-4fe9-a43d-8a2e83e2e038 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:45:09.283933 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf7da78d-a48d-4fe9-a43d-8a2e83e2e038 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 10:45:09.285351 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf7da78d-a48d-4fe9-a43d-8a2e83e2e038 None admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 10:45:09.299074 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application [None req-cf7da78d-a48d-4fe9-a43d-8a2e83e2e038 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 07 10:45:09.299074 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 07 10:45:09.299074 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 07 10:45:09.299074 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 07 10:45:09.299074 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 07 10:45:09.299074 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 07 10:45:09.299074 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 07 10:45:09.299074 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 07 10:45:09.299074 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 07 10:45:09.299074 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 07 10:45:09.299074 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 07 10:45:09.299074 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 07 10:45:09.299074 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 07 10:45:09.299074 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 07 10:45:09.299074 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 07 10:45:09.299074 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 07 10:45:09.300149 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 07 10:45:09.300149 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 07 10:45:09.300149 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 07 10:45:09.300149 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 07 10:45:09.300149 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 07 10:45:09.300149 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.get_or_create( Aug 07 10:45:09.300149 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 07 10:45:09.300149 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application with Lock( Aug 07 10:45:09.300149 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 07 10:45:09.300149 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self._enter() Aug 07 10:45:09.300149 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 07 10:45:09.300149 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 07 10:45:09.300149 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 07 10:45:09.300149 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.creator() Aug 07 10:45:09.300149 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 07 10:45:09.300149 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application created_value = creator( Aug 07 10:45:09.300149 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 07 10:45:09.301049 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 07 10:45:09.301049 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 07 10:45:09.301049 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application Aug 07 10:45:09.301049 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 629/1278] 10.222.0.25 () {56 vars in 1108 bytes} [Sun Aug 7 10:45:09 2022] GET /identity/v3/domains/Default => generated 87 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 07 10:45:09.306032 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f94b53a9-a913-432b-8df3-fa8398496cee None admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 10:45:09.404479 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f94b53a9-a913-432b-8df3-fa8398496cee None admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 10:45:09.438593 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b63fc663-887b-4ec5-8c89-e881521c5e16 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 10:45:09.439235 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b63fc663-887b-4ec5-8c89-e881521c5e16 None admin] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:45:09.439518 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b63fc663-887b-4ec5-8c89-e881521c5e16 None admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:45:09.439791 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b63fc663-887b-4ec5-8c89-e881521c5e16 None admin] PATH_INFO: `/v3/domains` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:45:09.440426 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b63fc663-887b-4ec5-8c89-e881521c5e16 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80144) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 10:45:09.440736 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b63fc663-887b-4ec5-8c89-e881521c5e16 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 10:45:09.442602 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b63fc663-887b-4ec5-8c89-e881521c5e16 None admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 10:45:09.454298 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 650/1279] 10.222.0.25 () {56 vars in 1109 bytes} [Sun Aug 7 10:45:09 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 10:45:09.460683 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf7da78d-a48d-4fe9-a43d-8a2e83e2e038 None admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 10:45:09.491468 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=100 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 10:45:09.558382 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf7da78d-a48d-4fe9-a43d-8a2e83e2e038 None admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 10:45:09.590582 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39872887-da4e-4420-8218-e1db6155bd9e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 10:45:09.591222 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39872887-da4e-4420-8218-e1db6155bd9e None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:45:09.591502 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39872887-da4e-4420-8218-e1db6155bd9e None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:45:09.591774 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39872887-da4e-4420-8218-e1db6155bd9e None admin] PATH_INFO: `/v3/users/placement` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:45:09.599016 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39872887-da4e-4420-8218-e1db6155bd9e None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 10:45:09.600432 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39872887-da4e-4420-8218-e1db6155bd9e None admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 10:45:09.610990 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application [None req-39872887-da4e-4420-8218-e1db6155bd9e None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Aug 07 10:45:09.610990 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 07 10:45:09.610990 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 07 10:45:09.610990 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 07 10:45:09.610990 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 07 10:45:09.610990 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 07 10:45:09.610990 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 07 10:45:09.610990 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 07 10:45:09.610990 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 07 10:45:09.610990 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 07 10:45:09.610990 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 07 10:45:09.610990 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 07 10:45:09.610990 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 07 10:45:09.610990 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 07 10:45:09.610990 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 07 10:45:09.610990 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 07 10:45:09.612165 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 07 10:45:09.612165 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 07 10:45:09.612165 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 07 10:45:09.612165 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 07 10:45:09.612165 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 07 10:45:09.612165 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 07 10:45:09.612165 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 07 10:45:09.612165 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 07 10:45:09.612165 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 07 10:45:09.612165 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.get_or_create( Aug 07 10:45:09.612165 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 07 10:45:09.612165 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application with Lock( Aug 07 10:45:09.612165 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 07 10:45:09.612165 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self._enter() Aug 07 10:45:09.612165 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 07 10:45:09.612165 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 07 10:45:09.612165 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 07 10:45:09.612165 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.creator() Aug 07 10:45:09.613323 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 07 10:45:09.613323 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application created_value = creator( Aug 07 10:45:09.613323 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 07 10:45:09.613323 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 07 10:45:09.613323 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 07 10:45:09.613323 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 07 10:45:09.613323 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 07 10:45:09.613323 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 07 10:45:09.613323 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Aug 07 10:45:09.613323 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application Aug 07 10:45:09.613323 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 630/1280] 10.222.0.25 () {56 vars in 1108 bytes} [Sun Aug 7 10:45:09 2022] GET /identity/v3/users/placement => generated 87 bytes in 152 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 07 10:45:09.618487 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b63fc663-887b-4ec5-8c89-e881521c5e16 None admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 10:45:09.622164 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=100 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 10:45:09.714947 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b63fc663-887b-4ec5-8c89-e881521c5e16 None admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 10:45:09.748320 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c246729-17fe-4394-9c76-54dc29dd9d5a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 10:45:09.748909 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c246729-17fe-4394-9c76-54dc29dd9d5a None admin] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:45:09.749113 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c246729-17fe-4394-9c76-54dc29dd9d5a None admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:45:09.749382 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c246729-17fe-4394-9c76-54dc29dd9d5a None admin] PATH_INFO: `/v3/users` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:45:09.750090 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c246729-17fe-4394-9c76-54dc29dd9d5a None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80144) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 10:45:09.750392 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c246729-17fe-4394-9c76-54dc29dd9d5a None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 10:45:09.751807 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c246729-17fe-4394-9c76-54dc29dd9d5a None admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 10:45:09.768432 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 651/1281] 10.222.0.25 () {56 vars in 1143 bytes} [Sun Aug 7 10:45:09 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 10:45:09.774172 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39872887-da4e-4420-8218-e1db6155bd9e None admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 10:45:09.860513 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=100 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 10:45:09.870480 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39872887-da4e-4420-8218-e1db6155bd9e None admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 10:45:09.903051 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-febc8ff5-1d7e-43c3-a625-99ac0cdfabb8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 10:45:09.903640 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-febc8ff5-1d7e-43c3-a625-99ac0cdfabb8 None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:45:09.903914 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-febc8ff5-1d7e-43c3-a625-99ac0cdfabb8 None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:45:09.904185 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-febc8ff5-1d7e-43c3-a625-99ac0cdfabb8 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:45:09.967833 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-febc8ff5-1d7e-43c3-a625-99ac0cdfabb8 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 10:45:09.969249 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-febc8ff5-1d7e-43c3-a625-99ac0cdfabb8 None admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 10:45:10.053218 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=100 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 10:45:10.063619 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 631/1282] 10.222.0.25 () {58 vars in 1282 bytes} [Sun Aug 7 10:45:09 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 290 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 07 10:45:10.069422 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c246729-17fe-4394-9c76-54dc29dd9d5a None admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 10:45:10.136834 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=100 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 10:45:10.174254 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c246729-17fe-4394-9c76-54dc29dd9d5a None admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 10:45:10.207121 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10e03667-a67a-41a2-bb2f-9bea91016586 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 10:45:10.207776 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10e03667-a67a-41a2-bb2f-9bea91016586 None admin] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:45:10.208060 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10e03667-a67a-41a2-bb2f-9bea91016586 None admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:45:10.208412 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10e03667-a67a-41a2-bb2f-9bea91016586 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:45:10.218017 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10e03667-a67a-41a2-bb2f-9bea91016586 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 10:45:10.219475 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10e03667-a67a-41a2-bb2f-9bea91016586 None admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 10:45:10.234066 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application [None req-10e03667-a67a-41a2-bb2f-9bea91016586 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 07 10:45:10.234066 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 07 10:45:10.234066 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 07 10:45:10.234066 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 07 10:45:10.234066 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 07 10:45:10.234066 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 07 10:45:10.234066 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 07 10:45:10.234066 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 07 10:45:10.234066 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 07 10:45:10.234066 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 07 10:45:10.234066 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 07 10:45:10.234066 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 07 10:45:10.234066 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 07 10:45:10.234066 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 07 10:45:10.234066 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 07 10:45:10.234066 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 07 10:45:10.235229 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 07 10:45:10.235229 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 07 10:45:10.235229 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 07 10:45:10.235229 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 07 10:45:10.235229 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 07 10:45:10.235229 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self.get_or_create( Aug 07 10:45:10.235229 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 07 10:45:10.235229 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application with Lock( Aug 07 10:45:10.235229 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 07 10:45:10.235229 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self._enter() Aug 07 10:45:10.235229 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 07 10:45:10.235229 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 07 10:45:10.235229 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 07 10:45:10.235229 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self.creator() Aug 07 10:45:10.235229 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 07 10:45:10.235229 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application created_value = creator( Aug 07 10:45:10.235229 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 07 10:45:10.236682 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 07 10:45:10.236682 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 07 10:45:10.236682 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application Aug 07 10:45:10.236682 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 652/1283] 10.222.0.25 () {56 vars in 1108 bytes} [Sun Aug 7 10:45:10 2022] GET /identity/v3/domains/Default => generated 87 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 07 10:45:10.241882 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-febc8ff5-1d7e-43c3-a625-99ac0cdfabb8 None admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 10:45:10.336489 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-febc8ff5-1d7e-43c3-a625-99ac0cdfabb8 None admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 10:45:10.368964 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ed8d8fc-bc60-4bdd-b403-63b1f9ac4188 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 10:45:10.369555 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ed8d8fc-bc60-4bdd-b403-63b1f9ac4188 None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:45:10.369834 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ed8d8fc-bc60-4bdd-b403-63b1f9ac4188 None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:45:10.370102 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ed8d8fc-bc60-4bdd-b403-63b1f9ac4188 None admin] PATH_INFO: `/v3/domains` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:45:10.370728 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ed8d8fc-bc60-4bdd-b403-63b1f9ac4188 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80143) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 10:45:10.371053 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ed8d8fc-bc60-4bdd-b403-63b1f9ac4188 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 10:45:10.372393 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ed8d8fc-bc60-4bdd-b403-63b1f9ac4188 None admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 10:45:10.383127 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 632/1284] 10.222.0.25 () {56 vars in 1109 bytes} [Sun Aug 7 10:45:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 10:45:10.388732 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10e03667-a67a-41a2-bb2f-9bea91016586 None admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 10:45:10.486555 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10e03667-a67a-41a2-bb2f-9bea91016586 None admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 10:45:10.499804 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=100 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 10:45:10.519921 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e724e6b-e564-49c4-9588-5d9ad0650037 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 10:45:10.521161 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e724e6b-e564-49c4-9588-5d9ad0650037 None admin] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:45:10.521435 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e724e6b-e564-49c4-9588-5d9ad0650037 None admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:45:10.521706 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e724e6b-e564-49c4-9588-5d9ad0650037 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:45:10.528071 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e724e6b-e564-49c4-9588-5d9ad0650037 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 10:45:10.529486 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e724e6b-e564-49c4-9588-5d9ad0650037 None admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 10:45:10.540665 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application [None req-4e724e6b-e564-49c4-9588-5d9ad0650037 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 07 10:45:10.540665 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 07 10:45:10.540665 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 07 10:45:10.540665 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 07 10:45:10.540665 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 07 10:45:10.540665 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 07 10:45:10.540665 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 07 10:45:10.540665 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 07 10:45:10.540665 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 07 10:45:10.540665 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 07 10:45:10.540665 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 07 10:45:10.540665 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 07 10:45:10.540665 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 07 10:45:10.540665 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 07 10:45:10.540665 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 07 10:45:10.541691 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 07 10:45:10.541691 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 07 10:45:10.541691 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 07 10:45:10.541691 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 07 10:45:10.541691 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 07 10:45:10.541691 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 07 10:45:10.541691 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self.get_or_create( Aug 07 10:45:10.541691 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 07 10:45:10.541691 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application with Lock( Aug 07 10:45:10.541691 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 07 10:45:10.541691 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self._enter() Aug 07 10:45:10.541691 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 07 10:45:10.541691 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 07 10:45:10.541691 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 07 10:45:10.541691 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self.creator() Aug 07 10:45:10.541691 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 07 10:45:10.541691 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application created_value = creator( Aug 07 10:45:10.542981 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 07 10:45:10.542981 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 07 10:45:10.542981 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 07 10:45:10.542981 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 07 10:45:10.542981 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 07 10:45:10.542981 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 07 10:45:10.542981 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 07 10:45:10.542981 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: ERROR keystone.server.flask.application Aug 07 10:45:10.542981 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 653/1285] 10.222.0.25 () {56 vars in 1111 bytes} [Sun Aug 7 10:45:10 2022] GET /identity/v3/projects/service => generated 88 bytes in 153 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 07 10:45:10.547505 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ed8d8fc-bc60-4bdd-b403-63b1f9ac4188 None admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 10:45:10.596039 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=100 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 10:45:10.644950 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ed8d8fc-bc60-4bdd-b403-63b1f9ac4188 None admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 10:45:10.677396 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ea467a0-8c74-48dc-bf5d-e9f15e576966 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 10:45:10.677987 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ea467a0-8c74-48dc-bf5d-e9f15e576966 None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:45:10.678265 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ea467a0-8c74-48dc-bf5d-e9f15e576966 None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:45:10.678533 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ea467a0-8c74-48dc-bf5d-e9f15e576966 None admin] PATH_INFO: `/v3/projects` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:45:10.679244 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ea467a0-8c74-48dc-bf5d-e9f15e576966 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80143) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 10:45:10.679547 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ea467a0-8c74-48dc-bf5d-e9f15e576966 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 10:45:10.680952 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ea467a0-8c74-48dc-bf5d-e9f15e576966 None admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 10:45:10.693792 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 633/1286] 10.222.0.25 () {56 vars in 1148 bytes} [Sun Aug 7 10:45:10 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 07 10:45:10.699746 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e724e6b-e564-49c4-9588-5d9ad0650037 None admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 10:45:10.796104 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e724e6b-e564-49c4-9588-5d9ad0650037 None admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 10:45:10.827785 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-621608d6-10f3-46c0-bece-6fe282d600ca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 10:45:10.828364 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-621608d6-10f3-46c0-bece-6fe282d600ca None admin] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:45:10.828640 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-621608d6-10f3-46c0-bece-6fe282d600ca None admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:45:10.828906 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-621608d6-10f3-46c0-bece-6fe282d600ca None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:45:10.829607 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-621608d6-10f3-46c0-bece-6fe282d600ca None admin] RBAC: Adding query filter params (role.id=81b1f1bcee2a473088fbbd327c1c5b23, scope.project.id=6de1f3cac02c4bcdb05dd2337176033d, user.id=6ee2e4c08af04c219ed395d7ca7085a6) {{(pid=80144) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 10:45:10.829914 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-621608d6-10f3-46c0-bece-6fe282d600ca None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 10:45:10.831335 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-621608d6-10f3-46c0-bece-6fe282d600ca None admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 10:45:10.837762 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 654/1287] 10.222.0.25 () {56 vars in 1374 bytes} [Sun Aug 7 10:45:10 2022] GET /identity/v3/role_assignments?user.id=6ee2e4c08af04c219ed395d7ca7085a6&scope.project.id=6de1f3cac02c4bcdb05dd2337176033d&role.id=81b1f1bcee2a473088fbbd327c1c5b23 => generated 608 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 10:45:12.884575 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5de2add9-ca03-4688-a652-65b50f9a436d None None] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:45:12.885101 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5de2add9-ca03-4688-a652-65b50f9a436d None None] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:45:12.885101 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5de2add9-ca03-4688-a652-65b50f9a436d None None] PATH_INFO: `/` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:45:12.886431 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 634/1288] 10.222.0.25 () {54 vars in 927 bytes} [Sun Aug 7 10:45:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 07 10:45:12.891556 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61a1c4b5-8975-41fd-b7b0-d73dcc965399 None None] REQUEST_METHOD: `POST` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:45:12.891986 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61a1c4b5-8975-41fd-b7b0-d73dcc965399 None None] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:45:12.892108 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61a1c4b5-8975-41fd-b7b0-d73dcc965399 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:45:12.909890 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.auth.core [None req-61a1c4b5-8975-41fd-b7b0-d73dcc965399 None None] MFA Rules not processed for user `bac4e6b76fe64217ac8fa44e71cffe6a`. Rule list: `[]` (Enabled: `True`). {{(pid=80144) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 10:45:12.911607 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.fernet_utils [None req-61a1c4b5-8975-41fd-b7b0-d73dcc965399 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80144) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 10:45:12.940250 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=100 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 10:45:12.976735 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 655/1289] 10.222.0.25 () {58 vars in 1024 bytes} [Sun Aug 7 10:45:12 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 07 10:45:12.999278 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5de2add9-ca03-4688-a652-65b50f9a436d None None] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 10:45:13.095637 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5de2add9-ca03-4688-a652-65b50f9a436d None None] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 10:45:13.107007 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=100 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 10:45:13.131795 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e73bd2ba-5a59-45ad-bd03-71a81e35adda None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 10:45:13.132425 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e73bd2ba-5a59-45ad-bd03-71a81e35adda None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:45:13.132701 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e73bd2ba-5a59-45ad-bd03-71a81e35adda None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:45:13.132999 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e73bd2ba-5a59-45ad-bd03-71a81e35adda None admin] PATH_INFO: `/v3/services/placement` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:45:13.133943 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e73bd2ba-5a59-45ad-bd03-71a81e35adda None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 10:45:13.135703 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e73bd2ba-5a59-45ad-bd03-71a81e35adda None admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 10:45:13.144879 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application [None req-e73bd2ba-5a59-45ad-bd03-71a81e35adda None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Aug 07 10:45:13.144879 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 07 10:45:13.144879 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 07 10:45:13.144879 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 07 10:45:13.144879 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 07 10:45:13.144879 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 07 10:45:13.144879 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 07 10:45:13.144879 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 07 10:45:13.144879 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Aug 07 10:45:13.144879 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application Aug 07 10:45:13.144879 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 07 10:45:13.144879 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application Aug 07 10:45:13.144879 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 07 10:45:13.144879 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 07 10:45:13.144879 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 07 10:45:13.144879 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 07 10:45:13.144879 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 07 10:45:13.146810 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 07 10:45:13.146810 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 07 10:45:13.146810 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 07 10:45:13.146810 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 07 10:45:13.146810 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 07 10:45:13.146810 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 07 10:45:13.146810 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 07 10:45:13.146810 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 07 10:45:13.146810 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 07 10:45:13.146810 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 07 10:45:13.146810 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 07 10:45:13.146810 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 07 10:45:13.146810 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 07 10:45:13.146810 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 07 10:45:13.146810 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 07 10:45:13.146810 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.get_or_create( Aug 07 10:45:13.148058 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 07 10:45:13.148058 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application with Lock( Aug 07 10:45:13.148058 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 07 10:45:13.148058 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self._enter() Aug 07 10:45:13.148058 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 07 10:45:13.148058 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 07 10:45:13.148058 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 07 10:45:13.148058 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.creator() Aug 07 10:45:13.148058 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 07 10:45:13.148058 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application created_value = creator( Aug 07 10:45:13.148058 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 07 10:45:13.148058 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 07 10:45:13.148058 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Aug 07 10:45:13.148058 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application Aug 07 10:45:13.148058 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 635/1290] 10.222.0.25 () {56 vars in 1117 bytes} [Sun Aug 7 10:45:12 2022] GET /identity/v3/services/placement => generated 90 bytes in 147 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 07 10:45:13.152053 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61a1c4b5-8975-41fd-b7b0-d73dcc965399 None None] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 10:45:13.251594 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61a1c4b5-8975-41fd-b7b0-d73dcc965399 None None] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 10:45:13.284857 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b3fc0ca-3c98-497e-aef7-a31b0dcfb5c8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 10:45:13.285452 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b3fc0ca-3c98-497e-aef7-a31b0dcfb5c8 None admin] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:45:13.286294 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b3fc0ca-3c98-497e-aef7-a31b0dcfb5c8 None admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:45:13.287122 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b3fc0ca-3c98-497e-aef7-a31b0dcfb5c8 None admin] PATH_INFO: `/v3/services` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:45:13.287746 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b3fc0ca-3c98-497e-aef7-a31b0dcfb5c8 None admin] RBAC: Adding query filter params (name=placement) {{(pid=80144) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 10:45:13.288054 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b3fc0ca-3c98-497e-aef7-a31b0dcfb5c8 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 10:45:13.289387 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b3fc0ca-3c98-497e-aef7-a31b0dcfb5c8 None admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 10:45:13.294009 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 656/1291] 10.222.0.25 () {56 vars in 1116 bytes} [Sun Aug 7 10:45:13 2022] GET /identity/v3/services?name=placement => generated 126 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 10:45:13.299804 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e73bd2ba-5a59-45ad-bd03-71a81e35adda None admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 10:45:13.396651 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e73bd2ba-5a59-45ad-bd03-71a81e35adda None admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 10:45:13.430001 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6522ffb8-0717-4cca-b4ab-e4b2db65d724 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 10:45:13.430607 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6522ffb8-0717-4cca-b4ab-e4b2db65d724 None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:45:13.430891 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6522ffb8-0717-4cca-b4ab-e4b2db65d724 None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:45:13.431197 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6522ffb8-0717-4cca-b4ab-e4b2db65d724 None admin] PATH_INFO: `/v3/services` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:45:13.431829 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6522ffb8-0717-4cca-b4ab-e4b2db65d724 None admin] RBAC: Adding query filter params (type=placement) {{(pid=80143) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 10:45:13.432138 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6522ffb8-0717-4cca-b4ab-e4b2db65d724 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 10:45:13.433494 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6522ffb8-0717-4cca-b4ab-e4b2db65d724 None admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 10:45:13.437767 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 636/1292] 10.222.0.25 () {56 vars in 1116 bytes} [Sun Aug 7 10:45:13 2022] GET /identity/v3/services?type=placement => generated 126 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 10:45:15.461656 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-442a8d12-2626-4150-9434-29d2015b38f2 None None] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:45:15.462160 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-442a8d12-2626-4150-9434-29d2015b38f2 None None] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:45:15.462264 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-442a8d12-2626-4150-9434-29d2015b38f2 None None] PATH_INFO: `/` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:45:15.463590 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 657/1293] 10.222.0.25 () {54 vars in 927 bytes} [Sun Aug 7 10:45:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 07 10:45:15.468666 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7019a89b-2953-4603-96ea-d819a5644cd7 None None] REQUEST_METHOD: `POST` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:45:15.469036 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7019a89b-2953-4603-96ea-d819a5644cd7 None None] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:45:15.469215 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7019a89b-2953-4603-96ea-d819a5644cd7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:45:15.487498 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.auth.core [None req-7019a89b-2953-4603-96ea-d819a5644cd7 None None] MFA Rules not processed for user `bac4e6b76fe64217ac8fa44e71cffe6a`. Rule list: `[]` (Enabled: `True`). {{(pid=80143) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 10:45:15.488666 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-7019a89b-2953-4603-96ea-d819a5644cd7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 10:45:15.518495 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=100 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 10:45:15.556380 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 637/1294] 10.222.0.25 () {58 vars in 1024 bytes} [Sun Aug 7 10:45:15 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 07 10:45:15.579568 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-442a8d12-2626-4150-9434-29d2015b38f2 None None] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 10:45:15.609151 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=100 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 10:45:15.677748 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-442a8d12-2626-4150-9434-29d2015b38f2 None None] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 10:45:15.709577 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68d87e94-e0eb-4612-8911-508f286e3305 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 10:45:15.710169 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68d87e94-e0eb-4612-8911-508f286e3305 None admin] REQUEST_METHOD: `POST` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:45:15.710447 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68d87e94-e0eb-4612-8911-508f286e3305 None admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:45:15.710762 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68d87e94-e0eb-4612-8911-508f286e3305 None admin] PATH_INFO: `/v3/services` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:45:15.711380 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68d87e94-e0eb-4612-8911-508f286e3305 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 10:45:15.712719 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68d87e94-e0eb-4612-8911-508f286e3305 None admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 10:45:15.726442 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 658/1295] 10.222.0.25 () {60 vars in 1141 bytes} [Sun Aug 7 10:45:15 2022] POST /identity/v3/services => generated 246 bytes in 147 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 07 10:45:17.706141 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73e60c87-654b-4a61-8ae0-13fbd14a608b None None] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:45:17.706687 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73e60c87-654b-4a61-8ae0-13fbd14a608b None None] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:45:17.706687 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73e60c87-654b-4a61-8ae0-13fbd14a608b None None] PATH_INFO: `/` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:45:17.708019 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 638/1296] 10.222.0.25 () {54 vars in 927 bytes} [Sun Aug 7 10:45:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 07 10:45:17.713079 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3985a473-f941-4024-babf-1674b281690d None None] REQUEST_METHOD: `POST` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:45:17.713525 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3985a473-f941-4024-babf-1674b281690d None None] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:45:17.713657 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3985a473-f941-4024-babf-1674b281690d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:45:17.731532 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.auth.core [None req-3985a473-f941-4024-babf-1674b281690d None None] MFA Rules not processed for user `bac4e6b76fe64217ac8fa44e71cffe6a`. Rule list: `[]` (Enabled: `True`). {{(pid=80144) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 10:45:17.732695 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.fernet_utils [None req-3985a473-f941-4024-babf-1674b281690d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80144) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 10:45:17.803260 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 659/1297] 10.222.0.25 () {58 vars in 1024 bytes} [Sun Aug 7 10:45:17 2022] POST /identity/v3/auth/tokens => generated 2221 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 07 10:45:17.825018 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73e60c87-654b-4a61-8ae0-13fbd14a608b None None] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 10:45:17.922471 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73e60c87-654b-4a61-8ae0-13fbd14a608b None None] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 10:45:17.954602 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1b26ee0-27fc-40e3-8972-378029ecef31 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 10:45:17.955253 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1b26ee0-27fc-40e3-8972-378029ecef31 None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:45:17.955538 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1b26ee0-27fc-40e3-8972-378029ecef31 None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:45:17.955812 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1b26ee0-27fc-40e3-8972-378029ecef31 None admin] PATH_INFO: `/v3/services/placement` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:45:17.956444 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1b26ee0-27fc-40e3-8972-378029ecef31 None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 10:45:17.958282 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1b26ee0-27fc-40e3-8972-378029ecef31 None admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 10:45:17.966704 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application [None req-a1b26ee0-27fc-40e3-8972-378029ecef31 None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Aug 07 10:45:17.966704 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 07 10:45:17.966704 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 07 10:45:17.966704 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 07 10:45:17.966704 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 07 10:45:17.966704 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 07 10:45:17.966704 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 07 10:45:17.966704 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 07 10:45:17.966704 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Aug 07 10:45:17.966704 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application Aug 07 10:45:17.966704 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 07 10:45:17.966704 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application Aug 07 10:45:17.966704 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 07 10:45:17.966704 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 07 10:45:17.966704 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 07 10:45:17.966704 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 07 10:45:17.966704 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 07 10:45:17.967955 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 07 10:45:17.967955 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 07 10:45:17.967955 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 07 10:45:17.967955 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 07 10:45:17.967955 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 07 10:45:17.967955 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 07 10:45:17.967955 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 07 10:45:17.967955 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 07 10:45:17.967955 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 07 10:45:17.967955 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 07 10:45:17.967955 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 07 10:45:17.967955 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 07 10:45:17.967955 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 07 10:45:17.967955 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 07 10:45:17.967955 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 07 10:45:17.967955 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.get_or_create( Aug 07 10:45:17.968853 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 07 10:45:17.968853 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application with Lock( Aug 07 10:45:17.968853 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 07 10:45:17.968853 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self._enter() Aug 07 10:45:17.968853 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 07 10:45:17.968853 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 07 10:45:17.968853 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 07 10:45:17.968853 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.creator() Aug 07 10:45:17.968853 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 07 10:45:17.968853 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application created_value = creator( Aug 07 10:45:17.968853 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 07 10:45:17.968853 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 07 10:45:17.968853 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Aug 07 10:45:17.968853 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application Aug 07 10:45:17.968853 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 639/1298] 10.222.0.25 () {56 vars in 1117 bytes} [Sun Aug 7 10:45:17 2022] GET /identity/v3/services/placement => generated 90 bytes in 143 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 07 10:45:17.974221 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3985a473-f941-4024-babf-1674b281690d None None] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 10:45:17.992185 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 10:45:18.068667 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3985a473-f941-4024-babf-1674b281690d None None] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 10:45:18.103129 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ed15641-27f1-4fbc-b8bc-fd8bcbc267ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 10:45:18.103876 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ed15641-27f1-4fbc-b8bc-fd8bcbc267ac None admin] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:45:18.104202 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ed15641-27f1-4fbc-b8bc-fd8bcbc267ac None admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:45:18.104552 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ed15641-27f1-4fbc-b8bc-fd8bcbc267ac None admin] PATH_INFO: `/v3/services` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:45:18.105494 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ed15641-27f1-4fbc-b8bc-fd8bcbc267ac None admin] RBAC: Adding query filter params (name=placement) {{(pid=80144) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 10:45:18.105883 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ed15641-27f1-4fbc-b8bc-fd8bcbc267ac None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 10:45:18.107694 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ed15641-27f1-4fbc-b8bc-fd8bcbc267ac None admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 10:45:18.114480 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 660/1299] 10.222.0.25 () {56 vars in 1116 bytes} [Sun Aug 7 10:45:17 2022] GET /identity/v3/services?name=placement => generated 358 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 10:45:18.120258 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1b26ee0-27fc-40e3-8972-378029ecef31 None admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 10:45:18.157252 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=100 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 10:45:18.227388 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1b26ee0-27fc-40e3-8972-378029ecef31 None admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 10:45:18.259961 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-383b5313-9256-439c-9de9-b1fd890dce23 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 10:45:18.260570 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-383b5313-9256-439c-9de9-b1fd890dce23 None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:45:18.260850 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-383b5313-9256-439c-9de9-b1fd890dce23 None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:45:18.261122 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-383b5313-9256-439c-9de9-b1fd890dce23 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:45:18.261814 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-383b5313-9256-439c-9de9-b1fd890dce23 None admin] RBAC: Adding query filter params (interface=public, service_id=df4e7ed76b994e4db74b40f8c7d61f8c, region_id=RegionOne) {{(pid=80143) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 10:45:18.262124 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-383b5313-9256-439c-9de9-b1fd890dce23 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 10:45:18.263519 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-383b5313-9256-439c-9de9-b1fd890dce23 None admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 10:45:18.268340 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 640/1300] 10.222.0.25 () {56 vars in 1251 bytes} [Sun Aug 7 10:45:18 2022] GET /identity/v3/endpoints?service_id=df4e7ed76b994e4db74b40f8c7d61f8c&interface=public®ion_id=RegionOne => generated 194 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 10:45:18.274795 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ed15641-27f1-4fbc-b8bc-fd8bcbc267ac None admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 10:45:18.354792 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=100 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 10:45:18.372541 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ed15641-27f1-4fbc-b8bc-fd8bcbc267ac None admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 10:45:18.404711 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8429b543-1a6f-4d70-a2cf-bba558b2e572 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 10:45:18.405307 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8429b543-1a6f-4d70-a2cf-bba558b2e572 None admin] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:45:18.405585 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8429b543-1a6f-4d70-a2cf-bba558b2e572 None admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:45:18.405853 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8429b543-1a6f-4d70-a2cf-bba558b2e572 None admin] PATH_INFO: `/v3/services` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:45:18.406444 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8429b543-1a6f-4d70-a2cf-bba558b2e572 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 10:45:18.407817 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8429b543-1a6f-4d70-a2cf-bba558b2e572 None admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 10:45:18.414680 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 661/1301] 10.222.0.25 () {56 vars in 1087 bytes} [Sun Aug 7 10:45:18 2022] GET /identity/v3/services => generated 2191 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 07 10:45:20.396754 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91e74e04-ae5c-4268-99fc-d2ccd26169d6 None None] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:45:20.397306 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91e74e04-ae5c-4268-99fc-d2ccd26169d6 None None] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:45:20.397306 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91e74e04-ae5c-4268-99fc-d2ccd26169d6 None None] PATH_INFO: `/` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:45:20.398646 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 641/1302] 10.222.0.25 () {54 vars in 927 bytes} [Sun Aug 7 10:45:20 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 07 10:45:20.403874 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36b18597-18a6-4234-9026-0cb82d6b9bb1 None None] REQUEST_METHOD: `POST` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:45:20.404302 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36b18597-18a6-4234-9026-0cb82d6b9bb1 None None] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:45:20.404430 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36b18597-18a6-4234-9026-0cb82d6b9bb1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:45:20.422154 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.auth.core [None req-36b18597-18a6-4234-9026-0cb82d6b9bb1 None None] MFA Rules not processed for user `bac4e6b76fe64217ac8fa44e71cffe6a`. Rule list: `[]` (Enabled: `True`). {{(pid=80144) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 10:45:20.423357 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.fernet_utils [None req-36b18597-18a6-4234-9026-0cb82d6b9bb1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80144) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 10:45:20.487546 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 662/1303] 10.222.0.25 () {58 vars in 1024 bytes} [Sun Aug 7 10:45:20 2022] POST /identity/v3/auth/tokens => generated 2221 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 07 10:45:20.511867 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91e74e04-ae5c-4268-99fc-d2ccd26169d6 None None] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 10:45:20.606005 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=100 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 10:45:20.613016 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91e74e04-ae5c-4268-99fc-d2ccd26169d6 None None] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 10:45:20.645793 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7bbefeb-d973-42cd-a32a-db04589ae9eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 10:45:20.646403 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7bbefeb-d973-42cd-a32a-db04589ae9eb None admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:45:20.646678 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7bbefeb-d973-42cd-a32a-db04589ae9eb None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:45:20.647024 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7bbefeb-d973-42cd-a32a-db04589ae9eb None admin] PATH_INFO: `/v3/services/placement` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:45:20.647578 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7bbefeb-d973-42cd-a32a-db04589ae9eb None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 10:45:20.648906 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7bbefeb-d973-42cd-a32a-db04589ae9eb None admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 10:45:20.657264 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application [None req-e7bbefeb-d973-42cd-a32a-db04589ae9eb None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Aug 07 10:45:20.657264 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 07 10:45:20.657264 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 07 10:45:20.657264 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 07 10:45:20.657264 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 07 10:45:20.657264 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 07 10:45:20.657264 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 07 10:45:20.657264 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 07 10:45:20.657264 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Aug 07 10:45:20.657264 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application Aug 07 10:45:20.657264 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 07 10:45:20.657264 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application Aug 07 10:45:20.657264 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 07 10:45:20.657264 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 07 10:45:20.657264 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 07 10:45:20.657264 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 07 10:45:20.657264 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 07 10:45:20.658532 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 07 10:45:20.658532 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 07 10:45:20.658532 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 07 10:45:20.658532 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 07 10:45:20.658532 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 07 10:45:20.658532 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 07 10:45:20.658532 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 07 10:45:20.658532 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 07 10:45:20.658532 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 07 10:45:20.658532 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 07 10:45:20.658532 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 07 10:45:20.658532 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 07 10:45:20.658532 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 07 10:45:20.658532 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 07 10:45:20.658532 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 07 10:45:20.658532 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.get_or_create( Aug 07 10:45:20.659598 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 07 10:45:20.659598 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application with Lock( Aug 07 10:45:20.659598 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 07 10:45:20.659598 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self._enter() Aug 07 10:45:20.659598 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 07 10:45:20.659598 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 07 10:45:20.659598 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 07 10:45:20.659598 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application return self.creator() Aug 07 10:45:20.659598 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 07 10:45:20.659598 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application created_value = creator( Aug 07 10:45:20.659598 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 07 10:45:20.659598 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 07 10:45:20.659598 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Aug 07 10:45:20.659598 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: ERROR keystone.server.flask.application Aug 07 10:45:20.659598 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 642/1304] 10.222.0.25 () {56 vars in 1117 bytes} [Sun Aug 7 10:45:20 2022] GET /identity/v3/services/placement => generated 90 bytes in 147 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 07 10:45:20.664482 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36b18597-18a6-4234-9026-0cb82d6b9bb1 None None] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 10:45:20.740116 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=100 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 10:45:20.763361 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36b18597-18a6-4234-9026-0cb82d6b9bb1 None None] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 10:45:20.796373 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af5d6293-c280-48bb-8aac-2baedc1de180 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 10:45:20.796971 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af5d6293-c280-48bb-8aac-2baedc1de180 None admin] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:45:20.797252 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af5d6293-c280-48bb-8aac-2baedc1de180 None admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:45:20.797526 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af5d6293-c280-48bb-8aac-2baedc1de180 None admin] PATH_INFO: `/v3/services` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:45:20.798147 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af5d6293-c280-48bb-8aac-2baedc1de180 None admin] RBAC: Adding query filter params (name=placement) {{(pid=80144) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 10:45:20.798453 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af5d6293-c280-48bb-8aac-2baedc1de180 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 10:45:20.799825 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af5d6293-c280-48bb-8aac-2baedc1de180 None admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 10:45:20.805754 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 663/1305] 10.222.0.25 () {56 vars in 1116 bytes} [Sun Aug 7 10:45:20 2022] GET /identity/v3/services?name=placement => generated 358 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 10:45:20.811443 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7bbefeb-d973-42cd-a32a-db04589ae9eb None admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 10:45:20.907389 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7bbefeb-d973-42cd-a32a-db04589ae9eb None admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 10:45:20.939313 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90304380-d40f-420d-88ec-46747e9f49dc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 10:45:20.939901 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90304380-d40f-420d-88ec-46747e9f49dc None admin] REQUEST_METHOD: `POST` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:45:20.940176 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90304380-d40f-420d-88ec-46747e9f49dc None admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:45:20.940444 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90304380-d40f-420d-88ec-46747e9f49dc None admin] PATH_INFO: `/v3/endpoints` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:45:20.941108 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90304380-d40f-420d-88ec-46747e9f49dc None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 10:45:20.943023 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90304380-d40f-420d-88ec-46747e9f49dc None admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 10:45:20.961458 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 643/1306] 10.222.0.25 () {60 vars in 1144 bytes} [Sun Aug 7 10:45:20 2022] POST /identity/v3/endpoints => generated 331 bytes in 150 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 07 10:45:30.802975 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=30 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 10:45:30.957403 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=91,keystone:INSERT=1 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 10:46:23.563612 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a32d4de-b6d4-42e9-94ea-3f162d76d308 None None] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:46:23.564212 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a32d4de-b6d4-42e9-94ea-3f162d76d308 None None] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:46:23.564212 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a32d4de-b6d4-42e9-94ea-3f162d76d308 None None] PATH_INFO: `/` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:46:23.569863 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 664/1307] 10.222.0.25 () {54 vars in 927 bytes} [Sun Aug 7 10:46:23 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 07 10:46:23.571337 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb5e9724-c1f9-4713-8975-002cac3f2717 None None] REQUEST_METHOD: `POST` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:46:23.571784 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb5e9724-c1f9-4713-8975-002cac3f2717 None None] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:46:23.571883 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb5e9724-c1f9-4713-8975-002cac3f2717 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:46:23.648073 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.auth.core [None req-bb5e9724-c1f9-4713-8975-002cac3f2717 None None] MFA Rules not processed for user `bac4e6b76fe64217ac8fa44e71cffe6a`. Rule list: `[]` (Enabled: `True`). {{(pid=80143) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 10:46:23.649386 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-bb5e9724-c1f9-4713-8975-002cac3f2717 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 10:46:23.847392 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 644/1308] 10.222.0.25 () {58 vars in 1024 bytes} [Sun Aug 7 10:46:23 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 257 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 07 10:46:23.900060 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18e300a5-d17c-4ce0-bfb7-a3fd2f351c02 None None] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:46:23.900379 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18e300a5-d17c-4ce0-bfb7-a3fd2f351c02 None None] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:46:23.900678 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18e300a5-d17c-4ce0-bfb7-a3fd2f351c02 None None] PATH_INFO: `/` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:46:23.902178 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 665/1309] 10.222.0.25 () {54 vars in 965 bytes} [Sun Aug 7 10:46:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 07 10:46:23.911147 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79d3d1bb-c1dc-4b96-abbe-7e70b5d62add None None] REQUEST_METHOD: `POST` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:46:23.912479 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79d3d1bb-c1dc-4b96-abbe-7e70b5d62add None None] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:46:23.913903 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79d3d1bb-c1dc-4b96-abbe-7e70b5d62add None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:46:24.026647 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.auth.core [None req-79d3d1bb-c1dc-4b96-abbe-7e70b5d62add None None] MFA Rules not processed for user `be1283c9bf47424c9aa3ad27cab03027`. Rule list: `[]` (Enabled: `True`). {{(pid=80143) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 10:46:24.027892 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-79d3d1bb-c1dc-4b96-abbe-7e70b5d62add None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 10:46:24.050007 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=100 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 10:46:24.109709 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 645/1310] 10.222.0.25 () {58 vars in 1062 bytes} [Sun Aug 7 10:46:23 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 202 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 07 10:46:24.119614 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18e300a5-d17c-4ce0-bfb7-a3fd2f351c02 None None] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 10:46:24.121416 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.fernet_utils [None req-18e300a5-d17c-4ce0-bfb7-a3fd2f351c02 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80144) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 10:46:24.242959 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18e300a5-d17c-4ce0-bfb7-a3fd2f351c02 None None] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 10:46:24.320150 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c224d9e0-6421-45e9-96f8-583673be434d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be1283c9bf47424c9aa3ad27cab03027', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6de1f3cac02c4bcdb05dd2337176033d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 10:46:24.320758 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c224d9e0-6421-45e9-96f8-583673be434d service neutron] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:46:24.321036 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c224d9e0-6421-45e9-96f8-583673be434d service neutron] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:46:24.321315 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c224d9e0-6421-45e9-96f8-583673be434d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:46:24.322902 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.fernet_utils [None req-c224d9e0-6421-45e9-96f8-583673be434d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80144) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 10:46:24.381327 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=100 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 10:46:24.447143 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c224d9e0-6421-45e9-96f8-583673be434d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 10:46:24.448579 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c224d9e0-6421-45e9-96f8-583673be434d service neutron] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 10:46:24.471582 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 666/1311] 10.222.0.25 () {60 vars in 1365 bytes} [Sun Aug 7 10:46:24 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 352 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 07 10:46:26.853411 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5d8d85c-0d83-440b-ad94-e13aa7097b0a None None] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:46:26.853941 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5d8d85c-0d83-440b-ad94-e13aa7097b0a None None] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:46:26.853941 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5d8d85c-0d83-440b-ad94-e13aa7097b0a None None] PATH_INFO: `/` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:46:26.855472 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 646/1312] 10.222.0.25 () {54 vars in 927 bytes} [Sun Aug 7 10:46:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 07 10:46:26.860599 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0f93ca3-fe22-4a0c-b92a-8afe10952764 None None] REQUEST_METHOD: `POST` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:46:26.861002 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0f93ca3-fe22-4a0c-b92a-8afe10952764 None None] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:46:26.861221 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0f93ca3-fe22-4a0c-b92a-8afe10952764 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:46:26.951691 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.auth.core [None req-e0f93ca3-fe22-4a0c-b92a-8afe10952764 None None] MFA Rules not processed for user `bac4e6b76fe64217ac8fa44e71cffe6a`. Rule list: `[]` (Enabled: `True`). {{(pid=80144) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 10:46:26.953112 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.fernet_utils [None req-e0f93ca3-fe22-4a0c-b92a-8afe10952764 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80144) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 10:46:27.062898 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 667/1313] 10.222.0.25 () {58 vars in 1024 bytes} [Sun Aug 7 10:46:26 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 203 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 07 10:46:27.115869 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e1ee2f7-1b82-4348-9a24-6f9e0b5bfdd7 None None] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:46:27.116072 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e1ee2f7-1b82-4348-9a24-6f9e0b5bfdd7 None None] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:46:27.116448 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e1ee2f7-1b82-4348-9a24-6f9e0b5bfdd7 None None] PATH_INFO: `/` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:46:27.118030 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 647/1314] 10.222.0.25 () {54 vars in 965 bytes} [Sun Aug 7 10:46:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 07 10:46:27.124427 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19feacf3-d5ca-4faf-9cc9-925614b5e875 None None] REQUEST_METHOD: `POST` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:46:27.124742 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19feacf3-d5ca-4faf-9cc9-925614b5e875 None None] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:46:27.124999 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19feacf3-d5ca-4faf-9cc9-925614b5e875 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:46:27.138096 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=100 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 10:46:27.216897 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.auth.core [None req-19feacf3-d5ca-4faf-9cc9-925614b5e875 None None] MFA Rules not processed for user `be1283c9bf47424c9aa3ad27cab03027`. Rule list: `[]` (Enabled: `True`). {{(pid=80144) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 10:46:27.218127 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.fernet_utils [None req-19feacf3-d5ca-4faf-9cc9-925614b5e875 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80144) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 10:46:27.300383 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 668/1315] 10.222.0.25 () {58 vars in 1062 bytes} [Sun Aug 7 10:46:27 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 177 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 07 10:46:27.309397 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e1ee2f7-1b82-4348-9a24-6f9e0b5bfdd7 None None] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 10:46:27.310951 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-3e1ee2f7-1b82-4348-9a24-6f9e0b5bfdd7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 10:46:27.419187 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e1ee2f7-1b82-4348-9a24-6f9e0b5bfdd7 None None] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 10:46:27.494107 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=100 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 10:46:27.497729 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d14c779-df73-4df8-99fc-6ccff12a0df5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be1283c9bf47424c9aa3ad27cab03027', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6de1f3cac02c4bcdb05dd2337176033d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 10:46:27.498589 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d14c779-df73-4df8-99fc-6ccff12a0df5 service neutron] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:46:27.498866 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d14c779-df73-4df8-99fc-6ccff12a0df5 service neutron] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:46:27.499166 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d14c779-df73-4df8-99fc-6ccff12a0df5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:46:27.500760 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-6d14c779-df73-4df8-99fc-6ccff12a0df5 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 10:46:27.607517 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d14c779-df73-4df8-99fc-6ccff12a0df5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 10:46:27.608931 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d14c779-df73-4df8-99fc-6ccff12a0df5 service neutron] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 10:46:27.627111 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 648/1316] 10.222.0.25 () {60 vars in 1365 bytes} [Sun Aug 7 10:46:27 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 318 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 07 10:46:30.185009 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17e3e6bb-91df-4d92-8180-55add4948356 None None] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:46:30.185575 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17e3e6bb-91df-4d92-8180-55add4948356 None None] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:46:30.185575 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17e3e6bb-91df-4d92-8180-55add4948356 None None] PATH_INFO: `/` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:46:30.186898 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 669/1317] 10.222.0.25 () {54 vars in 927 bytes} [Sun Aug 7 10:46:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 07 10:46:30.192742 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e37b524-dba5-4bf5-bb94-b016ecc2df30 None None] REQUEST_METHOD: `POST` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:46:30.193167 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e37b524-dba5-4bf5-bb94-b016ecc2df30 None None] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:46:30.193262 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e37b524-dba5-4bf5-bb94-b016ecc2df30 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:46:30.258616 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.auth.core [None req-1e37b524-dba5-4bf5-bb94-b016ecc2df30 None None] MFA Rules not processed for user `cfdc7e8b78b84c6aa44d115a396152bd`. Rule list: `[]` (Enabled: `True`). {{(pid=80143) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 10:46:30.259857 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-1e37b524-dba5-4bf5-bb94-b016ecc2df30 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 10:46:30.310985 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=100 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 10:46:30.355090 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 649/1318] 10.222.0.25 () {58 vars in 1024 bytes} [Sun Aug 7 10:46:30 2022] POST /identity/v3/auth/tokens => generated 3243 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 07 10:46:30.395298 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17e3e6bb-91df-4d92-8180-55add4948356 None None] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 10:46:30.396833 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.fernet_utils [None req-17e3e6bb-91df-4d92-8180-55add4948356 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80144) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 10:46:30.497577 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17e3e6bb-91df-4d92-8180-55add4948356 None None] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 10:46:30.500290 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=100 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 10:46:30.569118 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df935584-1ec7-4a5a-a22f-75c810df2fea service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be1283c9bf47424c9aa3ad27cab03027', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6de1f3cac02c4bcdb05dd2337176033d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 10:46:30.569724 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df935584-1ec7-4a5a-a22f-75c810df2fea service neutron] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:46:30.570012 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df935584-1ec7-4a5a-a22f-75c810df2fea service neutron] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:46:30.570286 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df935584-1ec7-4a5a-a22f-75c810df2fea service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:46:30.571908 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.fernet_utils [None req-df935584-1ec7-4a5a-a22f-75c810df2fea service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80144) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 10:46:30.679093 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df935584-1ec7-4a5a-a22f-75c810df2fea service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 10:46:30.680519 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df935584-1ec7-4a5a-a22f-75c810df2fea service neutron] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 10:46:30.698616 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 670/1319] 10.222.0.25 () {60 vars in 1365 bytes} [Sun Aug 7 10:46:30 2022] GET /identity/v3/auth/tokens => generated 3243 bytes in 304 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 07 10:46:33.412287 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb9719cd-c075-4b61-a604-7feb3124a110 None None] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:46:33.412840 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb9719cd-c075-4b61-a604-7feb3124a110 None None] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:46:33.412840 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb9719cd-c075-4b61-a604-7feb3124a110 None None] PATH_INFO: `/` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:46:33.414214 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 650/1320] 10.222.0.25 () {54 vars in 927 bytes} [Sun Aug 7 10:46:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 07 10:46:33.419616 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-235b8ce7-c3a9-4721-a030-b10b5671e271 None None] REQUEST_METHOD: `POST` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:46:33.420040 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-235b8ce7-c3a9-4721-a030-b10b5671e271 None None] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:46:33.420599 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-235b8ce7-c3a9-4721-a030-b10b5671e271 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:46:33.460745 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=100 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 10:46:33.485099 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.auth.core [None req-235b8ce7-c3a9-4721-a030-b10b5671e271 None None] MFA Rules not processed for user `bac4e6b76fe64217ac8fa44e71cffe6a`. Rule list: `[]` (Enabled: `True`). {{(pid=80144) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 10:46:33.486328 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.fernet_utils [None req-235b8ce7-c3a9-4721-a030-b10b5671e271 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80144) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 10:46:33.583275 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 671/1321] 10.222.0.25 () {58 vars in 1024 bytes} [Sun Aug 7 10:46:33 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 07 10:46:33.623171 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb9719cd-c075-4b61-a604-7feb3124a110 None None] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 10:46:33.624742 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-cb9719cd-c075-4b61-a604-7feb3124a110 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 10:46:33.726108 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb9719cd-c075-4b61-a604-7feb3124a110 None None] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 10:46:33.797285 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9f1ffba-01ef-4174-b87b-0fb8a935044e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be1283c9bf47424c9aa3ad27cab03027', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6de1f3cac02c4bcdb05dd2337176033d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 10:46:33.797894 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9f1ffba-01ef-4174-b87b-0fb8a935044e service neutron] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:46:33.798174 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9f1ffba-01ef-4174-b87b-0fb8a935044e service neutron] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:46:33.798444 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9f1ffba-01ef-4174-b87b-0fb8a935044e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:46:33.800712 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-e9f1ffba-01ef-4174-b87b-0fb8a935044e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 10:46:33.807971 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=100 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 10:46:33.910142 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9f1ffba-01ef-4174-b87b-0fb8a935044e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 10:46:33.911626 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9f1ffba-01ef-4174-b87b-0fb8a935044e service neutron] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 10:46:33.929779 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 651/1322] 10.222.0.25 () {60 vars in 1365 bytes} [Sun Aug 7 10:46:33 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 307 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 07 10:46:36.654386 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c9485d7-9004-45fe-9fd0-f70d69c05c22 None None] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:46:36.655026 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c9485d7-9004-45fe-9fd0-f70d69c05c22 None None] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:46:36.655026 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c9485d7-9004-45fe-9fd0-f70d69c05c22 None None] PATH_INFO: `/` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:46:36.656317 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 672/1323] 10.222.0.25 () {54 vars in 927 bytes} [Sun Aug 7 10:46:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 07 10:46:36.662325 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b4d5d01-ef6c-43f8-aee8-b6d28f19f791 None None] REQUEST_METHOD: `POST` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:46:36.662741 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b4d5d01-ef6c-43f8-aee8-b6d28f19f791 None None] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:46:36.662887 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b4d5d01-ef6c-43f8-aee8-b6d28f19f791 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:46:36.728560 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.auth.core [None req-9b4d5d01-ef6c-43f8-aee8-b6d28f19f791 None None] MFA Rules not processed for user `bac4e6b76fe64217ac8fa44e71cffe6a`. Rule list: `[]` (Enabled: `True`). {{(pid=80143) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 10:46:36.729685 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-9b4d5d01-ef6c-43f8-aee8-b6d28f19f791 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 10:46:36.792094 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=100 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 10:46:36.836611 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 652/1324] 10.222.0.25 () {58 vars in 1024 bytes} [Sun Aug 7 10:46:36 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 07 10:46:36.874749 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c9485d7-9004-45fe-9fd0-f70d69c05c22 None None] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 10:46:36.876392 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.fernet_utils [None req-5c9485d7-9004-45fe-9fd0-f70d69c05c22 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80144) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 10:46:36.991221 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c9485d7-9004-45fe-9fd0-f70d69c05c22 None None] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 10:46:37.004108 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=100 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 10:46:37.065665 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a77f0c2-ec4b-4a73-aa99-7019206ef610 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be1283c9bf47424c9aa3ad27cab03027', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6de1f3cac02c4bcdb05dd2337176033d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 10:46:37.066269 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a77f0c2-ec4b-4a73-aa99-7019206ef610 service neutron] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:46:37.066551 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a77f0c2-ec4b-4a73-aa99-7019206ef610 service neutron] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:46:37.066828 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a77f0c2-ec4b-4a73-aa99-7019206ef610 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:46:37.068470 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.fernet_utils [None req-3a77f0c2-ec4b-4a73-aa99-7019206ef610 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80144) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 10:46:37.178279 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a77f0c2-ec4b-4a73-aa99-7019206ef610 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 10:46:37.179746 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a77f0c2-ec4b-4a73-aa99-7019206ef610 service neutron] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 10:46:37.198366 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 673/1325] 10.222.0.25 () {60 vars in 1365 bytes} [Sun Aug 7 10:46:36 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 324 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 07 10:46:39.315439 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7548c2f-d676-496a-9a43-5b7fa90512ce None None] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:46:39.315949 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7548c2f-d676-496a-9a43-5b7fa90512ce None None] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:46:39.315949 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7548c2f-d676-496a-9a43-5b7fa90512ce None None] PATH_INFO: `/` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:46:39.317273 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 653/1326] 10.222.0.25 () {54 vars in 927 bytes} [Sun Aug 7 10:46:39 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 07 10:46:39.322659 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f01ff869-20c9-45c0-8ea9-c2c49b357593 None None] REQUEST_METHOD: `POST` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:46:39.323148 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f01ff869-20c9-45c0-8ea9-c2c49b357593 None None] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:46:39.323256 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f01ff869-20c9-45c0-8ea9-c2c49b357593 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:46:39.372142 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=100 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 10:46:39.390736 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.auth.core [None req-f01ff869-20c9-45c0-8ea9-c2c49b357593 None None] MFA Rules not processed for user `bac4e6b76fe64217ac8fa44e71cffe6a`. Rule list: `[]` (Enabled: `True`). {{(pid=80144) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 10:46:39.391965 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.fernet_utils [None req-f01ff869-20c9-45c0-8ea9-c2c49b357593 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80144) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 10:46:39.489439 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 674/1327] 10.222.0.25 () {58 vars in 1024 bytes} [Sun Aug 7 10:46:39 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 07 10:46:39.527322 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7548c2f-d676-496a-9a43-5b7fa90512ce None None] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 10:46:39.528863 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-b7548c2f-d676-496a-9a43-5b7fa90512ce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 10:46:39.635499 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7548c2f-d676-496a-9a43-5b7fa90512ce None None] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 10:46:39.712946 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-434c3175-c00d-48a1-beb4-33ad8bd985a8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be1283c9bf47424c9aa3ad27cab03027', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6de1f3cac02c4bcdb05dd2337176033d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 10:46:39.713554 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-434c3175-c00d-48a1-beb4-33ad8bd985a8 service neutron] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:46:39.713834 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-434c3175-c00d-48a1-beb4-33ad8bd985a8 service neutron] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:46:39.714107 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-434c3175-c00d-48a1-beb4-33ad8bd985a8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:46:39.715750 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-434c3175-c00d-48a1-beb4-33ad8bd985a8 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 10:46:39.733975 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=100 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 10:46:39.826960 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-434c3175-c00d-48a1-beb4-33ad8bd985a8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 10:46:39.828395 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-434c3175-c00d-48a1-beb4-33ad8bd985a8 service neutron] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 10:46:39.846418 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 654/1328] 10.222.0.25 () {60 vars in 1365 bytes} [Sun Aug 7 10:46:39 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 319 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 07 10:46:41.903375 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7c5114d-cb13-4b31-8ecf-2547c65b2fec None None] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:46:41.903903 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7c5114d-cb13-4b31-8ecf-2547c65b2fec None None] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:46:41.903903 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7c5114d-cb13-4b31-8ecf-2547c65b2fec None None] PATH_INFO: `/` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:46:41.905244 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 675/1329] 10.222.0.25 () {54 vars in 927 bytes} [Sun Aug 7 10:46:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 07 10:46:41.910357 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2267a9d-de8d-4e40-bfbb-103fa81219a0 None None] REQUEST_METHOD: `POST` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:46:41.910751 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2267a9d-de8d-4e40-bfbb-103fa81219a0 None None] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:46:41.910902 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2267a9d-de8d-4e40-bfbb-103fa81219a0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:46:41.977317 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.auth.core [None req-f2267a9d-de8d-4e40-bfbb-103fa81219a0 None None] MFA Rules not processed for user `cfdc7e8b78b84c6aa44d115a396152bd`. Rule list: `[]` (Enabled: `True`). {{(pid=80143) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 10:46:41.978522 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-f2267a9d-de8d-4e40-bfbb-103fa81219a0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 10:46:42.045264 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=100 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 10:46:42.078629 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 655/1330] 10.222.0.25 () {58 vars in 1024 bytes} [Sun Aug 7 10:46:41 2022] POST /identity/v3/auth/tokens => generated 3243 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 07 10:46:42.117520 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7c5114d-cb13-4b31-8ecf-2547c65b2fec None None] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 10:46:42.119123 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.fernet_utils [None req-d7c5114d-cb13-4b31-8ecf-2547c65b2fec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80144) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 10:46:42.224408 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7c5114d-cb13-4b31-8ecf-2547c65b2fec None None] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 10:46:42.244308 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=100 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 10:46:42.300247 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a62516be-4f49-42bd-8fa8-560d30aa8e4e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be1283c9bf47424c9aa3ad27cab03027', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6de1f3cac02c4bcdb05dd2337176033d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 10:46:42.300763 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a62516be-4f49-42bd-8fa8-560d30aa8e4e service neutron] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:46:42.301044 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a62516be-4f49-42bd-8fa8-560d30aa8e4e service neutron] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:46:42.301323 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a62516be-4f49-42bd-8fa8-560d30aa8e4e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:46:42.302964 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.fernet_utils [None req-a62516be-4f49-42bd-8fa8-560d30aa8e4e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80144) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 10:46:42.414938 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a62516be-4f49-42bd-8fa8-560d30aa8e4e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 10:46:42.416383 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a62516be-4f49-42bd-8fa8-560d30aa8e4e service neutron] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 10:46:42.436313 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 676/1331] 10.222.0.25 () {60 vars in 1365 bytes} [Sun Aug 7 10:46:42 2022] GET /identity/v3/auth/tokens => generated 3243 bytes in 319 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 07 10:46:44.669883 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d92eac99-e573-412b-8c43-934314a46ea6 None None] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:46:44.670387 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d92eac99-e573-412b-8c43-934314a46ea6 None None] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:46:44.670387 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d92eac99-e573-412b-8c43-934314a46ea6 None None] PATH_INFO: `/` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:46:44.671729 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 656/1332] 10.222.0.25 () {54 vars in 927 bytes} [Sun Aug 7 10:46:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 07 10:46:44.678136 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72fd3794-e7dd-491d-9ca7-82367a96d357 None None] REQUEST_METHOD: `POST` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:46:44.678558 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72fd3794-e7dd-491d-9ca7-82367a96d357 None None] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:46:44.678683 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72fd3794-e7dd-491d-9ca7-82367a96d357 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:46:44.740420 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=100 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 10:46:44.743234 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.auth.core [None req-72fd3794-e7dd-491d-9ca7-82367a96d357 None None] MFA Rules not processed for user `bac4e6b76fe64217ac8fa44e71cffe6a`. Rule list: `[]` (Enabled: `True`). {{(pid=80144) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 10:46:44.745039 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.fernet_utils [None req-72fd3794-e7dd-491d-9ca7-82367a96d357 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80144) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 10:46:44.839427 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 677/1333] 10.222.0.25 () {58 vars in 1024 bytes} [Sun Aug 7 10:46:44 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 07 10:46:44.876118 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d92eac99-e573-412b-8c43-934314a46ea6 None None] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 10:46:44.877890 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-d92eac99-e573-412b-8c43-934314a46ea6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 10:46:44.979127 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d92eac99-e573-412b-8c43-934314a46ea6 None None] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 10:46:45.049926 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26a7ea2d-0e9a-4bd9-b469-56dd66ae97c0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be1283c9bf47424c9aa3ad27cab03027', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6de1f3cac02c4bcdb05dd2337176033d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 10:46:45.050520 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26a7ea2d-0e9a-4bd9-b469-56dd66ae97c0 service neutron] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:46:45.050795 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26a7ea2d-0e9a-4bd9-b469-56dd66ae97c0 service neutron] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:46:45.051100 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26a7ea2d-0e9a-4bd9-b469-56dd66ae97c0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:46:45.052676 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-26a7ea2d-0e9a-4bd9-b469-56dd66ae97c0 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 10:46:45.078848 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=100 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 10:46:45.162044 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26a7ea2d-0e9a-4bd9-b469-56dd66ae97c0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 10:46:45.163513 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26a7ea2d-0e9a-4bd9-b469-56dd66ae97c0 service neutron] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 10:46:45.181872 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 657/1334] 10.222.0.25 () {60 vars in 1365 bytes} [Sun Aug 7 10:46:44 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 306 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 07 10:46:47.217760 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5154a6b-6eb4-4772-b69c-f92382153aec None None] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:46:47.218326 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5154a6b-6eb4-4772-b69c-f92382153aec None None] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:46:47.218326 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5154a6b-6eb4-4772-b69c-f92382153aec None None] PATH_INFO: `/` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:46:47.219963 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 678/1335] 10.222.0.25 () {54 vars in 927 bytes} [Sun Aug 7 10:46:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 07 10:46:47.225107 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f817663-bc12-42a0-b17b-76b98d49cd2a None None] REQUEST_METHOD: `POST` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:46:47.225534 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f817663-bc12-42a0-b17b-76b98d49cd2a None None] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:46:47.225687 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f817663-bc12-42a0-b17b-76b98d49cd2a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:46:47.290469 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.auth.core [None req-5f817663-bc12-42a0-b17b-76b98d49cd2a None None] MFA Rules not processed for user `bac4e6b76fe64217ac8fa44e71cffe6a`. Rule list: `[]` (Enabled: `True`). {{(pid=80143) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 10:46:47.291723 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-5f817663-bc12-42a0-b17b-76b98d49cd2a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 10:46:47.364229 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=100 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 10:46:47.387555 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 658/1336] 10.222.0.25 () {58 vars in 1024 bytes} [Sun Aug 7 10:46:47 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 07 10:46:47.428033 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5154a6b-6eb4-4772-b69c-f92382153aec None None] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 10:46:47.429579 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.fernet_utils [None req-a5154a6b-6eb4-4772-b69c-f92382153aec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80144) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 10:46:47.534777 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5154a6b-6eb4-4772-b69c-f92382153aec None None] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 10:46:47.561850 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=100 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 10:46:47.606207 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-096bd935-9f6e-471a-8923-84d3f4615504 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be1283c9bf47424c9aa3ad27cab03027', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6de1f3cac02c4bcdb05dd2337176033d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 10:46:47.606810 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-096bd935-9f6e-471a-8923-84d3f4615504 service neutron] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:46:47.607123 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-096bd935-9f6e-471a-8923-84d3f4615504 service neutron] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:46:47.607396 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-096bd935-9f6e-471a-8923-84d3f4615504 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:46:47.608971 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.fernet_utils [None req-096bd935-9f6e-471a-8923-84d3f4615504 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80144) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 10:46:47.715732 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-096bd935-9f6e-471a-8923-84d3f4615504 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 10:46:47.717152 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-096bd935-9f6e-471a-8923-84d3f4615504 service neutron] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 10:46:47.739165 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 679/1337] 10.222.0.25 () {60 vars in 1365 bytes} [Sun Aug 7 10:46:47 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 312 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 07 10:46:50.070946 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c0f7904-fe05-40da-888a-01e068890fa2 None None] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:46:50.071590 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c0f7904-fe05-40da-888a-01e068890fa2 None None] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:46:50.071590 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c0f7904-fe05-40da-888a-01e068890fa2 None None] PATH_INFO: `/` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:46:50.072822 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 659/1338] 10.222.0.25 () {54 vars in 927 bytes} [Sun Aug 7 10:46:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 07 10:46:50.077908 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50d61e66-28d3-42d3-a71d-7450bfff6afa None None] REQUEST_METHOD: `POST` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:46:50.078346 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50d61e66-28d3-42d3-a71d-7450bfff6afa None None] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:46:50.078455 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50d61e66-28d3-42d3-a71d-7450bfff6afa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:46:50.145147 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.auth.core [None req-50d61e66-28d3-42d3-a71d-7450bfff6afa None None] MFA Rules not processed for user `bac4e6b76fe64217ac8fa44e71cffe6a`. Rule list: `[]` (Enabled: `True`). {{(pid=80144) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 10:46:50.146342 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.fernet_utils [None req-50d61e66-28d3-42d3-a71d-7450bfff6afa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80144) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 10:46:50.154547 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=100 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 10:46:50.244371 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 680/1339] 10.222.0.25 () {58 vars in 1024 bytes} [Sun Aug 7 10:46:50 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 07 10:46:50.280889 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c0f7904-fe05-40da-888a-01e068890fa2 None None] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 10:46:50.282422 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-8c0f7904-fe05-40da-888a-01e068890fa2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 10:46:50.384614 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c0f7904-fe05-40da-888a-01e068890fa2 None None] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 10:46:50.456320 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9da8ce2a-38e1-4ce4-926d-7fb50b402a22 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be1283c9bf47424c9aa3ad27cab03027', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6de1f3cac02c4bcdb05dd2337176033d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 10:46:50.456919 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9da8ce2a-38e1-4ce4-926d-7fb50b402a22 service neutron] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:46:50.457196 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9da8ce2a-38e1-4ce4-926d-7fb50b402a22 service neutron] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:46:50.457467 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9da8ce2a-38e1-4ce4-926d-7fb50b402a22 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:46:50.459105 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-9da8ce2a-38e1-4ce4-926d-7fb50b402a22 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 10:46:50.495107 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=100 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 10:46:50.569222 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9da8ce2a-38e1-4ce4-926d-7fb50b402a22 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 10:46:50.570682 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9da8ce2a-38e1-4ce4-926d-7fb50b402a22 service neutron] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 10:46:50.588292 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 660/1340] 10.222.0.25 () {60 vars in 1365 bytes} [Sun Aug 7 10:46:50 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 308 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 07 10:46:54.539234 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1cfc25b-9075-4b8f-a235-62ad777fd6da None None] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:46:54.539766 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1cfc25b-9075-4b8f-a235-62ad777fd6da None None] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:46:54.539766 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1cfc25b-9075-4b8f-a235-62ad777fd6da None None] PATH_INFO: `/` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:46:54.541136 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 681/1341] 10.222.0.25 () {54 vars in 927 bytes} [Sun Aug 7 10:46:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 07 10:46:54.546352 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1f2f925-4dd8-4299-874b-052b54392a62 None None] REQUEST_METHOD: `POST` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:46:54.546771 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1f2f925-4dd8-4299-874b-052b54392a62 None None] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:46:54.546915 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1f2f925-4dd8-4299-874b-052b54392a62 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:46:54.614612 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.auth.core [None req-a1f2f925-4dd8-4299-874b-052b54392a62 None None] MFA Rules not processed for user `bac4e6b76fe64217ac8fa44e71cffe6a`. Rule list: `[]` (Enabled: `True`). {{(pid=80143) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 10:46:54.616012 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-a1f2f925-4dd8-4299-874b-052b54392a62 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 10:46:54.702527 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=100 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 10:46:54.714849 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 661/1342] 10.222.0.25 () {58 vars in 1024 bytes} [Sun Aug 7 10:46:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 07 10:46:54.754946 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1cfc25b-9075-4b8f-a235-62ad777fd6da None None] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 10:46:54.756599 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.fernet_utils [None req-a1cfc25b-9075-4b8f-a235-62ad777fd6da None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80144) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 10:46:54.860848 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1cfc25b-9075-4b8f-a235-62ad777fd6da None None] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 10:46:54.902379 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=100 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 10:46:54.938972 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f8d33d2-8a56-459d-be5a-3faa0e5b3db8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be1283c9bf47424c9aa3ad27cab03027', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6de1f3cac02c4bcdb05dd2337176033d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 10:46:54.940721 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f8d33d2-8a56-459d-be5a-3faa0e5b3db8 service neutron] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:46:54.940998 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f8d33d2-8a56-459d-be5a-3faa0e5b3db8 service neutron] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:46:54.941270 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f8d33d2-8a56-459d-be5a-3faa0e5b3db8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:46:54.942880 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.fernet_utils [None req-1f8d33d2-8a56-459d-be5a-3faa0e5b3db8 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80144) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 10:46:55.054129 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f8d33d2-8a56-459d-be5a-3faa0e5b3db8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 10:46:55.055602 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f8d33d2-8a56-459d-be5a-3faa0e5b3db8 service neutron] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 10:46:55.073718 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 682/1343] 10.222.0.25 () {60 vars in 1365 bytes} [Sun Aug 7 10:46:54 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 319 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 07 10:46:57.904435 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee871020-f9ef-4829-b39a-a52cf5987596 None None] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:46:57.904986 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee871020-f9ef-4829-b39a-a52cf5987596 None None] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:46:57.904986 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee871020-f9ef-4829-b39a-a52cf5987596 None None] PATH_INFO: `/` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:46:57.906314 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 662/1344] 10.222.0.25 () {54 vars in 927 bytes} [Sun Aug 7 10:46:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 07 10:46:57.911631 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2218b912-7e62-4ddc-baf9-df03e17b0757 None None] REQUEST_METHOD: `POST` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:46:57.912069 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2218b912-7e62-4ddc-baf9-df03e17b0757 None None] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:46:57.912261 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2218b912-7e62-4ddc-baf9-df03e17b0757 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:46:57.977836 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.auth.core [None req-2218b912-7e62-4ddc-baf9-df03e17b0757 None None] MFA Rules not processed for user `bac4e6b76fe64217ac8fa44e71cffe6a`. Rule list: `[]` (Enabled: `True`). {{(pid=80144) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 10:46:57.978939 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.fernet_utils [None req-2218b912-7e62-4ddc-baf9-df03e17b0757 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80144) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 10:46:57.997762 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=100 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 10:46:58.077851 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 683/1345] 10.222.0.25 () {58 vars in 1024 bytes} [Sun Aug 7 10:46:57 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 07 10:46:58.117009 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee871020-f9ef-4829-b39a-a52cf5987596 None None] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 10:46:58.118653 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-ee871020-f9ef-4829-b39a-a52cf5987596 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 10:46:58.225516 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee871020-f9ef-4829-b39a-a52cf5987596 None None] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 10:46:58.301556 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32cbc96a-f70c-42c9-94d0-07db196dacbc service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be1283c9bf47424c9aa3ad27cab03027', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6de1f3cac02c4bcdb05dd2337176033d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 10:46:58.302173 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32cbc96a-f70c-42c9-94d0-07db196dacbc service neutron] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:46:58.302450 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32cbc96a-f70c-42c9-94d0-07db196dacbc service neutron] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:46:58.302720 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32cbc96a-f70c-42c9-94d0-07db196dacbc service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:46:58.304343 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-32cbc96a-f70c-42c9-94d0-07db196dacbc service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 10:46:58.347470 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=100 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 10:46:58.422001 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32cbc96a-f70c-42c9-94d0-07db196dacbc service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 10:46:58.423476 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32cbc96a-f70c-42c9-94d0-07db196dacbc service neutron] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 10:46:58.441738 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 663/1346] 10.222.0.25 () {60 vars in 1365 bytes} [Sun Aug 7 10:46:58 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 325 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 07 10:47:02.477257 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c6c4f37-ec1e-4ee6-9e00-09b491197bd7 None None] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:47:02.477785 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c6c4f37-ec1e-4ee6-9e00-09b491197bd7 None None] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:47:02.477785 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c6c4f37-ec1e-4ee6-9e00-09b491197bd7 None None] PATH_INFO: `/` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:47:02.479302 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 684/1347] 10.222.0.25 () {54 vars in 927 bytes} [Sun Aug 7 10:47:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 07 10:47:02.484660 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7788481f-7a1d-4767-8648-fae866cf31e4 None None] REQUEST_METHOD: `POST` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:47:02.485127 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7788481f-7a1d-4767-8648-fae866cf31e4 None None] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:47:02.485239 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7788481f-7a1d-4767-8648-fae866cf31e4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:47:02.559685 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.auth.core [None req-7788481f-7a1d-4767-8648-fae866cf31e4 None None] MFA Rules not processed for user `bac4e6b76fe64217ac8fa44e71cffe6a`. Rule list: `[]` (Enabled: `True`). {{(pid=80143) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 10:47:02.560905 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-7788481f-7a1d-4767-8648-fae866cf31e4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 10:47:02.662210 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=100 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 10:47:02.664961 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 664/1348] 10.222.0.25 () {58 vars in 1024 bytes} [Sun Aug 7 10:47:02 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 181 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 07 10:47:02.703536 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c6c4f37-ec1e-4ee6-9e00-09b491197bd7 None None] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 10:47:02.705754 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.fernet_utils [None req-8c6c4f37-ec1e-4ee6-9e00-09b491197bd7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80144) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 10:47:02.819029 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c6c4f37-ec1e-4ee6-9e00-09b491197bd7 None None] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 10:47:02.870345 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=100 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 10:47:02.903139 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3971bbc6-cc45-4eef-94bb-8bce42236edc service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be1283c9bf47424c9aa3ad27cab03027', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6de1f3cac02c4bcdb05dd2337176033d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 10:47:02.903847 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3971bbc6-cc45-4eef-94bb-8bce42236edc service neutron] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:47:02.904133 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3971bbc6-cc45-4eef-94bb-8bce42236edc service neutron] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:47:02.904403 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3971bbc6-cc45-4eef-94bb-8bce42236edc service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:47:02.906133 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.fernet_utils [None req-3971bbc6-cc45-4eef-94bb-8bce42236edc service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80144) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 10:47:03.141508 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3971bbc6-cc45-4eef-94bb-8bce42236edc service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 10:47:03.143403 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3971bbc6-cc45-4eef-94bb-8bce42236edc service neutron] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 10:47:03.161680 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 685/1349] 10.222.0.25 () {60 vars in 1365 bytes} [Sun Aug 7 10:47:02 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 459 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 07 10:47:08.110276 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8eb9d79-8a95-488d-9922-64941937b986 None None] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:47:08.111771 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8eb9d79-8a95-488d-9922-64941937b986 None None] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:47:08.112206 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8eb9d79-8a95-488d-9922-64941937b986 None None] PATH_INFO: `/` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:47:08.114021 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 665/1350] 10.222.0.25 () {54 vars in 921 bytes} [Sun Aug 7 10:47:08 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 07 10:47:08.120347 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f61f7f76-dd25-4bff-97f9-aaf4d592a9fb None None] REQUEST_METHOD: `POST` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:47:08.120782 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f61f7f76-dd25-4bff-97f9-aaf4d592a9fb None None] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:47:08.120907 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f61f7f76-dd25-4bff-97f9-aaf4d592a9fb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:47:08.218148 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.auth.core [None req-f61f7f76-dd25-4bff-97f9-aaf4d592a9fb None None] MFA Rules not processed for user `6ee2e4c08af04c219ed395d7ca7085a6`. Rule list: `[]` (Enabled: `True`). {{(pid=80144) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 10:47:08.219570 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.fernet_utils [None req-f61f7f76-dd25-4bff-97f9-aaf4d592a9fb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80144) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 10:47:08.270627 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=100 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 10:47:08.341665 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 686/1351] 10.222.0.25 () {58 vars in 1018 bytes} [Sun Aug 7 10:47:08 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 222 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 07 10:47:12.835570 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed43c566-653a-447f-93f2-0ef3564d958f None None] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:47:12.836134 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed43c566-653a-447f-93f2-0ef3564d958f None None] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:47:12.836134 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed43c566-653a-447f-93f2-0ef3564d958f None None] PATH_INFO: `/` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:47:12.837753 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 666/1352] 10.222.0.25 () {54 vars in 921 bytes} [Sun Aug 7 10:47:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 07 10:47:12.843160 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6df442b1-c168-4be5-a91f-e8f7b8da6c1e None None] REQUEST_METHOD: `POST` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:47:12.843986 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6df442b1-c168-4be5-a91f-e8f7b8da6c1e None None] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:47:12.844339 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6df442b1-c168-4be5-a91f-e8f7b8da6c1e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:47:12.925283 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.auth.core [None req-6df442b1-c168-4be5-a91f-e8f7b8da6c1e None None] MFA Rules not processed for user `6ee2e4c08af04c219ed395d7ca7085a6`. Rule list: `[]` (Enabled: `True`). {{(pid=80144) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 10:47:12.926476 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.fernet_utils [None req-6df442b1-c168-4be5-a91f-e8f7b8da6c1e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80144) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 10:47:13.002669 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=100 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 10:47:13.041480 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 687/1353] 10.222.0.25 () {58 vars in 1018 bytes} [Sun Aug 7 10:47:12 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 199 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 07 10:47:14.053190 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-559ae411-2c4a-43b1-a191-914029922369 None None] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:47:14.053698 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-559ae411-2c4a-43b1-a191-914029922369 None None] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:47:14.053780 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-559ae411-2c4a-43b1-a191-914029922369 None None] PATH_INFO: `/` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:47:14.055275 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 667/1354] 10.222.0.25 () {54 vars in 921 bytes} [Sun Aug 7 10:47:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 07 10:47:14.060337 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2992363a-b3e1-467f-a1f6-06de77a60356 None None] REQUEST_METHOD: `POST` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:47:14.060795 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2992363a-b3e1-467f-a1f6-06de77a60356 None None] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:47:14.060918 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2992363a-b3e1-467f-a1f6-06de77a60356 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:47:14.138579 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.auth.core [None req-2992363a-b3e1-467f-a1f6-06de77a60356 None None] MFA Rules not processed for user `6ee2e4c08af04c219ed395d7ca7085a6`. Rule list: `[]` (Enabled: `True`). {{(pid=80144) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 10:47:14.139730 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.fernet_utils [None req-2992363a-b3e1-467f-a1f6-06de77a60356 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80144) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 10:47:14.247452 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 688/1355] 10.222.0.25 () {58 vars in 1018 bytes} [Sun Aug 7 10:47:14 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 188 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 07 10:47:17.655598 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73d2c137-cd24-4434-8c80-fac5d9996ed9 None None] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:47:17.656423 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73d2c137-cd24-4434-8c80-fac5d9996ed9 None None] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:47:17.656852 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73d2c137-cd24-4434-8c80-fac5d9996ed9 None None] PATH_INFO: `/` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:47:17.666412 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d97fffc7-f88a-4910-9c8c-03659533050e None None] REQUEST_METHOD: `POST` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:47:17.666908 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d97fffc7-f88a-4910-9c8c-03659533050e None None] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:47:17.667772 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d97fffc7-f88a-4910-9c8c-03659533050e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:47:17.672335 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 668/1356] 10.222.0.25 () {54 vars in 927 bytes} [Sun Aug 7 10:47:17 2022] GET /identity => generated 270 bytes in 10 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 07 10:47:17.711588 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=100 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 10:47:17.753599 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.auth.core [None req-d97fffc7-f88a-4910-9c8c-03659533050e None None] MFA Rules not processed for user `bac4e6b76fe64217ac8fa44e71cffe6a`. Rule list: `[]` (Enabled: `True`). {{(pid=80144) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 10:47:17.754804 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.fernet_utils [None req-d97fffc7-f88a-4910-9c8c-03659533050e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80144) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 10:47:17.857238 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 689/1357] 10.222.0.25 () {58 vars in 1024 bytes} [Sun Aug 7 10:47:17 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 192 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 07 10:47:17.914588 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f480406f-1066-4afc-bf88-4ef6bb6c9015 None None] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:47:17.915381 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f480406f-1066-4afc-bf88-4ef6bb6c9015 None None] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:47:17.916217 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f480406f-1066-4afc-bf88-4ef6bb6c9015 None None] PATH_INFO: `/` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:47:17.919430 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 669/1358] 10.222.0.25 () {54 vars in 962 bytes} [Sun Aug 7 10:47:17 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 07 10:47:17.927803 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb4c4f83-5449-4940-b3c8-315a1b703aeb None None] REQUEST_METHOD: `POST` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:47:17.928386 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb4c4f83-5449-4940-b3c8-315a1b703aeb None None] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:47:17.928931 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb4c4f83-5449-4940-b3c8-315a1b703aeb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:47:18.019716 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.auth.core [None req-bb4c4f83-5449-4940-b3c8-315a1b703aeb None None] MFA Rules not processed for user `31d16d1b96104b4eb690398432b9829e`. Rule list: `[]` (Enabled: `True`). {{(pid=80144) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 10:47:18.021233 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.fernet_utils [None req-bb4c4f83-5449-4940-b3c8-315a1b703aeb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80144) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 10:47:18.078475 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=100 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 10:47:18.153049 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 690/1359] 10.222.0.25 () {58 vars in 1059 bytes} [Sun Aug 7 10:47:17 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 227 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 07 10:47:18.162277 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f480406f-1066-4afc-bf88-4ef6bb6c9015 None None] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 10:47:18.163836 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-f480406f-1066-4afc-bf88-4ef6bb6c9015 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 10:47:18.318735 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f480406f-1066-4afc-bf88-4ef6bb6c9015 None None] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 10:47:18.394753 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=100 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 10:47:18.444833 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15a97a47-0873-415c-a2b9-b612c4d660e3 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31d16d1b96104b4eb690398432b9829e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6de1f3cac02c4bcdb05dd2337176033d', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'reader', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 10:47:18.445602 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15a97a47-0873-415c-a2b9-b612c4d660e3 service nova] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:47:18.445995 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15a97a47-0873-415c-a2b9-b612c4d660e3 service nova] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:47:18.446362 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15a97a47-0873-415c-a2b9-b612c4d660e3 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:47:18.448114 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-15a97a47-0873-415c-a2b9-b612c4d660e3 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 10:47:18.543290 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15a97a47-0873-415c-a2b9-b612c4d660e3 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 10:47:18.545138 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15a97a47-0873-415c-a2b9-b612c4d660e3 service nova] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 10:47:18.564290 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 670/1360] 10.222.0.25 () {60 vars in 1365 bytes} [Sun Aug 7 10:47:18 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 402 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 07 10:47:20.850425 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a211a736-603e-4a3f-9c10-88043eb26d3d None None] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:47:20.851140 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a211a736-603e-4a3f-9c10-88043eb26d3d None None] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:47:20.851140 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a211a736-603e-4a3f-9c10-88043eb26d3d None None] PATH_INFO: `/` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:47:20.852504 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 691/1361] 10.222.0.25 () {54 vars in 927 bytes} [Sun Aug 7 10:47:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 07 10:47:20.857718 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-258184d6-8053-4166-9b92-f7fe314de302 None None] REQUEST_METHOD: `POST` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:47:20.858607 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-258184d6-8053-4166-9b92-f7fe314de302 None None] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:47:20.859210 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-258184d6-8053-4166-9b92-f7fe314de302 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:47:20.933629 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.auth.core [None req-258184d6-8053-4166-9b92-f7fe314de302 None None] MFA Rules not processed for user `bac4e6b76fe64217ac8fa44e71cffe6a`. Rule list: `[]` (Enabled: `True`). {{(pid=80143) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 10:47:20.934815 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-258184d6-8053-4166-9b92-f7fe314de302 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 10:47:20.976543 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=100 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 10:47:21.051691 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 671/1362] 10.222.0.25 () {58 vars in 1024 bytes} [Sun Aug 7 10:47:20 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 195 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 07 10:47:21.101109 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a211a736-603e-4a3f-9c10-88043eb26d3d None None] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 10:47:21.102652 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.fernet_utils [None req-a211a736-603e-4a3f-9c10-88043eb26d3d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80144) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 10:47:21.268625 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a211a736-603e-4a3f-9c10-88043eb26d3d None None] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 10:47:21.279633 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=100 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 10:47:21.429892 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b911783-a785-4b0e-af2d-e7ce58d077ec service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31d16d1b96104b4eb690398432b9829e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6de1f3cac02c4bcdb05dd2337176033d', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'reader', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 10:47:21.430538 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b911783-a785-4b0e-af2d-e7ce58d077ec service nova] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:47:21.430816 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b911783-a785-4b0e-af2d-e7ce58d077ec service nova] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:47:21.431115 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b911783-a785-4b0e-af2d-e7ce58d077ec service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:47:21.433074 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.fernet_utils [None req-1b911783-a785-4b0e-af2d-e7ce58d077ec service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80144) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 10:47:21.529941 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b911783-a785-4b0e-af2d-e7ce58d077ec service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 10:47:21.531620 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b911783-a785-4b0e-af2d-e7ce58d077ec service nova] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 10:47:21.551338 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 692/1363] 10.222.0.25 () {60 vars in 1365 bytes} [Sun Aug 7 10:47:21 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 450 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 07 10:47:23.976634 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9d74299-b30a-408d-989f-bbf0bed16616 None None] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:47:23.977061 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9d74299-b30a-408d-989f-bbf0bed16616 None None] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:47:23.977168 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9d74299-b30a-408d-989f-bbf0bed16616 None None] PATH_INFO: `/` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:47:23.978516 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 672/1364] 10.222.0.25 () {54 vars in 927 bytes} [Sun Aug 7 10:47:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 07 10:47:23.984034 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0a41ed1-81d3-4364-8a9c-7f1564910575 None None] REQUEST_METHOD: `POST` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:47:23.985387 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0a41ed1-81d3-4364-8a9c-7f1564910575 None None] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:47:23.986457 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0a41ed1-81d3-4364-8a9c-7f1564910575 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:47:24.004203 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=100 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 10:47:24.062746 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.auth.core [None req-b0a41ed1-81d3-4364-8a9c-7f1564910575 None None] MFA Rules not processed for user `bac4e6b76fe64217ac8fa44e71cffe6a`. Rule list: `[]` (Enabled: `True`). {{(pid=80144) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 10:47:24.065399 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.fernet_utils [None req-b0a41ed1-81d3-4364-8a9c-7f1564910575 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80144) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 10:47:24.169988 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 693/1365] 10.222.0.25 () {58 vars in 1024 bytes} [Sun Aug 7 10:47:23 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 187 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 07 10:47:24.215526 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e903333-23ba-4407-aa86-9f00d7692d73 None None] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:47:24.215803 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e903333-23ba-4407-aa86-9f00d7692d73 None None] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:47:24.216314 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e903333-23ba-4407-aa86-9f00d7692d73 None None] PATH_INFO: `/` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:47:24.217646 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 673/1366] 10.222.0.25 () {54 vars in 962 bytes} [Sun Aug 7 10:47:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 07 10:47:24.224245 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af31d280-5d74-48ac-afc1-d6d09a9a23a9 None None] REQUEST_METHOD: `POST` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:47:24.224467 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af31d280-5d74-48ac-afc1-d6d09a9a23a9 None None] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:47:24.224746 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af31d280-5d74-48ac-afc1-d6d09a9a23a9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:47:24.306331 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.auth.core [None req-af31d280-5d74-48ac-afc1-d6d09a9a23a9 None None] MFA Rules not processed for user `31d16d1b96104b4eb690398432b9829e`. Rule list: `[]` (Enabled: `True`). {{(pid=80144) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 10:47:24.307676 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.fernet_utils [None req-af31d280-5d74-48ac-afc1-d6d09a9a23a9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80144) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 10:47:24.345927 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=100 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 10:47:24.445248 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 694/1367] 10.222.0.25 () {58 vars in 1059 bytes} [Sun Aug 7 10:47:24 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 222 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 07 10:47:24.453655 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e903333-23ba-4407-aa86-9f00d7692d73 None None] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 10:47:24.455947 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-3e903333-23ba-4407-aa86-9f00d7692d73 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 10:47:24.612950 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e903333-23ba-4407-aa86-9f00d7692d73 None None] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 10:47:24.634844 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=100 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 10:47:24.732374 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a83d2d3-9d00-476e-9b63-2cf5c08aecc0 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31d16d1b96104b4eb690398432b9829e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6de1f3cac02c4bcdb05dd2337176033d', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'reader', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 10:47:24.732987 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a83d2d3-9d00-476e-9b63-2cf5c08aecc0 service nova] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:47:24.733262 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a83d2d3-9d00-476e-9b63-2cf5c08aecc0 service nova] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:47:24.733532 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a83d2d3-9d00-476e-9b63-2cf5c08aecc0 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:47:24.735803 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-8a83d2d3-9d00-476e-9b63-2cf5c08aecc0 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 10:47:24.830866 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a83d2d3-9d00-476e-9b63-2cf5c08aecc0 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 10:47:24.832541 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a83d2d3-9d00-476e-9b63-2cf5c08aecc0 service nova] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 10:47:24.851278 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 674/1368] 10.222.0.25 () {60 vars in 1365 bytes} [Sun Aug 7 10:47:24 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 398 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 07 10:47:25.804606 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa753b2a-ea67-4a36-a913-aaab8a0c83ce None None] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:47:25.805118 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa753b2a-ea67-4a36-a913-aaab8a0c83ce None None] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:47:25.805118 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa753b2a-ea67-4a36-a913-aaab8a0c83ce None None] PATH_INFO: `/` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:47:25.806636 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 695/1369] 10.222.0.25 () {54 vars in 927 bytes} [Sun Aug 7 10:47:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 07 10:47:25.811610 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c633e27-dc49-4aba-8b76-58b598d6ba84 None None] REQUEST_METHOD: `POST` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:47:25.812024 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c633e27-dc49-4aba-8b76-58b598d6ba84 None None] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:47:25.812194 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c633e27-dc49-4aba-8b76-58b598d6ba84 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:47:25.845440 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=100 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 10:47:25.883080 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.auth.core [None req-5c633e27-dc49-4aba-8b76-58b598d6ba84 None None] MFA Rules not processed for user `bac4e6b76fe64217ac8fa44e71cffe6a`. Rule list: `[]` (Enabled: `True`). {{(pid=80143) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 10:47:25.884513 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-5c633e27-dc49-4aba-8b76-58b598d6ba84 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 10:47:25.987507 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 675/1370] 10.222.0.25 () {58 vars in 1024 bytes} [Sun Aug 7 10:47:25 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 177 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 07 10:47:26.329179 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4c97cde-68bb-4922-89d3-c85d48aafa82 None None] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:47:26.329179 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4c97cde-68bb-4922-89d3-c85d48aafa82 None None] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:47:26.329179 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4c97cde-68bb-4922-89d3-c85d48aafa82 None None] PATH_INFO: `/` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:47:26.331799 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 696/1371] 10.222.0.25 () {54 vars in 964 bytes} [Sun Aug 7 10:47:26 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 07 10:47:26.335328 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24582f9f-f797-4d9f-958a-c5ac31e914b1 None None] REQUEST_METHOD: `POST` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:47:26.335622 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24582f9f-f797-4d9f-958a-c5ac31e914b1 None None] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:47:26.335899 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24582f9f-f797-4d9f-958a-c5ac31e914b1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:47:26.413900 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.auth.core [None req-24582f9f-f797-4d9f-958a-c5ac31e914b1 None None] MFA Rules not processed for user `88dbd6dfd1164994a3e16da553561910`. Rule list: `[]` (Enabled: `True`). {{(pid=80143) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 10:47:26.415163 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-24582f9f-f797-4d9f-958a-c5ac31e914b1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 10:47:26.469294 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=100 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 10:47:26.504532 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 676/1372] 10.222.0.25 () {58 vars in 1061 bytes} [Sun Aug 7 10:47:26 2022] POST /identity/v3/auth/tokens => generated 3120 bytes in 170 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 07 10:47:26.512717 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4c97cde-68bb-4922-89d3-c85d48aafa82 None None] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 10:47:26.514488 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.fernet_utils [None req-a4c97cde-68bb-4922-89d3-c85d48aafa82 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80144) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 10:47:26.628646 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4c97cde-68bb-4922-89d3-c85d48aafa82 None None] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 10:47:26.672638 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=100 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 10:47:26.708203 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5efbbec-54b3-4033-a00b-4be3b2c19fc9 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88dbd6dfd1164994a3e16da553561910', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6de1f3cac02c4bcdb05dd2337176033d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 10:47:26.708965 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5efbbec-54b3-4033-a00b-4be3b2c19fc9 service cinder] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:47:26.709344 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5efbbec-54b3-4033-a00b-4be3b2c19fc9 service cinder] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:47:26.709719 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5efbbec-54b3-4033-a00b-4be3b2c19fc9 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:47:26.711436 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.fernet_utils [None req-f5efbbec-54b3-4033-a00b-4be3b2c19fc9 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80144) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 10:47:26.825604 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5efbbec-54b3-4033-a00b-4be3b2c19fc9 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 10:47:26.827212 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5efbbec-54b3-4033-a00b-4be3b2c19fc9 service cinder] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 10:47:26.847403 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 697/1373] 10.222.0.25 () {60 vars in 1365 bytes} [Sun Aug 7 10:47:26 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 335 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 07 10:47:27.041104 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73e63570-06bd-43ca-a696-7b2929db1be0 None None] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:47:27.041777 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73e63570-06bd-43ca-a696-7b2929db1be0 None None] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:47:27.041777 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73e63570-06bd-43ca-a696-7b2929db1be0 None None] PATH_INFO: `/` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:47:27.043212 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 677/1374] 10.222.0.25 () {54 vars in 927 bytes} [Sun Aug 7 10:47:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 07 10:47:27.050546 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c70801f5-0802-4eea-bf81-ea185dd475d0 None None] REQUEST_METHOD: `POST` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:47:27.050546 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c70801f5-0802-4eea-bf81-ea185dd475d0 None None] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:47:27.050546 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c70801f5-0802-4eea-bf81-ea185dd475d0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:47:27.120215 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.auth.core [None req-c70801f5-0802-4eea-bf81-ea185dd475d0 None None] MFA Rules not processed for user `bac4e6b76fe64217ac8fa44e71cffe6a`. Rule list: `[]` (Enabled: `True`). {{(pid=80144) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 10:47:27.121428 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.fernet_utils [None req-c70801f5-0802-4eea-bf81-ea185dd475d0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80144) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 10:47:27.143896 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=100 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 10:47:27.227477 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 698/1375] 10.222.0.25 () {58 vars in 1024 bytes} [Sun Aug 7 10:47:27 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 176 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 07 10:47:27.271036 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73e63570-06bd-43ca-a696-7b2929db1be0 None None] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 10:47:27.273155 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-73e63570-06bd-43ca-a696-7b2929db1be0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 10:47:27.430803 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73e63570-06bd-43ca-a696-7b2929db1be0 None None] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 10:47:27.483670 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=100 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 10:47:27.548891 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bc8c473-2d8a-40e1-8c1f-7bffe3345071 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31d16d1b96104b4eb690398432b9829e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6de1f3cac02c4bcdb05dd2337176033d', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'reader', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 10:47:27.549582 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bc8c473-2d8a-40e1-8c1f-7bffe3345071 service nova] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:47:27.549926 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bc8c473-2d8a-40e1-8c1f-7bffe3345071 service nova] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:47:27.550280 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bc8c473-2d8a-40e1-8c1f-7bffe3345071 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:47:27.552032 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-5bc8c473-2d8a-40e1-8c1f-7bffe3345071 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 10:47:27.648736 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bc8c473-2d8a-40e1-8c1f-7bffe3345071 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 10:47:27.650364 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bc8c473-2d8a-40e1-8c1f-7bffe3345071 service nova] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 10:47:27.669116 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 678/1376] 10.222.0.25 () {60 vars in 1365 bytes} [Sun Aug 7 10:47:27 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 399 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 07 10:47:29.725247 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb8b8ef5-1979-4652-b946-d58c92e57307 None None] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:47:29.725777 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb8b8ef5-1979-4652-b946-d58c92e57307 None None] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:47:29.725875 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb8b8ef5-1979-4652-b946-d58c92e57307 None None] PATH_INFO: `/` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:47:29.727185 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 699/1377] 10.222.0.25 () {54 vars in 927 bytes} [Sun Aug 7 10:47:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 07 10:47:29.732525 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10f7df59-5ff6-42bd-940d-a06d558a712e None None] REQUEST_METHOD: `POST` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:47:29.732973 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10f7df59-5ff6-42bd-940d-a06d558a712e None None] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:47:29.733090 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10f7df59-5ff6-42bd-940d-a06d558a712e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:47:29.802298 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.auth.core [None req-10f7df59-5ff6-42bd-940d-a06d558a712e None None] MFA Rules not processed for user `bac4e6b76fe64217ac8fa44e71cffe6a`. Rule list: `[]` (Enabled: `True`). {{(pid=80143) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 10:47:29.803529 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-10f7df59-5ff6-42bd-940d-a06d558a712e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 10:47:29.810401 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=100 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 10:47:29.917492 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 679/1378] 10.222.0.25 () {58 vars in 1024 bytes} [Sun Aug 7 10:47:29 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 186 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 07 10:47:29.965964 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb8b8ef5-1979-4652-b946-d58c92e57307 None None] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 10:47:29.967581 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.fernet_utils [None req-cb8b8ef5-1979-4652-b946-d58c92e57307 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80144) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 10:47:30.112071 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb8b8ef5-1979-4652-b946-d58c92e57307 None None] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 10:47:30.116046 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=100 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 10:47:30.232756 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c69e1f41-9ac7-4ea0-8cf8-3d5954997d19 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31d16d1b96104b4eb690398432b9829e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6de1f3cac02c4bcdb05dd2337176033d', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'reader', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 10:47:30.233371 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c69e1f41-9ac7-4ea0-8cf8-3d5954997d19 service nova] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:47:30.233651 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c69e1f41-9ac7-4ea0-8cf8-3d5954997d19 service nova] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:47:30.233923 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c69e1f41-9ac7-4ea0-8cf8-3d5954997d19 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:47:30.235718 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.fernet_utils [None req-c69e1f41-9ac7-4ea0-8cf8-3d5954997d19 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80144) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 10:47:30.356339 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c69e1f41-9ac7-4ea0-8cf8-3d5954997d19 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 10:47:30.361288 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c69e1f41-9ac7-4ea0-8cf8-3d5954997d19 service nova] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 10:47:30.383138 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 700/1379] 10.222.0.25 () {60 vars in 1365 bytes} [Sun Aug 7 10:47:29 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 417 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 07 10:47:32.473952 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22f646be-58a5-414b-aa7e-59f79177e03a None None] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:47:32.474557 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22f646be-58a5-414b-aa7e-59f79177e03a None None] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:47:32.474557 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22f646be-58a5-414b-aa7e-59f79177e03a None None] PATH_INFO: `/` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:47:32.476116 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 680/1380] 10.222.0.25 () {54 vars in 927 bytes} [Sun Aug 7 10:47:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 07 10:47:32.481740 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b63860ac-3193-4eaf-b861-2c3480a84687 None None] REQUEST_METHOD: `POST` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:47:32.482302 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b63860ac-3193-4eaf-b861-2c3480a84687 None None] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:47:32.482302 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b63860ac-3193-4eaf-b861-2c3480a84687 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:47:32.495267 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=100 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 10:47:32.556602 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.auth.core [None req-b63860ac-3193-4eaf-b861-2c3480a84687 None None] MFA Rules not processed for user `bac4e6b76fe64217ac8fa44e71cffe6a`. Rule list: `[]` (Enabled: `True`). {{(pid=80144) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 10:47:32.557821 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.fernet_utils [None req-b63860ac-3193-4eaf-b861-2c3480a84687 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80144) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 10:47:32.662965 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 701/1381] 10.222.0.25 () {58 vars in 1024 bytes} [Sun Aug 7 10:47:32 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 179 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 07 10:47:32.705064 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22f646be-58a5-414b-aa7e-59f79177e03a None None] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 10:47:32.706776 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-22f646be-58a5-414b-aa7e-59f79177e03a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 10:47:32.858204 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=100 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 10:47:32.871377 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22f646be-58a5-414b-aa7e-59f79177e03a None None] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 10:47:32.992720 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c96c5a2-f017-4f34-82db-6dafe48440ee service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31d16d1b96104b4eb690398432b9829e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6de1f3cac02c4bcdb05dd2337176033d', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'reader', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 10:47:32.993345 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c96c5a2-f017-4f34-82db-6dafe48440ee service nova] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:47:32.993688 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c96c5a2-f017-4f34-82db-6dafe48440ee service nova] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:47:32.994127 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c96c5a2-f017-4f34-82db-6dafe48440ee service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:47:32.995821 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-5c96c5a2-f017-4f34-82db-6dafe48440ee service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 10:47:33.093275 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c96c5a2-f017-4f34-82db-6dafe48440ee service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 10:47:33.095271 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c96c5a2-f017-4f34-82db-6dafe48440ee service nova] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 10:47:33.106273 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=100 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 10:47:33.118638 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 681/1382] 10.222.0.25 () {60 vars in 1365 bytes} [Sun Aug 7 10:47:32 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 414 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 07 10:47:35.179996 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52cd98b8-e0c9-4228-86b2-f93126923f59 None None] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:47:35.180786 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52cd98b8-e0c9-4228-86b2-f93126923f59 None None] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:47:35.180786 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52cd98b8-e0c9-4228-86b2-f93126923f59 None None] PATH_INFO: `/` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:47:35.182469 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 702/1383] 10.222.0.25 () {54 vars in 927 bytes} [Sun Aug 7 10:47:35 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 07 10:47:35.188015 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38cffd5c-3b52-4a38-a8f6-77aa3b56cf76 None None] REQUEST_METHOD: `POST` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:47:35.188482 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38cffd5c-3b52-4a38-a8f6-77aa3b56cf76 None None] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:47:35.188581 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38cffd5c-3b52-4a38-a8f6-77aa3b56cf76 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:47:35.262628 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.auth.core [None req-38cffd5c-3b52-4a38-a8f6-77aa3b56cf76 None None] MFA Rules not processed for user `bac4e6b76fe64217ac8fa44e71cffe6a`. Rule list: `[]` (Enabled: `True`). {{(pid=80143) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 10:47:35.263868 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-38cffd5c-3b52-4a38-a8f6-77aa3b56cf76 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 10:47:35.363838 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 682/1384] 10.222.0.25 () {58 vars in 1024 bytes} [Sun Aug 7 10:47:35 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 177 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 07 10:47:35.407878 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52cd98b8-e0c9-4228-86b2-f93126923f59 None None] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 10:47:35.409550 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.fernet_utils [None req-52cd98b8-e0c9-4228-86b2-f93126923f59 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80144) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 10:47:35.523225 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=100 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 10:47:35.596212 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52cd98b8-e0c9-4228-86b2-f93126923f59 None None] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 10:47:35.726426 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37b8569b-36c6-4340-b66d-3a4176e8c713 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31d16d1b96104b4eb690398432b9829e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6de1f3cac02c4bcdb05dd2337176033d', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'reader', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 10:47:35.727177 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37b8569b-36c6-4340-b66d-3a4176e8c713 service nova] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:47:35.727508 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37b8569b-36c6-4340-b66d-3a4176e8c713 service nova] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:47:35.727835 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37b8569b-36c6-4340-b66d-3a4176e8c713 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:47:35.729885 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.fernet_utils [None req-37b8569b-36c6-4340-b66d-3a4176e8c713 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80144) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 10:47:35.792325 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=100 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 10:47:35.839948 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37b8569b-36c6-4340-b66d-3a4176e8c713 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 10:47:35.841545 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37b8569b-36c6-4340-b66d-3a4176e8c713 service nova] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 10:47:35.860736 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 703/1385] 10.222.0.25 () {60 vars in 1365 bytes} [Sun Aug 7 10:47:35 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 453 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 07 10:47:37.881263 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a25f9682-70b4-4b74-95bf-6c17bd8655a8 None None] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:47:37.881776 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a25f9682-70b4-4b74-95bf-6c17bd8655a8 None None] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:47:37.881776 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a25f9682-70b4-4b74-95bf-6c17bd8655a8 None None] PATH_INFO: `/` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:47:37.883391 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 683/1386] 10.222.0.25 () {54 vars in 927 bytes} [Sun Aug 7 10:47:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 07 10:47:37.889509 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01efb789-7178-44af-949d-52e822d5db6d None None] REQUEST_METHOD: `POST` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:47:37.889921 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01efb789-7178-44af-949d-52e822d5db6d None None] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:47:37.890061 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01efb789-7178-44af-949d-52e822d5db6d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:47:37.958648 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.auth.core [None req-01efb789-7178-44af-949d-52e822d5db6d None None] MFA Rules not processed for user `bac4e6b76fe64217ac8fa44e71cffe6a`. Rule list: `[]` (Enabled: `True`). {{(pid=80144) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 10:47:37.959907 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.fernet_utils [None req-01efb789-7178-44af-949d-52e822d5db6d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80144) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 10:47:38.067874 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 704/1387] 10.222.0.25 () {58 vars in 1024 bytes} [Sun Aug 7 10:47:37 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 179 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 07 10:47:38.122141 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a25f9682-70b4-4b74-95bf-6c17bd8655a8 None None] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 10:47:38.123720 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-a25f9682-70b4-4b74-95bf-6c17bd8655a8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 10:47:38.207244 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=100 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 10:47:38.279193 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a25f9682-70b4-4b74-95bf-6c17bd8655a8 None None] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 10:47:38.402056 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b8932db-3eb8-4100-a7ce-e03b8e08e421 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31d16d1b96104b4eb690398432b9829e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6de1f3cac02c4bcdb05dd2337176033d', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'reader', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 10:47:38.402673 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b8932db-3eb8-4100-a7ce-e03b8e08e421 service nova] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:47:38.402978 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b8932db-3eb8-4100-a7ce-e03b8e08e421 service nova] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:47:38.403254 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b8932db-3eb8-4100-a7ce-e03b8e08e421 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:47:38.404861 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-7b8932db-3eb8-4100-a7ce-e03b8e08e421 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 10:47:38.446905 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=100 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 10:47:38.503762 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b8932db-3eb8-4100-a7ce-e03b8e08e421 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 10:47:38.505828 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b8932db-3eb8-4100-a7ce-e03b8e08e421 service nova] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 10:47:38.527499 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 684/1388] 10.222.0.25 () {60 vars in 1365 bytes} [Sun Aug 7 10:47:38 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 406 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 07 10:47:40.600762 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-619f0f0a-b3eb-4b16-97d0-9a00cc340efb None None] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:47:40.601295 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-619f0f0a-b3eb-4b16-97d0-9a00cc340efb None None] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:47:40.601295 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-619f0f0a-b3eb-4b16-97d0-9a00cc340efb None None] PATH_INFO: `/` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:47:40.602637 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 705/1389] 10.222.0.25 () {54 vars in 927 bytes} [Sun Aug 7 10:47:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 07 10:47:40.607872 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce1c3178-3026-4db4-bec7-253086f3adb2 None None] REQUEST_METHOD: `POST` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:47:40.609485 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce1c3178-3026-4db4-bec7-253086f3adb2 None None] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:47:40.609762 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce1c3178-3026-4db4-bec7-253086f3adb2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:47:40.676855 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.auth.core [None req-ce1c3178-3026-4db4-bec7-253086f3adb2 None None] MFA Rules not processed for user `bac4e6b76fe64217ac8fa44e71cffe6a`. Rule list: `[]` (Enabled: `True`). {{(pid=80143) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 10:47:40.677965 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-ce1c3178-3026-4db4-bec7-253086f3adb2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 10:47:40.785904 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 685/1390] 10.222.0.25 () {58 vars in 1024 bytes} [Sun Aug 7 10:47:40 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 07 10:47:40.831756 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-619f0f0a-b3eb-4b16-97d0-9a00cc340efb None None] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 10:47:40.833385 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.fernet_utils [None req-619f0f0a-b3eb-4b16-97d0-9a00cc340efb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80144) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 10:47:40.865412 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=100 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 10:47:40.981668 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-619f0f0a-b3eb-4b16-97d0-9a00cc340efb None None] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 10:47:41.085523 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=100 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 10:47:41.098709 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-158ca0ca-29eb-4379-9ee5-31c0ae7bac0c service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31d16d1b96104b4eb690398432b9829e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6de1f3cac02c4bcdb05dd2337176033d', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'reader', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 10:47:41.099360 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-158ca0ca-29eb-4379-9ee5-31c0ae7bac0c service nova] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:47:41.099642 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-158ca0ca-29eb-4379-9ee5-31c0ae7bac0c service nova] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:47:41.099916 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-158ca0ca-29eb-4379-9ee5-31c0ae7bac0c service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:47:41.101542 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.fernet_utils [None req-158ca0ca-29eb-4379-9ee5-31c0ae7bac0c service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80144) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 10:47:41.196888 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-158ca0ca-29eb-4379-9ee5-31c0ae7bac0c service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 10:47:41.198481 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-158ca0ca-29eb-4379-9ee5-31c0ae7bac0c service nova] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 10:47:41.216958 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 706/1391] 10.222.0.25 () {60 vars in 1365 bytes} [Sun Aug 7 10:47:40 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 386 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 07 10:47:42.032129 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ea17418-89e8-4b7e-8dec-1a189712e429 None None] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:47:42.032699 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ea17418-89e8-4b7e-8dec-1a189712e429 None None] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:47:42.032699 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ea17418-89e8-4b7e-8dec-1a189712e429 None None] PATH_INFO: `/` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:47:42.034062 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 686/1392] 10.222.0.25 () {54 vars in 927 bytes} [Sun Aug 7 10:47:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 07 10:47:42.039446 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34704c40-10da-4ae3-a2cb-84cba697cf71 None None] REQUEST_METHOD: `POST` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:47:42.039932 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34704c40-10da-4ae3-a2cb-84cba697cf71 None None] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:47:42.040138 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34704c40-10da-4ae3-a2cb-84cba697cf71 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:47:42.107769 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.auth.core [None req-34704c40-10da-4ae3-a2cb-84cba697cf71 None None] MFA Rules not processed for user `bac4e6b76fe64217ac8fa44e71cffe6a`. Rule list: `[]` (Enabled: `True`). {{(pid=80144) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 10:47:42.109878 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.fernet_utils [None req-34704c40-10da-4ae3-a2cb-84cba697cf71 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80144) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 10:47:42.169374 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=100 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 10:47:42.210515 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 707/1393] 10.222.0.25 () {58 vars in 1024 bytes} [Sun Aug 7 10:47:42 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 07 10:47:42.272859 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77d1b6f3-d5f2-4dc9-b804-f9a456dc2810 None None] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:47:42.273259 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77d1b6f3-d5f2-4dc9-b804-f9a456dc2810 None None] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:47:42.273603 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77d1b6f3-d5f2-4dc9-b804-f9a456dc2810 None None] PATH_INFO: `/` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:47:42.275060 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 687/1394] 10.222.0.25 () {54 vars in 966 bytes} [Sun Aug 7 10:47:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 07 10:47:42.280219 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7be63ab-7a25-40cd-884e-ca11314c9788 None None] REQUEST_METHOD: `POST` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:47:42.281226 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7be63ab-7a25-40cd-884e-ca11314c9788 None None] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:47:42.281652 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7be63ab-7a25-40cd-884e-ca11314c9788 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:47:42.359838 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.auth.core [None req-b7be63ab-7a25-40cd-884e-ca11314c9788 None None] MFA Rules not processed for user `a11186d9d0d6488b953ede2f1628148d`. Rule list: `[]` (Enabled: `True`). {{(pid=80144) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 10:47:42.360907 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.fernet_utils [None req-b7be63ab-7a25-40cd-884e-ca11314c9788 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80144) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 10:47:42.447792 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 708/1395] 10.222.0.25 () {58 vars in 1063 bytes} [Sun Aug 7 10:47:42 2022] POST /identity/v3/auth/tokens => generated 3120 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 07 10:47:42.455782 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77d1b6f3-d5f2-4dc9-b804-f9a456dc2810 None None] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 10:47:42.457349 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-77d1b6f3-d5f2-4dc9-b804-f9a456dc2810 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 10:47:42.475397 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=100 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 10:47:42.568338 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77d1b6f3-d5f2-4dc9-b804-f9a456dc2810 None None] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 10:47:42.643709 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae4e8ac9-f545-4f5e-8f22-58418d8a902e service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a11186d9d0d6488b953ede2f1628148d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6de1f3cac02c4bcdb05dd2337176033d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 10:47:42.644238 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae4e8ac9-f545-4f5e-8f22-58418d8a902e service glance] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:47:42.644518 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae4e8ac9-f545-4f5e-8f22-58418d8a902e service glance] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:47:42.644792 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae4e8ac9-f545-4f5e-8f22-58418d8a902e service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:47:42.646403 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-ae4e8ac9-f545-4f5e-8f22-58418d8a902e service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 10:47:42.714648 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=100 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 10:47:42.758832 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae4e8ac9-f545-4f5e-8f22-58418d8a902e service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 10:47:42.760433 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae4e8ac9-f545-4f5e-8f22-58418d8a902e service glance] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 10:47:42.779310 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 688/1396] 10.222.0.25 () {60 vars in 1365 bytes} [Sun Aug 7 10:47:42 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 324 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 07 10:47:43.365011 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5042196f-a3bb-4fa3-8435-f01a90314483 None None] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:47:43.365575 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5042196f-a3bb-4fa3-8435-f01a90314483 None None] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:47:43.365575 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5042196f-a3bb-4fa3-8435-f01a90314483 None None] PATH_INFO: `/` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:47:43.367141 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 709/1397] 10.222.0.25 () {54 vars in 927 bytes} [Sun Aug 7 10:47:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 07 10:47:43.373038 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da181e9a-378d-4e69-91a3-068d2d2ee1eb None None] REQUEST_METHOD: `POST` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:47:43.373587 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da181e9a-378d-4e69-91a3-068d2d2ee1eb None None] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:47:43.373679 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da181e9a-378d-4e69-91a3-068d2d2ee1eb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:47:43.455656 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.auth.core [None req-da181e9a-378d-4e69-91a3-068d2d2ee1eb None None] MFA Rules not processed for user `bac4e6b76fe64217ac8fa44e71cffe6a`. Rule list: `[]` (Enabled: `True`). {{(pid=80143) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 10:47:43.456947 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-da181e9a-378d-4e69-91a3-068d2d2ee1eb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 10:47:43.534184 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b9e6333-e1f7-462f-8b1d-a53e2036b261 None None] REQUEST_METHOD: `POST` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:47:43.534471 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b9e6333-e1f7-462f-8b1d-a53e2036b261 None None] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:47:43.534750 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b9e6333-e1f7-462f-8b1d-a53e2036b261 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:47:43.562410 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 689/1398] 10.222.0.25 () {58 vars in 1024 bytes} [Sun Aug 7 10:47:43 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 190 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 07 10:47:43.605544 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.auth.core [None req-5b9e6333-e1f7-462f-8b1d-a53e2036b261 None None] MFA Rules not processed for user `de1d6c24b1a14d10a20d82a4c65793d8`. Rule list: `[]` (Enabled: `True`). {{(pid=80144) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 10:47:43.606755 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.fernet_utils [None req-5b9e6333-e1f7-462f-8b1d-a53e2036b261 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80144) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 10:47:43.608504 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da181e9a-378d-4e69-91a3-068d2d2ee1eb None None] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 10:47:43.609780 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-da181e9a-378d-4e69-91a3-068d2d2ee1eb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 10:47:43.616653 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=100 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 10:47:43.637898 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=100 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 10:47:43.699794 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 710/1399] 10.222.0.25 () {58 vars in 1009 bytes} [Sun Aug 7 10:47:43 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 07 10:47:43.735656 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da181e9a-378d-4e69-91a3-068d2d2ee1eb None None] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 10:47:43.747935 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c719aae-aea3-4e27-9519-f08b1279e5a4 None None] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:47:43.748367 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c719aae-aea3-4e27-9519-f08b1279e5a4 None None] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:47:43.748732 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c719aae-aea3-4e27-9519-f08b1279e5a4 None None] PATH_INFO: `/` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:47:43.751054 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 711/1400] 10.222.0.25 () {54 vars in 943 bytes} [Sun Aug 7 10:47:43 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 07 10:47:43.757549 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43a6a9cf-aced-4cf6-b755-81d58873df12 None None] REQUEST_METHOD: `POST` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:47:43.757902 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43a6a9cf-aced-4cf6-b755-81d58873df12 None None] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:47:43.758263 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43a6a9cf-aced-4cf6-b755-81d58873df12 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:47:43.838815 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.auth.core [None req-43a6a9cf-aced-4cf6-b755-81d58873df12 None None] MFA Rules not processed for user `69490ea2a9a340bf91aca38bf5655b45`. Rule list: `[]` (Enabled: `True`). {{(pid=80144) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 10:47:43.840133 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.fernet_utils [None req-43a6a9cf-aced-4cf6-b755-81d58873df12 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80144) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 10:47:43.857453 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d6fbcd4-e748-4b1c-ab52-a114fb30c724 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31d16d1b96104b4eb690398432b9829e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6de1f3cac02c4bcdb05dd2337176033d', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'reader', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 10:47:43.859182 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d6fbcd4-e748-4b1c-ab52-a114fb30c724 service nova] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:47:43.859182 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d6fbcd4-e748-4b1c-ab52-a114fb30c724 service nova] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:47:43.859540 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d6fbcd4-e748-4b1c-ab52-a114fb30c724 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:47:43.861425 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-5d6fbcd4-e748-4b1c-ab52-a114fb30c724 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 10:47:43.890307 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=100 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 10:47:43.948610 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=100 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 10:47:43.973530 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 712/1401] 10.222.0.25 () {58 vars in 1040 bytes} [Sun Aug 7 10:47:43 2022] POST /identity/v3/auth/tokens => generated 3304 bytes in 216 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 07 10:47:43.977754 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d6fbcd4-e748-4b1c-ab52-a114fb30c724 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 10:47:43.979425 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d6fbcd4-e748-4b1c-ab52-a114fb30c724 service nova] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 10:47:43.984352 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43a6a9cf-aced-4cf6-b755-81d58873df12 None None] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 10:47:43.994957 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43a6a9cf-aced-4cf6-b755-81d58873df12 None None] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 10:47:43.998699 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 690/1402] 10.222.0.25 () {60 vars in 1365 bytes} [Sun Aug 7 10:47:43 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 391 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 07 10:47:44.114724 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f87b3fb2-8001-453e-b53f-2f499a904f1c service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69490ea2a9a340bf91aca38bf5655b45', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6de1f3cac02c4bcdb05dd2337176033d', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 10:47:44.115302 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f87b3fb2-8001-453e-b53f-2f499a904f1c service swift] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:47:44.115581 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f87b3fb2-8001-453e-b53f-2f499a904f1c service swift] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:47:44.115856 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f87b3fb2-8001-453e-b53f-2f499a904f1c service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:47:44.117652 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.fernet_utils [None req-f87b3fb2-8001-453e-b53f-2f499a904f1c service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80144) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 10:47:44.213034 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f87b3fb2-8001-453e-b53f-2f499a904f1c service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 10:47:44.214660 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f87b3fb2-8001-453e-b53f-2f499a904f1c service swift] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 10:47:44.224191 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 713/1403] 10.222.0.25 () {60 vars in 1384 bytes} [Sun Aug 7 10:47:43 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 241 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Aug 07 10:47:46.162242 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-592bb7d0-98b3-4dfe-9f7d-63f587795a37 None None] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:47:46.162779 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-592bb7d0-98b3-4dfe-9f7d-63f587795a37 None None] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:47:46.162779 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-592bb7d0-98b3-4dfe-9f7d-63f587795a37 None None] PATH_INFO: `/` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:47:46.164299 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 691/1404] 10.222.0.25 () {54 vars in 927 bytes} [Sun Aug 7 10:47:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 07 10:47:46.169512 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8015297-ee86-48d3-aaa3-9d0e1ead498d None None] REQUEST_METHOD: `POST` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:47:46.169942 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8015297-ee86-48d3-aaa3-9d0e1ead498d None None] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:47:46.170118 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8015297-ee86-48d3-aaa3-9d0e1ead498d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:47:46.185056 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=100 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 10:47:46.249725 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.auth.core [None req-b8015297-ee86-48d3-aaa3-9d0e1ead498d None None] MFA Rules not processed for user `bac4e6b76fe64217ac8fa44e71cffe6a`. Rule list: `[]` (Enabled: `True`). {{(pid=80144) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 10:47:46.251087 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.fernet_utils [None req-b8015297-ee86-48d3-aaa3-9d0e1ead498d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80144) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 10:47:46.365576 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 714/1405] 10.222.0.25 () {58 vars in 1024 bytes} [Sun Aug 7 10:47:46 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 197 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 07 10:47:46.413416 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-592bb7d0-98b3-4dfe-9f7d-63f587795a37 None None] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 10:47:46.414850 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-592bb7d0-98b3-4dfe-9f7d-63f587795a37 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 10:47:46.576038 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=100 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 10:47:46.579368 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-592bb7d0-98b3-4dfe-9f7d-63f587795a37 None None] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 10:47:46.704948 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3d03373-491f-42d4-848e-eda66376fb7a service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31d16d1b96104b4eb690398432b9829e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6de1f3cac02c4bcdb05dd2337176033d', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'reader', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 10:47:46.705471 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3d03373-491f-42d4-848e-eda66376fb7a service nova] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:47:46.705748 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3d03373-491f-42d4-848e-eda66376fb7a service nova] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:47:46.706019 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3d03373-491f-42d4-848e-eda66376fb7a service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:47:46.707661 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-c3d03373-491f-42d4-848e-eda66376fb7a service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 10:47:46.812724 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3d03373-491f-42d4-848e-eda66376fb7a service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 10:47:46.814876 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3d03373-491f-42d4-848e-eda66376fb7a service nova] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 10:47:46.837924 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 692/1406] 10.222.0.25 () {60 vars in 1365 bytes} [Sun Aug 7 10:47:46 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 425 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 07 10:47:48.897247 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac1b2b89-268f-4f60-bb9a-a3252e0e7e27 None None] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:47:48.897782 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac1b2b89-268f-4f60-bb9a-a3252e0e7e27 None None] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:47:48.897782 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac1b2b89-268f-4f60-bb9a-a3252e0e7e27 None None] PATH_INFO: `/` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:47:48.899523 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 715/1407] 10.222.0.25 () {54 vars in 927 bytes} [Sun Aug 7 10:47:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 07 10:47:48.904825 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2dbccdd-b5f1-4bcf-ad00-db9162dfa541 None None] REQUEST_METHOD: `POST` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:47:48.905896 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2dbccdd-b5f1-4bcf-ad00-db9162dfa541 None None] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:47:48.905986 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2dbccdd-b5f1-4bcf-ad00-db9162dfa541 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:47:48.910593 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=100 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 10:47:48.978530 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.auth.core [None req-c2dbccdd-b5f1-4bcf-ad00-db9162dfa541 None None] MFA Rules not processed for user `bac4e6b76fe64217ac8fa44e71cffe6a`. Rule list: `[]` (Enabled: `True`). {{(pid=80143) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 10:47:48.980394 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-c2dbccdd-b5f1-4bcf-ad00-db9162dfa541 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 10:47:49.091669 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 693/1408] 10.222.0.25 () {58 vars in 1024 bytes} [Sun Aug 7 10:47:48 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 188 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 07 10:47:49.146376 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac1b2b89-268f-4f60-bb9a-a3252e0e7e27 None None] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 10:47:49.148004 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.fernet_utils [None req-ac1b2b89-268f-4f60-bb9a-a3252e0e7e27 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80144) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 10:47:49.245240 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=100 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 10:47:49.312725 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac1b2b89-268f-4f60-bb9a-a3252e0e7e27 None None] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 10:47:49.453875 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f01e0d61-1af4-47de-8692-efaa38b28042 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31d16d1b96104b4eb690398432b9829e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6de1f3cac02c4bcdb05dd2337176033d', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'reader', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 10:47:49.454608 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f01e0d61-1af4-47de-8692-efaa38b28042 service nova] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:47:49.455116 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f01e0d61-1af4-47de-8692-efaa38b28042 service nova] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:47:49.455464 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f01e0d61-1af4-47de-8692-efaa38b28042 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:47:49.457108 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.fernet_utils [None req-f01e0d61-1af4-47de-8692-efaa38b28042 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80144) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 10:47:49.534059 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=100 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 10:47:49.588934 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f01e0d61-1af4-47de-8692-efaa38b28042 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 10:47:49.591180 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f01e0d61-1af4-47de-8692-efaa38b28042 service nova] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 10:47:49.619538 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 716/1409] 10.222.0.25 () {60 vars in 1365 bytes} [Sun Aug 7 10:47:49 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 473 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 07 10:47:59.089700 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=64 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 10:47:59.616737 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=23 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 10:48:06.413588 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8576f13-9478-44f5-9fa5-9751419c1391 None None] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:48:06.414116 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8576f13-9478-44f5-9fa5-9751419c1391 None None] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:48:06.414116 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8576f13-9478-44f5-9fa5-9751419c1391 None None] PATH_INFO: `/` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:48:06.415525 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 694/1410] 10.222.0.25 () {54 vars in 918 bytes} [Sun Aug 7 10:48:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 07 10:48:06.420759 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-738b2478-f230-4940-9954-e635a2f2282b None None] REQUEST_METHOD: `POST` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:48:06.421221 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-738b2478-f230-4940-9954-e635a2f2282b None None] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:48:06.421809 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-738b2478-f230-4940-9954-e635a2f2282b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:48:06.496111 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.auth.core [None req-738b2478-f230-4940-9954-e635a2f2282b None None] MFA Rules not processed for user `6ee2e4c08af04c219ed395d7ca7085a6`. Rule list: `[]` (Enabled: `True`). {{(pid=80144) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 10:48:06.497314 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.fernet_utils [None req-738b2478-f230-4940-9954-e635a2f2282b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80144) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 10:48:06.602819 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 717/1411] 10.222.0.25 () {58 vars in 1015 bytes} [Sun Aug 7 10:48:06 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 183 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 07 10:48:06.617756 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17bcd65d-a4f8-416d-b1f7-7bf54366fa97 None None] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:48:06.618049 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17bcd65d-a4f8-416d-b1f7-7bf54366fa97 None None] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:48:06.618327 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17bcd65d-a4f8-416d-b1f7-7bf54366fa97 None None] PATH_INFO: `/` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:48:06.619689 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 695/1412] 10.222.0.25 () {54 vars in 918 bytes} [Sun Aug 7 10:48:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 07 10:48:06.624741 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1db91dc5-8566-4793-ad0d-9b291b036654 None None] REQUEST_METHOD: `POST` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:48:06.625023 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1db91dc5-8566-4793-ad0d-9b291b036654 None None] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:48:06.625294 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1db91dc5-8566-4793-ad0d-9b291b036654 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:48:06.704418 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=100 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 10:48:06.707238 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.auth.core [None req-1db91dc5-8566-4793-ad0d-9b291b036654 None None] MFA Rules not processed for user `31d16d1b96104b4eb690398432b9829e`. Rule list: `[]` (Enabled: `True`). {{(pid=80144) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 10:48:06.708956 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.fernet_utils [None req-1db91dc5-8566-4793-ad0d-9b291b036654 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80144) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 10:48:06.821540 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 718/1413] 10.222.0.25 () {58 vars in 1015 bytes} [Sun Aug 7 10:48:06 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 198 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 07 10:48:09.289550 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f39f88d8-8d4c-42e8-b4cd-932e0cb7b6b0 None None] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:48:09.290104 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f39f88d8-8d4c-42e8-b4cd-932e0cb7b6b0 None None] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:48:09.290104 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f39f88d8-8d4c-42e8-b4cd-932e0cb7b6b0 None None] PATH_INFO: `/` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:48:09.291544 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 696/1414] 127.0.0.1 () {54 vars in 917 bytes} [Sun Aug 7 10:48:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 07 10:48:09.297633 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91dfca6a-9c2c-49a7-8a88-b732fd538fde None None] REQUEST_METHOD: `POST` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:48:09.298056 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91dfca6a-9c2c-49a7-8a88-b732fd538fde None None] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:48:09.298210 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91dfca6a-9c2c-49a7-8a88-b732fd538fde None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:48:09.363604 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.auth.core [None req-91dfca6a-9c2c-49a7-8a88-b732fd538fde None None] MFA Rules not processed for user `bac4e6b76fe64217ac8fa44e71cffe6a`. Rule list: `[]` (Enabled: `True`). {{(pid=80144) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 10:48:09.364801 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.fernet_utils [None req-91dfca6a-9c2c-49a7-8a88-b732fd538fde None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80144) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 10:48:09.422406 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=100 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 10:48:09.463601 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 719/1415] 127.0.0.1 () {58 vars in 1014 bytes} [Sun Aug 7 10:48:09 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 07 10:48:09.508699 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f39f88d8-8d4c-42e8-b4cd-932e0cb7b6b0 None None] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 10:48:09.510341 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-f39f88d8-8d4c-42e8-b4cd-932e0cb7b6b0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 10:48:09.659957 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f39f88d8-8d4c-42e8-b4cd-932e0cb7b6b0 None None] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 10:48:09.731661 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=100 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 10:48:09.771865 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f34b0362-5517-4b4e-8c5f-b681f75adced service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31d16d1b96104b4eb690398432b9829e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6de1f3cac02c4bcdb05dd2337176033d', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'reader', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 10:48:09.772462 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f34b0362-5517-4b4e-8c5f-b681f75adced service nova] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:48:09.772739 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f34b0362-5517-4b4e-8c5f-b681f75adced service nova] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:48:09.773013 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f34b0362-5517-4b4e-8c5f-b681f75adced service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:48:09.774591 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-f34b0362-5517-4b4e-8c5f-b681f75adced service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 10:48:09.863785 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f34b0362-5517-4b4e-8c5f-b681f75adced service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 10:48:09.865259 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f34b0362-5517-4b4e-8c5f-b681f75adced service nova] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 10:48:09.882756 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 697/1416] 10.222.0.25 () {60 vars in 1365 bytes} [Sun Aug 7 10:48:09 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 375 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 07 10:48:10.780670 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7ae6c76-4869-41c1-bcc2-35b1075d0db7 None None] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:48:10.781197 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7ae6c76-4869-41c1-bcc2-35b1075d0db7 None None] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:48:10.781332 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7ae6c76-4869-41c1-bcc2-35b1075d0db7 None None] PATH_INFO: `/` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:48:10.782578 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 720/1417] 127.0.0.1 () {54 vars in 901 bytes} [Sun Aug 7 10:48:10 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 07 10:48:10.787552 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71713d1d-dee9-4b10-abc9-3d8938c118ed None None] REQUEST_METHOD: `POST` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:48:10.787979 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71713d1d-dee9-4b10-abc9-3d8938c118ed None None] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:48:10.788083 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71713d1d-dee9-4b10-abc9-3d8938c118ed None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:48:10.866227 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.auth.core [None req-71713d1d-dee9-4b10-abc9-3d8938c118ed None None] MFA Rules not processed for user `bac4e6b76fe64217ac8fa44e71cffe6a`. Rule list: `[]` (Enabled: `True`). {{(pid=80143) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 10:48:10.867461 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-71713d1d-dee9-4b10-abc9-3d8938c118ed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 10:48:10.894982 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=100 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 10:48:10.967045 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 698/1418] 127.0.0.1 () {58 vars in 998 bytes} [Sun Aug 7 10:48:10 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 180 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 07 10:48:11.124878 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7ae6c76-4869-41c1-bcc2-35b1075d0db7 None None] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 10:48:11.129562 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.fernet_utils [None req-d7ae6c76-4869-41c1-bcc2-35b1075d0db7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80144) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 10:48:11.308437 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7ae6c76-4869-41c1-bcc2-35b1075d0db7 None None] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 10:48:11.360693 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=100 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 10:48:11.457841 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efb6fda3-0505-4afb-8f39-474dd5bc9375 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31d16d1b96104b4eb690398432b9829e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6de1f3cac02c4bcdb05dd2337176033d', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'reader', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 10:48:11.458452 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efb6fda3-0505-4afb-8f39-474dd5bc9375 service nova] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:48:11.458730 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efb6fda3-0505-4afb-8f39-474dd5bc9375 service nova] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:48:11.459036 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efb6fda3-0505-4afb-8f39-474dd5bc9375 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:48:11.460643 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.fernet_utils [None req-efb6fda3-0505-4afb-8f39-474dd5bc9375 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80144) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 10:48:11.580360 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efb6fda3-0505-4afb-8f39-474dd5bc9375 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 10:48:11.581958 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efb6fda3-0505-4afb-8f39-474dd5bc9375 service nova] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 10:48:11.603441 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 721/1419] 10.222.0.25 () {60 vars in 1365 bytes} [Sun Aug 7 10:48:11 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 482 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 07 10:48:12.505654 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1c449eb-8a71-4fed-b44d-58d3b7c4fc35 None None] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:48:12.506242 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1c449eb-8a71-4fed-b44d-58d3b7c4fc35 None None] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:48:12.506497 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1c449eb-8a71-4fed-b44d-58d3b7c4fc35 None None] PATH_INFO: `/` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:48:12.507888 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 699/1420] 127.0.0.1 () {54 vars in 901 bytes} [Sun Aug 7 10:48:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 07 10:48:12.513623 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7df7891-4378-40f3-bf6a-bf900f483901 None None] REQUEST_METHOD: `POST` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:48:12.514111 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7df7891-4378-40f3-bf6a-bf900f483901 None None] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:48:12.514215 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7df7891-4378-40f3-bf6a-bf900f483901 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:48:12.567406 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=100 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 10:48:12.589946 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.auth.core [None req-a7df7891-4378-40f3-bf6a-bf900f483901 None None] MFA Rules not processed for user `bac4e6b76fe64217ac8fa44e71cffe6a`. Rule list: `[]` (Enabled: `True`). {{(pid=80144) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 10:48:12.591212 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.fernet_utils [None req-a7df7891-4378-40f3-bf6a-bf900f483901 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80144) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 10:48:12.687858 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 722/1421] 127.0.0.1 () {58 vars in 998 bytes} [Sun Aug 7 10:48:12 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 176 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 07 10:48:12.838406 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1c449eb-8a71-4fed-b44d-58d3b7c4fc35 None None] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 10:48:12.839973 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-b1c449eb-8a71-4fed-b44d-58d3b7c4fc35 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 10:48:12.985429 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1c449eb-8a71-4fed-b44d-58d3b7c4fc35 None None] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 10:48:13.001560 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=100 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 10:48:13.102221 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-395fb23a-30cf-4ad7-82a7-9222175863f3 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31d16d1b96104b4eb690398432b9829e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6de1f3cac02c4bcdb05dd2337176033d', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'reader', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 10:48:13.102830 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-395fb23a-30cf-4ad7-82a7-9222175863f3 service nova] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:48:13.103143 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-395fb23a-30cf-4ad7-82a7-9222175863f3 service nova] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:48:13.103421 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-395fb23a-30cf-4ad7-82a7-9222175863f3 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:48:13.105035 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-395fb23a-30cf-4ad7-82a7-9222175863f3 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 10:48:13.193456 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-395fb23a-30cf-4ad7-82a7-9222175863f3 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 10:48:13.195123 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-395fb23a-30cf-4ad7-82a7-9222175863f3 service nova] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 10:48:13.213272 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 700/1422] 10.222.0.25 () {60 vars in 1365 bytes} [Sun Aug 7 10:48:12 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 375 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 07 10:48:14.067911 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f03df0fe-78ca-4943-be33-bda998cfa01f None None] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:48:14.068446 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f03df0fe-78ca-4943-be33-bda998cfa01f None None] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:48:14.068446 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f03df0fe-78ca-4943-be33-bda998cfa01f None None] PATH_INFO: `/` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:48:14.069815 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 723/1423] 127.0.0.1 () {54 vars in 901 bytes} [Sun Aug 7 10:48:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 07 10:48:14.074709 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-903587f5-be01-4994-9a0d-21fd55da041a None None] REQUEST_METHOD: `POST` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:48:14.075737 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-903587f5-be01-4994-9a0d-21fd55da041a None None] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:48:14.075933 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-903587f5-be01-4994-9a0d-21fd55da041a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:48:14.095578 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=100 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 10:48:14.153297 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.auth.core [None req-903587f5-be01-4994-9a0d-21fd55da041a None None] MFA Rules not processed for user `bac4e6b76fe64217ac8fa44e71cffe6a`. Rule list: `[]` (Enabled: `True`). {{(pid=80143) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 10:48:14.154504 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-903587f5-be01-4994-9a0d-21fd55da041a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 10:48:14.250318 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 701/1424] 127.0.0.1 () {58 vars in 998 bytes} [Sun Aug 7 10:48:14 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 177 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 07 10:48:14.403352 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f03df0fe-78ca-4943-be33-bda998cfa01f None None] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 10:48:14.404989 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.fernet_utils [None req-f03df0fe-78ca-4943-be33-bda998cfa01f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80144) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 10:48:14.515823 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=100 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 10:48:14.552422 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f03df0fe-78ca-4943-be33-bda998cfa01f None None] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 10:48:14.663707 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a328e026-bb49-462c-9890-8ba1341fda16 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31d16d1b96104b4eb690398432b9829e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6de1f3cac02c4bcdb05dd2337176033d', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'reader', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 10:48:14.664312 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a328e026-bb49-462c-9890-8ba1341fda16 service nova] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:48:14.664591 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a328e026-bb49-462c-9890-8ba1341fda16 service nova] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:48:14.664862 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a328e026-bb49-462c-9890-8ba1341fda16 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:48:14.666448 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.fernet_utils [None req-a328e026-bb49-462c-9890-8ba1341fda16 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80144) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 10:48:14.741355 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=100 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 10:48:14.755094 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a328e026-bb49-462c-9890-8ba1341fda16 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 10:48:14.756670 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a328e026-bb49-462c-9890-8ba1341fda16 service nova] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 10:48:14.774406 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 724/1425] 10.222.0.25 () {60 vars in 1365 bytes} [Sun Aug 7 10:48:14 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 371 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 07 10:48:15.638131 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a6490fc-93e1-415f-9fac-4dd4a80f5ef2 None None] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:48:15.638692 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a6490fc-93e1-415f-9fac-4dd4a80f5ef2 None None] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:48:15.638692 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a6490fc-93e1-415f-9fac-4dd4a80f5ef2 None None] PATH_INFO: `/` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:48:15.640056 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 702/1426] 127.0.0.1 () {54 vars in 901 bytes} [Sun Aug 7 10:48:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 07 10:48:15.644992 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d803b7c1-e553-4a56-93fd-32c304a5c78b None None] REQUEST_METHOD: `POST` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:48:15.645412 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d803b7c1-e553-4a56-93fd-32c304a5c78b None None] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:48:15.645538 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d803b7c1-e553-4a56-93fd-32c304a5c78b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:48:15.720988 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.auth.core [None req-d803b7c1-e553-4a56-93fd-32c304a5c78b None None] MFA Rules not processed for user `bac4e6b76fe64217ac8fa44e71cffe6a`. Rule list: `[]` (Enabled: `True`). {{(pid=80144) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 10:48:15.722117 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.fernet_utils [None req-d803b7c1-e553-4a56-93fd-32c304a5c78b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80144) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 10:48:15.817790 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 725/1427] 127.0.0.1 () {58 vars in 998 bytes} [Sun Aug 7 10:48:15 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 07 10:48:15.969897 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a6490fc-93e1-415f-9fac-4dd4a80f5ef2 None None] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 10:48:15.971518 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-3a6490fc-93e1-415f-9fac-4dd4a80f5ef2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 10:48:16.055809 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=100 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 10:48:16.118304 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a6490fc-93e1-415f-9fac-4dd4a80f5ef2 None None] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 10:48:16.229288 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6958af01-f35b-4105-b91e-d30c984c68f5 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31d16d1b96104b4eb690398432b9829e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6de1f3cac02c4bcdb05dd2337176033d', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'reader', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 10:48:16.229816 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6958af01-f35b-4105-b91e-d30c984c68f5 service nova] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:48:16.230094 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6958af01-f35b-4105-b91e-d30c984c68f5 service nova] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:48:16.230364 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6958af01-f35b-4105-b91e-d30c984c68f5 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:48:16.232591 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-6958af01-f35b-4105-b91e-d30c984c68f5 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 10:48:16.279694 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=100 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 10:48:16.320418 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6958af01-f35b-4105-b91e-d30c984c68f5 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 10:48:16.321983 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6958af01-f35b-4105-b91e-d30c984c68f5 service nova] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 10:48:16.339764 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 703/1428] 10.222.0.25 () {60 vars in 1365 bytes} [Sun Aug 7 10:48:15 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 370 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 07 10:48:17.140086 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bb86d77-f8fe-4521-ba98-2a5350aab097 None None] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:48:17.140658 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bb86d77-f8fe-4521-ba98-2a5350aab097 None None] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:48:17.140658 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bb86d77-f8fe-4521-ba98-2a5350aab097 None None] PATH_INFO: `/` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:48:17.142014 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 726/1429] 127.0.0.1 () {54 vars in 901 bytes} [Sun Aug 7 10:48:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 07 10:48:17.146915 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31f1e09a-7846-4486-9264-9712e438935b None None] REQUEST_METHOD: `POST` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:48:17.147327 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31f1e09a-7846-4486-9264-9712e438935b None None] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:48:17.147488 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31f1e09a-7846-4486-9264-9712e438935b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:48:17.223217 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.auth.core [None req-31f1e09a-7846-4486-9264-9712e438935b None None] MFA Rules not processed for user `bac4e6b76fe64217ac8fa44e71cffe6a`. Rule list: `[]` (Enabled: `True`). {{(pid=80143) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 10:48:17.224397 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-31f1e09a-7846-4486-9264-9712e438935b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 10:48:17.318189 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 704/1430] 127.0.0.1 () {58 vars in 998 bytes} [Sun Aug 7 10:48:17 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 07 10:48:17.469207 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bb86d77-f8fe-4521-ba98-2a5350aab097 None None] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 10:48:17.470720 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.fernet_utils [None req-5bb86d77-f8fe-4521-ba98-2a5350aab097 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80144) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 10:48:17.521956 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=100 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 10:48:17.620174 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bb86d77-f8fe-4521-ba98-2a5350aab097 None None] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 10:48:17.730480 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c370950-9045-4fb3-986c-004f4fc1735e service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31d16d1b96104b4eb690398432b9829e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6de1f3cac02c4bcdb05dd2337176033d', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'reader', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 10:48:17.731033 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c370950-9045-4fb3-986c-004f4fc1735e service nova] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:48:17.731310 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c370950-9045-4fb3-986c-004f4fc1735e service nova] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:48:17.731577 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c370950-9045-4fb3-986c-004f4fc1735e service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:48:17.733150 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.fernet_utils [None req-9c370950-9045-4fb3-986c-004f4fc1735e service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80144) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 10:48:17.740392 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=100 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 10:48:17.821825 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c370950-9045-4fb3-986c-004f4fc1735e service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 10:48:17.823417 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c370950-9045-4fb3-986c-004f4fc1735e service nova] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 10:48:17.841584 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 727/1431] 10.222.0.25 () {60 vars in 1365 bytes} [Sun Aug 7 10:48:17 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 373 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 07 10:48:18.651091 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec18638b-de9b-4ded-90b7-1a48fd9e3eb8 None None] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:48:18.651656 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec18638b-de9b-4ded-90b7-1a48fd9e3eb8 None None] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:48:18.651656 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec18638b-de9b-4ded-90b7-1a48fd9e3eb8 None None] PATH_INFO: `/` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:48:18.653698 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 705/1432] 127.0.0.1 () {54 vars in 901 bytes} [Sun Aug 7 10:48:18 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 07 10:48:18.658969 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa404e27-7975-4b94-bed6-0deba4969eb5 None None] REQUEST_METHOD: `POST` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:48:18.659370 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa404e27-7975-4b94-bed6-0deba4969eb5 None None] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:48:18.659528 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa404e27-7975-4b94-bed6-0deba4969eb5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:48:18.734001 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.auth.core [None req-aa404e27-7975-4b94-bed6-0deba4969eb5 None None] MFA Rules not processed for user `bac4e6b76fe64217ac8fa44e71cffe6a`. Rule list: `[]` (Enabled: `True`). {{(pid=80144) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 10:48:18.735142 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.fernet_utils [None req-aa404e27-7975-4b94-bed6-0deba4969eb5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80144) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 10:48:18.801045 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=100 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 10:48:18.830289 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 728/1433] 127.0.0.1 () {58 vars in 998 bytes} [Sun Aug 7 10:48:18 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 07 10:48:18.981694 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec18638b-de9b-4ded-90b7-1a48fd9e3eb8 None None] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 10:48:18.983260 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-ec18638b-de9b-4ded-90b7-1a48fd9e3eb8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 10:48:19.001477 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=100 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 10:48:19.127186 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec18638b-de9b-4ded-90b7-1a48fd9e3eb8 None None] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 10:48:19.213016 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=100 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 10:48:19.238791 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0fd0abf-7702-4064-a3c3-33fcd305bfdd service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31d16d1b96104b4eb690398432b9829e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6de1f3cac02c4bcdb05dd2337176033d', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'reader', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 10:48:19.239432 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0fd0abf-7702-4064-a3c3-33fcd305bfdd service nova] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:48:19.239718 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0fd0abf-7702-4064-a3c3-33fcd305bfdd service nova] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:48:19.240000 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0fd0abf-7702-4064-a3c3-33fcd305bfdd service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:48:19.241610 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-e0fd0abf-7702-4064-a3c3-33fcd305bfdd service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 10:48:19.329814 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0fd0abf-7702-4064-a3c3-33fcd305bfdd service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 10:48:19.331430 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0fd0abf-7702-4064-a3c3-33fcd305bfdd service nova] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 10:48:19.348990 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 706/1434] 10.222.0.25 () {60 vars in 1365 bytes} [Sun Aug 7 10:48:18 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 368 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 07 10:48:20.157099 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfa374c7-c869-4d59-8829-21b284b4755e None None] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:48:20.157676 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfa374c7-c869-4d59-8829-21b284b4755e None None] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:48:20.157676 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfa374c7-c869-4d59-8829-21b284b4755e None None] PATH_INFO: `/` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:48:20.159086 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 729/1435] 127.0.0.1 () {54 vars in 901 bytes} [Sun Aug 7 10:48:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 07 10:48:20.163901 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-734bd8e4-ada2-4b18-b32d-7705025e7be0 None None] REQUEST_METHOD: `POST` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:48:20.164311 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-734bd8e4-ada2-4b18-b32d-7705025e7be0 None None] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:48:20.164551 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-734bd8e4-ada2-4b18-b32d-7705025e7be0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:48:20.240489 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.auth.core [None req-734bd8e4-ada2-4b18-b32d-7705025e7be0 None None] MFA Rules not processed for user `bac4e6b76fe64217ac8fa44e71cffe6a`. Rule list: `[]` (Enabled: `True`). {{(pid=80143) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 10:48:20.241680 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-734bd8e4-ada2-4b18-b32d-7705025e7be0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 10:48:20.283714 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=100 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 10:48:20.337495 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 707/1436] 127.0.0.1 () {58 vars in 998 bytes} [Sun Aug 7 10:48:20 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 07 10:48:20.490825 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfa374c7-c869-4d59-8829-21b284b4755e None None] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 10:48:20.492479 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.fernet_utils [None req-cfa374c7-c869-4d59-8829-21b284b4755e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80144) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 10:48:20.640152 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfa374c7-c869-4d59-8829-21b284b4755e None None] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 10:48:20.699768 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=100 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 10:48:20.757739 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4999708e-c490-47d3-a438-d0dcbafcd40d service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31d16d1b96104b4eb690398432b9829e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6de1f3cac02c4bcdb05dd2337176033d', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'reader', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 10:48:20.758341 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4999708e-c490-47d3-a438-d0dcbafcd40d service nova] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:48:20.758617 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4999708e-c490-47d3-a438-d0dcbafcd40d service nova] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:48:20.758887 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4999708e-c490-47d3-a438-d0dcbafcd40d service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:48:20.760499 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.fernet_utils [None req-4999708e-c490-47d3-a438-d0dcbafcd40d service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80144) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 10:48:20.848441 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4999708e-c490-47d3-a438-d0dcbafcd40d service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 10:48:20.850016 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4999708e-c490-47d3-a438-d0dcbafcd40d service nova] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 10:48:20.868511 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 730/1437] 10.222.0.25 () {60 vars in 1365 bytes} [Sun Aug 7 10:48:20 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 378 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 07 10:48:21.664856 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20699d02-0122-4f97-aeab-c5d57fc4c2c4 None None] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:48:21.665439 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20699d02-0122-4f97-aeab-c5d57fc4c2c4 None None] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:48:21.665439 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20699d02-0122-4f97-aeab-c5d57fc4c2c4 None None] PATH_INFO: `/` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:48:21.666783 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 708/1438] 127.0.0.1 () {54 vars in 901 bytes} [Sun Aug 7 10:48:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 07 10:48:21.672112 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27b76930-1ebc-4044-bde7-24c7122d8b63 None None] REQUEST_METHOD: `POST` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:48:21.672523 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27b76930-1ebc-4044-bde7-24c7122d8b63 None None] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:48:21.672637 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27b76930-1ebc-4044-bde7-24c7122d8b63 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:48:21.748752 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.auth.core [None req-27b76930-1ebc-4044-bde7-24c7122d8b63 None None] MFA Rules not processed for user `bac4e6b76fe64217ac8fa44e71cffe6a`. Rule list: `[]` (Enabled: `True`). {{(pid=80144) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 10:48:21.749941 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.fernet_utils [None req-27b76930-1ebc-4044-bde7-24c7122d8b63 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80144) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 10:48:21.751983 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=100 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 10:48:21.847865 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 731/1439] 127.0.0.1 () {58 vars in 998 bytes} [Sun Aug 7 10:48:21 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 177 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 07 10:48:21.999274 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20699d02-0122-4f97-aeab-c5d57fc4c2c4 None None] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 10:48:22.000711 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-20699d02-0122-4f97-aeab-c5d57fc4c2c4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 10:48:22.149199 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20699d02-0122-4f97-aeab-c5d57fc4c2c4 None None] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 10:48:22.176114 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=100 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 10:48:22.262814 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a38ced9e-bb58-4942-bde9-f1b9941aadcc service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31d16d1b96104b4eb690398432b9829e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6de1f3cac02c4bcdb05dd2337176033d', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'reader', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 10:48:22.263467 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a38ced9e-bb58-4942-bde9-f1b9941aadcc service nova] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:48:22.263749 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a38ced9e-bb58-4942-bde9-f1b9941aadcc service nova] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:48:22.264024 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a38ced9e-bb58-4942-bde9-f1b9941aadcc service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:48:22.265637 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-a38ced9e-bb58-4942-bde9-f1b9941aadcc service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 10:48:22.354544 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a38ced9e-bb58-4942-bde9-f1b9941aadcc service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 10:48:22.356730 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a38ced9e-bb58-4942-bde9-f1b9941aadcc service nova] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 10:48:22.374363 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 709/1440] 10.222.0.25 () {60 vars in 1365 bytes} [Sun Aug 7 10:48:21 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 376 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 07 10:48:24.286063 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34619176-6a47-4d58-a227-98030818ac89 None None] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:48:24.286593 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34619176-6a47-4d58-a227-98030818ac89 None None] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:48:24.286593 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34619176-6a47-4d58-a227-98030818ac89 None None] PATH_INFO: `/` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:48:24.287974 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 732/1441] 127.0.0.1 () {54 vars in 917 bytes} [Sun Aug 7 10:48:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 07 10:48:24.293027 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff3d8343-f458-40e0-a979-335f9cc90546 None None] REQUEST_METHOD: `POST` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:48:24.294070 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff3d8343-f458-40e0-a979-335f9cc90546 None None] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:48:24.294176 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff3d8343-f458-40e0-a979-335f9cc90546 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:48:24.335675 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=100 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 10:48:24.362389 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.auth.core [None req-ff3d8343-f458-40e0-a979-335f9cc90546 None None] MFA Rules not processed for user `bac4e6b76fe64217ac8fa44e71cffe6a`. Rule list: `[]` (Enabled: `True`). {{(pid=80143) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 10:48:24.363894 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-ff3d8343-f458-40e0-a979-335f9cc90546 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 10:48:24.458537 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 710/1442] 127.0.0.1 () {58 vars in 1014 bytes} [Sun Aug 7 10:48:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 07 10:48:24.497663 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34619176-6a47-4d58-a227-98030818ac89 None None] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 10:48:24.499225 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.fernet_utils [None req-34619176-6a47-4d58-a227-98030818ac89 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80144) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 10:48:24.601079 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34619176-6a47-4d58-a227-98030818ac89 None None] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 10:48:24.633256 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=100 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 10:48:24.674087 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71d16e97-4674-4256-9941-1080815631fe service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be1283c9bf47424c9aa3ad27cab03027', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6de1f3cac02c4bcdb05dd2337176033d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 10:48:24.674687 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71d16e97-4674-4256-9941-1080815631fe service neutron] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:48:24.674984 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71d16e97-4674-4256-9941-1080815631fe service neutron] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:48:24.675255 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71d16e97-4674-4256-9941-1080815631fe service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:48:24.676829 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.fernet_utils [None req-71d16e97-4674-4256-9941-1080815631fe service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80144) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 10:48:24.783349 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71d16e97-4674-4256-9941-1080815631fe service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 10:48:24.784750 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71d16e97-4674-4256-9941-1080815631fe service neutron] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 10:48:24.802808 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 733/1443] 10.222.0.25 () {60 vars in 1365 bytes} [Sun Aug 7 10:48:24 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 306 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 07 10:48:26.962159 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a047c06-0b09-46ea-925d-a79b3d4c623f None None] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:48:26.962727 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a047c06-0b09-46ea-925d-a79b3d4c623f None None] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:48:26.962727 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a047c06-0b09-46ea-925d-a79b3d4c623f None None] PATH_INFO: `/` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:48:26.964130 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 711/1444] 127.0.0.1 () {54 vars in 917 bytes} [Sun Aug 7 10:48:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 07 10:48:26.969504 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f97b2a1a-35a1-4350-8245-f27d573ecc50 None None] REQUEST_METHOD: `POST` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:48:26.969923 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f97b2a1a-35a1-4350-8245-f27d573ecc50 None None] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:48:26.970050 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f97b2a1a-35a1-4350-8245-f27d573ecc50 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:48:27.036826 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.auth.core [None req-f97b2a1a-35a1-4350-8245-f27d573ecc50 None None] MFA Rules not processed for user `bac4e6b76fe64217ac8fa44e71cffe6a`. Rule list: `[]` (Enabled: `True`). {{(pid=80144) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 10:48:27.038035 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.fernet_utils [None req-f97b2a1a-35a1-4350-8245-f27d573ecc50 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80144) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 10:48:27.051417 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=100 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 10:48:27.136209 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 734/1445] 127.0.0.1 () {58 vars in 1014 bytes} [Sun Aug 7 10:48:26 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 07 10:48:27.173799 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a047c06-0b09-46ea-925d-a79b3d4c623f None None] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 10:48:27.175396 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-0a047c06-0b09-46ea-925d-a79b3d4c623f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 10:48:27.276972 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a047c06-0b09-46ea-925d-a79b3d4c623f None None] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 10:48:27.289225 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=100 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 10:48:27.349278 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbacd842-ca6b-475d-8d80-78469a7715c3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be1283c9bf47424c9aa3ad27cab03027', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6de1f3cac02c4bcdb05dd2337176033d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 10:48:27.349885 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbacd842-ca6b-475d-8d80-78469a7715c3 service neutron] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:48:27.350160 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbacd842-ca6b-475d-8d80-78469a7715c3 service neutron] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:48:27.350424 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbacd842-ca6b-475d-8d80-78469a7715c3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:48:27.352042 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-fbacd842-ca6b-475d-8d80-78469a7715c3 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 10:48:27.457680 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbacd842-ca6b-475d-8d80-78469a7715c3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 10:48:27.459131 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbacd842-ca6b-475d-8d80-78469a7715c3 service neutron] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 10:48:27.476976 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 712/1446] 10.222.0.25 () {60 vars in 1365 bytes} [Sun Aug 7 10:48:27 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 304 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 07 10:48:29.787839 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91010e11-569a-4c03-8363-9075558e6438 None None] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:48:29.788392 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91010e11-569a-4c03-8363-9075558e6438 None None] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:48:29.788392 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91010e11-569a-4c03-8363-9075558e6438 None None] PATH_INFO: `/` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:48:29.789747 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 735/1447] 127.0.0.1 () {54 vars in 917 bytes} [Sun Aug 7 10:48:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 07 10:48:29.795646 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7370cec9-b5c1-457b-b313-bda737de4a0a None None] REQUEST_METHOD: `POST` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:48:29.796103 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7370cec9-b5c1-457b-b313-bda737de4a0a None None] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:48:29.796213 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7370cec9-b5c1-457b-b313-bda737de4a0a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:48:29.843905 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=100 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 10:48:29.862417 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.auth.core [None req-7370cec9-b5c1-457b-b313-bda737de4a0a None None] MFA Rules not processed for user `bac4e6b76fe64217ac8fa44e71cffe6a`. Rule list: `[]` (Enabled: `True`). {{(pid=80143) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 10:48:29.863927 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-7370cec9-b5c1-457b-b313-bda737de4a0a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 10:48:29.960595 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 713/1448] 127.0.0.1 () {58 vars in 1014 bytes} [Sun Aug 7 10:48:29 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 07 10:48:29.999947 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91010e11-569a-4c03-8363-9075558e6438 None None] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 10:48:30.001469 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.fernet_utils [None req-91010e11-569a-4c03-8363-9075558e6438 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80144) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 10:48:30.106520 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91010e11-569a-4c03-8363-9075558e6438 None None] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 10:48:30.147189 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=100 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 10:48:30.180320 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11a2a46d-0d25-4125-9fa0-2e749150ea68 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be1283c9bf47424c9aa3ad27cab03027', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6de1f3cac02c4bcdb05dd2337176033d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 10:48:30.180922 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11a2a46d-0d25-4125-9fa0-2e749150ea68 service neutron] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:48:30.181199 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11a2a46d-0d25-4125-9fa0-2e749150ea68 service neutron] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:48:30.181471 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11a2a46d-0d25-4125-9fa0-2e749150ea68 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:48:30.183085 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.fernet_utils [None req-11a2a46d-0d25-4125-9fa0-2e749150ea68 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80144) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 10:48:30.288692 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11a2a46d-0d25-4125-9fa0-2e749150ea68 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 10:48:30.290134 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11a2a46d-0d25-4125-9fa0-2e749150ea68 service neutron] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 10:48:30.308613 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 736/1449] 10.222.0.25 () {60 vars in 1365 bytes} [Sun Aug 7 10:48:29 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 309 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 07 10:48:32.420396 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0305f56e-d3f4-4169-b7ff-02965f1c7782 None None] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:48:32.420926 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0305f56e-d3f4-4169-b7ff-02965f1c7782 None None] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:48:32.420926 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0305f56e-d3f4-4169-b7ff-02965f1c7782 None None] PATH_INFO: `/` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:48:32.422306 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 714/1450] 127.0.0.1 () {54 vars in 917 bytes} [Sun Aug 7 10:48:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 07 10:48:32.427503 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39765343-9b88-47b0-ae45-24afc7512e6f None None] REQUEST_METHOD: `POST` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:48:32.427879 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39765343-9b88-47b0-ae45-24afc7512e6f None None] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:48:32.428059 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39765343-9b88-47b0-ae45-24afc7512e6f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:48:32.494392 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.auth.core [None req-39765343-9b88-47b0-ae45-24afc7512e6f None None] MFA Rules not processed for user `bac4e6b76fe64217ac8fa44e71cffe6a`. Rule list: `[]` (Enabled: `True`). {{(pid=80144) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 10:48:32.495630 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.fernet_utils [None req-39765343-9b88-47b0-ae45-24afc7512e6f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80144) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 10:48:32.517056 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=100 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 10:48:32.592260 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 737/1451] 127.0.0.1 () {58 vars in 1014 bytes} [Sun Aug 7 10:48:32 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 07 10:48:32.630875 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0305f56e-d3f4-4169-b7ff-02965f1c7782 None None] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 10:48:32.632484 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-0305f56e-d3f4-4169-b7ff-02965f1c7782 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 10:48:32.737132 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0305f56e-d3f4-4169-b7ff-02965f1c7782 None None] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 10:48:32.756746 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=100 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 10:48:32.809545 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8a6d6d9-ceea-46b8-aec7-523e349ab4d3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be1283c9bf47424c9aa3ad27cab03027', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6de1f3cac02c4bcdb05dd2337176033d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 10:48:32.810152 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8a6d6d9-ceea-46b8-aec7-523e349ab4d3 service neutron] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:48:32.810425 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8a6d6d9-ceea-46b8-aec7-523e349ab4d3 service neutron] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:48:32.810692 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8a6d6d9-ceea-46b8-aec7-523e349ab4d3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:48:32.812293 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-d8a6d6d9-ceea-46b8-aec7-523e349ab4d3 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 10:48:32.919357 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8a6d6d9-ceea-46b8-aec7-523e349ab4d3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 10:48:32.920782 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8a6d6d9-ceea-46b8-aec7-523e349ab4d3 service neutron] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 10:48:32.938657 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 715/1452] 10.222.0.25 () {60 vars in 1365 bytes} [Sun Aug 7 10:48:32 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 308 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 07 10:48:35.008583 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9311662-638b-4b6e-b324-a333276146be None None] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:48:35.009134 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9311662-638b-4b6e-b324-a333276146be None None] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:48:35.009134 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9311662-638b-4b6e-b324-a333276146be None None] PATH_INFO: `/` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:48:35.010479 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 738/1453] 127.0.0.1 () {54 vars in 917 bytes} [Sun Aug 7 10:48:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 07 10:48:35.016021 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3339f9aa-7a15-45ad-a152-a6438e71be6f None None] REQUEST_METHOD: `POST` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:48:35.016458 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3339f9aa-7a15-45ad-a152-a6438e71be6f None None] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:48:35.016564 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3339f9aa-7a15-45ad-a152-a6438e71be6f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:48:35.081632 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=100 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 10:48:35.084286 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.auth.core [None req-3339f9aa-7a15-45ad-a152-a6438e71be6f None None] MFA Rules not processed for user `bac4e6b76fe64217ac8fa44e71cffe6a`. Rule list: `[]` (Enabled: `True`). {{(pid=80143) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 10:48:35.086027 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-3339f9aa-7a15-45ad-a152-a6438e71be6f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 10:48:35.181507 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 716/1454] 127.0.0.1 () {58 vars in 1014 bytes} [Sun Aug 7 10:48:35 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 07 10:48:35.220802 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9311662-638b-4b6e-b324-a333276146be None None] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 10:48:35.222344 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.fernet_utils [None req-b9311662-638b-4b6e-b324-a333276146be None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80144) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 10:48:35.326085 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9311662-638b-4b6e-b324-a333276146be None None] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 10:48:35.374345 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=100 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 10:48:35.399439 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf3b5cc6-bb79-4daf-a397-4824fa2878b6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be1283c9bf47424c9aa3ad27cab03027', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6de1f3cac02c4bcdb05dd2337176033d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 10:48:35.400046 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf3b5cc6-bb79-4daf-a397-4824fa2878b6 service neutron] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:48:35.400324 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf3b5cc6-bb79-4daf-a397-4824fa2878b6 service neutron] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:48:35.400596 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf3b5cc6-bb79-4daf-a397-4824fa2878b6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:48:35.402913 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.fernet_utils [None req-cf3b5cc6-bb79-4daf-a397-4824fa2878b6 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80144) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 10:48:35.510544 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf3b5cc6-bb79-4daf-a397-4824fa2878b6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 10:48:35.512850 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf3b5cc6-bb79-4daf-a397-4824fa2878b6 service neutron] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 10:48:35.530784 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 739/1455] 10.222.0.25 () {60 vars in 1365 bytes} [Sun Aug 7 10:48:35 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 311 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 07 10:48:37.590752 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccb8543c-1dd7-45c3-920a-7b016aa50272 None None] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:48:37.591329 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccb8543c-1dd7-45c3-920a-7b016aa50272 None None] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:48:37.591329 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccb8543c-1dd7-45c3-920a-7b016aa50272 None None] PATH_INFO: `/` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:48:37.592677 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 717/1456] 127.0.0.1 () {54 vars in 917 bytes} [Sun Aug 7 10:48:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 07 10:48:37.597864 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37f12edf-63f9-488b-9fb7-ce9fc2c893a4 None None] REQUEST_METHOD: `POST` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:48:37.598257 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37f12edf-63f9-488b-9fb7-ce9fc2c893a4 None None] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:48:37.598413 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37f12edf-63f9-488b-9fb7-ce9fc2c893a4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:48:37.670300 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.auth.core [None req-37f12edf-63f9-488b-9fb7-ce9fc2c893a4 None None] MFA Rules not processed for user `bac4e6b76fe64217ac8fa44e71cffe6a`. Rule list: `[]` (Enabled: `True`). {{(pid=80144) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 10:48:37.671607 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.fernet_utils [None req-37f12edf-63f9-488b-9fb7-ce9fc2c893a4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80144) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 10:48:37.701420 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=100 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 10:48:37.769474 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 740/1457] 127.0.0.1 () {58 vars in 1014 bytes} [Sun Aug 7 10:48:37 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 07 10:48:37.809126 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccb8543c-1dd7-45c3-920a-7b016aa50272 None None] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 10:48:37.810651 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-ccb8543c-1dd7-45c3-920a-7b016aa50272 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 10:48:37.916041 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccb8543c-1dd7-45c3-920a-7b016aa50272 None None] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 10:48:37.945206 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=100 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 10:48:37.990597 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ccbbf7f-ed8d-463e-baff-d9175be4f3ad service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be1283c9bf47424c9aa3ad27cab03027', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6de1f3cac02c4bcdb05dd2337176033d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 10:48:37.991254 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ccbbf7f-ed8d-463e-baff-d9175be4f3ad service neutron] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:48:37.991532 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ccbbf7f-ed8d-463e-baff-d9175be4f3ad service neutron] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:48:37.991805 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ccbbf7f-ed8d-463e-baff-d9175be4f3ad service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:48:37.993399 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-2ccbbf7f-ed8d-463e-baff-d9175be4f3ad service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 10:48:38.100020 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ccbbf7f-ed8d-463e-baff-d9175be4f3ad service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 10:48:38.101442 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ccbbf7f-ed8d-463e-baff-d9175be4f3ad service neutron] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 10:48:38.121422 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 718/1458] 10.222.0.25 () {60 vars in 1365 bytes} [Sun Aug 7 10:48:37 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 313 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 07 10:48:40.164829 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d8fd0a2-11b3-4dfa-84c3-144681538b9b None None] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:48:40.165392 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d8fd0a2-11b3-4dfa-84c3-144681538b9b None None] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:48:40.165392 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d8fd0a2-11b3-4dfa-84c3-144681538b9b None None] PATH_INFO: `/` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:48:40.166719 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 741/1459] 127.0.0.1 () {54 vars in 917 bytes} [Sun Aug 7 10:48:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 07 10:48:40.172716 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c29c78e8-6bc8-4e99-bca3-d1e5e731b58b None None] REQUEST_METHOD: `POST` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:48:40.173152 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c29c78e8-6bc8-4e99-bca3-d1e5e731b58b None None] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:48:40.173250 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c29c78e8-6bc8-4e99-bca3-d1e5e731b58b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:48:40.238426 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.auth.core [None req-c29c78e8-6bc8-4e99-bca3-d1e5e731b58b None None] MFA Rules not processed for user `bac4e6b76fe64217ac8fa44e71cffe6a`. Rule list: `[]` (Enabled: `True`). {{(pid=80143) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 10:48:40.239664 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-c29c78e8-6bc8-4e99-bca3-d1e5e731b58b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 10:48:40.247976 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=100 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 10:48:40.336001 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 719/1460] 127.0.0.1 () {58 vars in 1014 bytes} [Sun Aug 7 10:48:40 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 07 10:48:40.375497 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d8fd0a2-11b3-4dfa-84c3-144681538b9b None None] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 10:48:40.377023 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.fernet_utils [None req-7d8fd0a2-11b3-4dfa-84c3-144681538b9b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80144) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 10:48:40.481688 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d8fd0a2-11b3-4dfa-84c3-144681538b9b None None] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 10:48:40.538299 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=100 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 10:48:40.556660 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f7c89c7-04d5-401c-b268-e3989a968c16 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be1283c9bf47424c9aa3ad27cab03027', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6de1f3cac02c4bcdb05dd2337176033d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 10:48:40.557263 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f7c89c7-04d5-401c-b268-e3989a968c16 service neutron] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:48:40.557541 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f7c89c7-04d5-401c-b268-e3989a968c16 service neutron] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:48:40.557811 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f7c89c7-04d5-401c-b268-e3989a968c16 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:48:40.559496 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.fernet_utils [None req-1f7c89c7-04d5-401c-b268-e3989a968c16 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80144) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 10:48:40.666481 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f7c89c7-04d5-401c-b268-e3989a968c16 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 10:48:40.668938 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f7c89c7-04d5-401c-b268-e3989a968c16 service neutron] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 10:48:40.688057 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 742/1461] 10.222.0.25 () {60 vars in 1365 bytes} [Sun Aug 7 10:48:40 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 313 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 07 10:48:42.719989 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01b736c1-4f1f-4993-b13d-26586d820c26 None None] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:48:42.720611 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01b736c1-4f1f-4993-b13d-26586d820c26 None None] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:48:42.720611 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01b736c1-4f1f-4993-b13d-26586d820c26 None None] PATH_INFO: `/` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:48:42.721783 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 720/1462] 127.0.0.1 () {54 vars in 917 bytes} [Sun Aug 7 10:48:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 07 10:48:42.726973 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e567990d-6fef-47eb-9d86-0c2497c3a390 None None] REQUEST_METHOD: `POST` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:48:42.727431 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e567990d-6fef-47eb-9d86-0c2497c3a390 None None] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:48:42.727550 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e567990d-6fef-47eb-9d86-0c2497c3a390 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:48:42.792749 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.auth.core [None req-e567990d-6fef-47eb-9d86-0c2497c3a390 None None] MFA Rules not processed for user `bac4e6b76fe64217ac8fa44e71cffe6a`. Rule list: `[]` (Enabled: `True`). {{(pid=80144) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 10:48:42.793939 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.fernet_utils [None req-e567990d-6fef-47eb-9d86-0c2497c3a390 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80144) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 10:48:42.831799 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=100 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 10:48:42.889483 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 743/1463] 127.0.0.1 () {58 vars in 1014 bytes} [Sun Aug 7 10:48:42 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 07 10:48:42.927995 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01b736c1-4f1f-4993-b13d-26586d820c26 None None] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 10:48:42.929519 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-01b736c1-4f1f-4993-b13d-26586d820c26 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 10:48:43.031514 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01b736c1-4f1f-4993-b13d-26586d820c26 None None] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 10:48:43.069105 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=100 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 10:48:43.106761 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be2cdea0-0e2f-4ef5-b4dd-8161b5afcf4b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be1283c9bf47424c9aa3ad27cab03027', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6de1f3cac02c4bcdb05dd2337176033d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 10:48:43.107395 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be2cdea0-0e2f-4ef5-b4dd-8161b5afcf4b service neutron] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:48:43.107677 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be2cdea0-0e2f-4ef5-b4dd-8161b5afcf4b service neutron] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:48:43.107953 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be2cdea0-0e2f-4ef5-b4dd-8161b5afcf4b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:48:43.109555 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-be2cdea0-0e2f-4ef5-b4dd-8161b5afcf4b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 10:48:43.216516 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be2cdea0-0e2f-4ef5-b4dd-8161b5afcf4b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 10:48:43.217966 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be2cdea0-0e2f-4ef5-b4dd-8161b5afcf4b service neutron] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 10:48:43.236101 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 721/1464] 10.222.0.25 () {60 vars in 1365 bytes} [Sun Aug 7 10:48:42 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 309 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 07 10:48:45.285256 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30c58982-9cfa-47a4-ac7a-57c79a1e0cfd None None] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:48:45.285827 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30c58982-9cfa-47a4-ac7a-57c79a1e0cfd None None] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:48:45.285827 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30c58982-9cfa-47a4-ac7a-57c79a1e0cfd None None] PATH_INFO: `/` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:48:45.287249 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 744/1465] 127.0.0.1 () {54 vars in 917 bytes} [Sun Aug 7 10:48:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 07 10:48:45.292981 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b819a0b-6295-4bb1-9cb8-6b8fee7851c3 None None] REQUEST_METHOD: `POST` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:48:45.293432 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b819a0b-6295-4bb1-9cb8-6b8fee7851c3 None None] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:48:45.293536 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b819a0b-6295-4bb1-9cb8-6b8fee7851c3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:48:45.358140 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.auth.core [None req-4b819a0b-6295-4bb1-9cb8-6b8fee7851c3 None None] MFA Rules not processed for user `bac4e6b76fe64217ac8fa44e71cffe6a`. Rule list: `[]` (Enabled: `True`). {{(pid=80143) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 10:48:45.359363 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-4b819a0b-6295-4bb1-9cb8-6b8fee7851c3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 10:48:45.377116 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=100 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 10:48:45.455189 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 722/1466] 127.0.0.1 () {58 vars in 1014 bytes} [Sun Aug 7 10:48:45 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 07 10:48:45.492135 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30c58982-9cfa-47a4-ac7a-57c79a1e0cfd None None] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 10:48:45.493703 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.fernet_utils [None req-30c58982-9cfa-47a4-ac7a-57c79a1e0cfd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80144) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 10:48:45.596090 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30c58982-9cfa-47a4-ac7a-57c79a1e0cfd None None] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 10:48:45.663670 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=100 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 10:48:45.672413 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d48b093e-e259-4d8c-af64-bd6a277f1184 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be1283c9bf47424c9aa3ad27cab03027', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6de1f3cac02c4bcdb05dd2337176033d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 10:48:45.673026 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d48b093e-e259-4d8c-af64-bd6a277f1184 service neutron] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:48:45.673304 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d48b093e-e259-4d8c-af64-bd6a277f1184 service neutron] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:48:45.673575 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d48b093e-e259-4d8c-af64-bd6a277f1184 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:48:45.675209 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.fernet_utils [None req-d48b093e-e259-4d8c-af64-bd6a277f1184 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80144) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 10:48:45.784487 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d48b093e-e259-4d8c-af64-bd6a277f1184 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 10:48:45.785905 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d48b093e-e259-4d8c-af64-bd6a277f1184 service neutron] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 10:48:45.803521 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 745/1467] 10.222.0.25 () {60 vars in 1365 bytes} [Sun Aug 7 10:48:45 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 312 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 07 10:48:47.820166 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cac6d4d-938e-402d-9465-86c5a7421f8c None None] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:48:47.820739 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cac6d4d-938e-402d-9465-86c5a7421f8c None None] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:48:47.820739 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cac6d4d-938e-402d-9465-86c5a7421f8c None None] PATH_INFO: `/` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:48:47.822044 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 723/1468] 127.0.0.1 () {54 vars in 917 bytes} [Sun Aug 7 10:48:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 07 10:48:47.828280 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dabecabe-d57d-4920-869e-8e95dff983d1 None None] REQUEST_METHOD: `POST` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:48:47.828731 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dabecabe-d57d-4920-869e-8e95dff983d1 None None] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:48:47.828840 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dabecabe-d57d-4920-869e-8e95dff983d1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:48:47.893767 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.auth.core [None req-dabecabe-d57d-4920-869e-8e95dff983d1 None None] MFA Rules not processed for user `bac4e6b76fe64217ac8fa44e71cffe6a`. Rule list: `[]` (Enabled: `True`). {{(pid=80144) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 10:48:47.894984 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.fernet_utils [None req-dabecabe-d57d-4920-869e-8e95dff983d1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80144) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 10:48:47.942529 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=100 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 10:48:47.991280 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 746/1469] 127.0.0.1 () {58 vars in 1014 bytes} [Sun Aug 7 10:48:47 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 07 10:48:48.028772 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cac6d4d-938e-402d-9465-86c5a7421f8c None None] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 10:48:48.030375 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-9cac6d4d-938e-402d-9465-86c5a7421f8c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 10:48:48.135972 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cac6d4d-938e-402d-9465-86c5a7421f8c None None] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 10:48:48.182182 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=100 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 10:48:48.207960 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcc57d5d-3479-417e-8ce2-52476e87e96d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be1283c9bf47424c9aa3ad27cab03027', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6de1f3cac02c4bcdb05dd2337176033d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 10:48:48.208555 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcc57d5d-3479-417e-8ce2-52476e87e96d service neutron] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:48:48.208831 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcc57d5d-3479-417e-8ce2-52476e87e96d service neutron] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:48:48.209101 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcc57d5d-3479-417e-8ce2-52476e87e96d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:48:48.210700 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-bcc57d5d-3479-417e-8ce2-52476e87e96d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 10:48:48.317966 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcc57d5d-3479-417e-8ce2-52476e87e96d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 10:48:48.319421 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcc57d5d-3479-417e-8ce2-52476e87e96d service neutron] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 10:48:48.337439 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 724/1470] 10.222.0.25 () {60 vars in 1365 bytes} [Sun Aug 7 10:48:48 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 309 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 07 10:48:57.989468 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=22 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 10:48:58.335173 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=66 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 10:49:07.847499 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feba697b-7fe5-48c9-9a35-ac5f908b483e None None] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:49:07.848041 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feba697b-7fe5-48c9-9a35-ac5f908b483e None None] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:49:07.848135 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feba697b-7fe5-48c9-9a35-ac5f908b483e None None] PATH_INFO: `/` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:49:07.849450 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 747/1471] 10.222.0.25 () {54 vars in 927 bytes} [Sun Aug 7 10:49:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 07 10:49:07.855386 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f346db98-aec9-4f2a-9826-4638ebac9580 None None] REQUEST_METHOD: `POST` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:49:07.855836 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f346db98-aec9-4f2a-9826-4638ebac9580 None None] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:49:07.855934 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f346db98-aec9-4f2a-9826-4638ebac9580 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:49:07.923406 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.auth.core [None req-f346db98-aec9-4f2a-9826-4638ebac9580 None None] MFA Rules not processed for user `bac4e6b76fe64217ac8fa44e71cffe6a`. Rule list: `[]` (Enabled: `True`). {{(pid=80143) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 10:49:07.924613 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-f346db98-aec9-4f2a-9826-4638ebac9580 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 10:49:08.022295 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 725/1472] 10.222.0.25 () {58 vars in 1024 bytes} [Sun Aug 7 10:49:07 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 07 10:49:08.108205 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-859bb4d5-6c5f-4336-8d62-da9d44d27655 None None] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:49:08.108489 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-859bb4d5-6c5f-4336-8d62-da9d44d27655 None None] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:49:08.108764 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-859bb4d5-6c5f-4336-8d62-da9d44d27655 None None] PATH_INFO: `/` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:49:08.110116 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 748/1473] 10.222.0.25 () {54 vars in 966 bytes} [Sun Aug 7 10:49:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 07 10:49:08.115320 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac42aea1-4611-41ba-9a19-bb1125b856f4 None None] REQUEST_METHOD: `POST` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:49:08.115612 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac42aea1-4611-41ba-9a19-bb1125b856f4 None None] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:49:08.115890 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac42aea1-4611-41ba-9a19-bb1125b856f4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:49:08.192723 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.auth.core [None req-ac42aea1-4611-41ba-9a19-bb1125b856f4 None None] MFA Rules not processed for user `a11186d9d0d6488b953ede2f1628148d`. Rule list: `[]` (Enabled: `True`). {{(pid=80143) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 10:49:08.193919 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-ac42aea1-4611-41ba-9a19-bb1125b856f4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 10:49:08.214631 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=100 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 10:49:08.270987 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 726/1474] 10.222.0.25 () {58 vars in 1063 bytes} [Sun Aug 7 10:49:08 2022] POST /identity/v3/auth/tokens => generated 3120 bytes in 157 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 07 10:49:08.279136 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-859bb4d5-6c5f-4336-8d62-da9d44d27655 None None] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 10:49:08.280687 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.fernet_utils [None req-859bb4d5-6c5f-4336-8d62-da9d44d27655 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80144) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 10:49:08.385848 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-859bb4d5-6c5f-4336-8d62-da9d44d27655 None None] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 10:49:08.458587 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-689211de-afa6-48d4-80c0-cc8bd2bee2e4 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a11186d9d0d6488b953ede2f1628148d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6de1f3cac02c4bcdb05dd2337176033d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 10:49:08.459220 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-689211de-afa6-48d4-80c0-cc8bd2bee2e4 service glance] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:49:08.459503 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-689211de-afa6-48d4-80c0-cc8bd2bee2e4 service glance] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:49:08.459776 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-689211de-afa6-48d4-80c0-cc8bd2bee2e4 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:49:08.461374 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.fernet_utils [None req-689211de-afa6-48d4-80c0-cc8bd2bee2e4 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80144) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 10:49:08.514809 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=100 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 10:49:08.577325 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-689211de-afa6-48d4-80c0-cc8bd2bee2e4 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 10:49:08.578759 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-689211de-afa6-48d4-80c0-cc8bd2bee2e4 service glance] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 10:49:08.598781 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 749/1475] 10.222.0.25 () {60 vars in 1365 bytes} [Sun Aug 7 10:49:08 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 320 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 07 10:49:10.673670 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f31324de-70b2-41c2-b0b1-9f65752cda6a None None] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:49:10.674202 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f31324de-70b2-41c2-b0b1-9f65752cda6a None None] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:49:10.674202 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f31324de-70b2-41c2-b0b1-9f65752cda6a None None] PATH_INFO: `/` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:49:10.675593 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 727/1476] 10.222.0.25 () {54 vars in 927 bytes} [Sun Aug 7 10:49:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 07 10:49:10.680798 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4e17ea2-7519-414f-ae96-d8ab8dd9947e None None] REQUEST_METHOD: `POST` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:49:10.681877 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4e17ea2-7519-414f-ae96-d8ab8dd9947e None None] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:49:10.682399 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4e17ea2-7519-414f-ae96-d8ab8dd9947e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:49:10.747284 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.auth.core [None req-f4e17ea2-7519-414f-ae96-d8ab8dd9947e None None] MFA Rules not processed for user `bac4e6b76fe64217ac8fa44e71cffe6a`. Rule list: `[]` (Enabled: `True`). {{(pid=80144) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 10:49:10.748473 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.fernet_utils [None req-f4e17ea2-7519-414f-ae96-d8ab8dd9947e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80144) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 10:49:10.843583 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 750/1477] 10.222.0.25 () {58 vars in 1024 bytes} [Sun Aug 7 10:49:10 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 07 10:49:10.867366 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f31324de-70b2-41c2-b0b1-9f65752cda6a None None] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 10:49:10.868976 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-f31324de-70b2-41c2-b0b1-9f65752cda6a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 10:49:10.994494 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f31324de-70b2-41c2-b0b1-9f65752cda6a None None] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 10:49:11.164637 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=100 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 10:49:11.208749 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48f097e4-bb6d-41f3-814f-642bab48cdd1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5ac47e3b09f43d19a23a0f457fe9e23', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 10:49:11.209352 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48f097e4-bb6d-41f3-814f-642bab48cdd1 admin admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:49:11.209631 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48f097e4-bb6d-41f3-814f-642bab48cdd1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:49:11.209904 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48f097e4-bb6d-41f3-814f-642bab48cdd1 admin admin] PATH_INFO: `/v3/projects` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:49:11.210539 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48f097e4-bb6d-41f3-814f-642bab48cdd1 admin admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 10:49:11.212936 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: WARNING py.warnings [None req-48f097e4-bb6d-41f3-814f-642bab48cdd1 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_projects": "(role:reader and system_scope:all) or (role:reader and domain_id:%(target.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 07 10:49:11.212936 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: warnings.warn(msg) Aug 07 10:49:11.212936 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: Aug 07 10:49:11.214065 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48f097e4-bb6d-41f3-814f-642bab48cdd1 admin admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 10:49:11.231992 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 728/1478] 10.222.0.25 () {56 vars in 1108 bytes} [Sun Aug 7 10:49:10 2022] GET /identity/v3/projects => generated 2553 bytes in 365 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 07 10:49:13.319577 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-512e0223-e237-4bdd-83e7-e54f2458e206 None None] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:49:13.320245 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-512e0223-e237-4bdd-83e7-e54f2458e206 None None] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:49:13.320245 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-512e0223-e237-4bdd-83e7-e54f2458e206 None None] PATH_INFO: `/` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:49:13.322525 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 751/1479] 10.222.0.25 () {54 vars in 927 bytes} [Sun Aug 7 10:49:13 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 07 10:49:13.328181 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d876c4e4-03d1-4ecb-8df8-e13bc231bfb6 None None] REQUEST_METHOD: `POST` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:49:13.328658 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d876c4e4-03d1-4ecb-8df8-e13bc231bfb6 None None] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:49:13.328759 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d876c4e4-03d1-4ecb-8df8-e13bc231bfb6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:49:13.399190 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.auth.core [None req-d876c4e4-03d1-4ecb-8df8-e13bc231bfb6 None None] MFA Rules not processed for user `bac4e6b76fe64217ac8fa44e71cffe6a`. Rule list: `[]` (Enabled: `True`). {{(pid=80143) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 10:49:13.400504 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-d876c4e4-03d1-4ecb-8df8-e13bc231bfb6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 10:49:13.505391 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 729/1480] 10.222.0.25 () {58 vars in 1024 bytes} [Sun Aug 7 10:49:13 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 178 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 07 10:49:13.560026 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-512e0223-e237-4bdd-83e7-e54f2458e206 None None] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 10:49:13.561639 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.fernet_utils [None req-512e0223-e237-4bdd-83e7-e54f2458e206 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80144) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 10:49:13.572523 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=100 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 10:49:13.723382 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-512e0223-e237-4bdd-83e7-e54f2458e206 None None] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 10:49:13.812616 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=100 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 10:49:13.845900 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7aedcf6b-607c-43c5-ad40-f758bac13815 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31d16d1b96104b4eb690398432b9829e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6de1f3cac02c4bcdb05dd2337176033d', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'reader', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 10:49:13.846428 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aedcf6b-607c-43c5-ad40-f758bac13815 service nova] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:49:13.846733 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aedcf6b-607c-43c5-ad40-f758bac13815 service nova] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:49:13.847043 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aedcf6b-607c-43c5-ad40-f758bac13815 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:49:13.848632 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.fernet_utils [None req-7aedcf6b-607c-43c5-ad40-f758bac13815 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80144) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 10:49:13.939816 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7aedcf6b-607c-43c5-ad40-f758bac13815 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 10:49:13.941504 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7aedcf6b-607c-43c5-ad40-f758bac13815 service nova] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 10:49:13.959657 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 752/1481] 10.222.0.25 () {60 vars in 1365 bytes} [Sun Aug 7 10:49:13 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 400 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 07 10:49:16.150847 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39f29141-5ee0-4121-95e6-70cb8af3831b None None] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:49:16.151436 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39f29141-5ee0-4121-95e6-70cb8af3831b None None] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:49:16.151436 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39f29141-5ee0-4121-95e6-70cb8af3831b None None] PATH_INFO: `/` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:49:16.152823 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 730/1482] 10.222.0.25 () {54 vars in 927 bytes} [Sun Aug 7 10:49:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 07 10:49:16.157878 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3cca339-0b79-41f3-aacd-aa06196e0800 None None] REQUEST_METHOD: `POST` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:49:16.158348 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3cca339-0b79-41f3-aacd-aa06196e0800 None None] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:49:16.158455 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3cca339-0b79-41f3-aacd-aa06196e0800 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:49:16.225086 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.auth.core [None req-c3cca339-0b79-41f3-aacd-aa06196e0800 None None] MFA Rules not processed for user `bac4e6b76fe64217ac8fa44e71cffe6a`. Rule list: `[]` (Enabled: `True`). {{(pid=80144) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 10:49:16.226290 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.fernet_utils [None req-c3cca339-0b79-41f3-aacd-aa06196e0800 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80144) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 10:49:16.272584 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=100 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 10:49:16.322555 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 753/1483] 10.222.0.25 () {58 vars in 1024 bytes} [Sun Aug 7 10:49:16 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 07 10:49:16.359128 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39f29141-5ee0-4121-95e6-70cb8af3831b None None] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 10:49:16.360721 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-39f29141-5ee0-4121-95e6-70cb8af3831b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 10:49:16.384744 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=100 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 10:49:16.464268 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39f29141-5ee0-4121-95e6-70cb8af3831b None None] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 10:49:16.536855 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ef3000b-a36a-48b9-a2d4-806528d6648b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be1283c9bf47424c9aa3ad27cab03027', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6de1f3cac02c4bcdb05dd2337176033d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 10:49:16.537457 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ef3000b-a36a-48b9-a2d4-806528d6648b service neutron] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:49:16.537735 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ef3000b-a36a-48b9-a2d4-806528d6648b service neutron] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:49:16.537999 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ef3000b-a36a-48b9-a2d4-806528d6648b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:49:16.539626 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-2ef3000b-a36a-48b9-a2d4-806528d6648b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 10:49:16.612006 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=100 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 10:49:16.647040 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ef3000b-a36a-48b9-a2d4-806528d6648b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 10:49:16.648469 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ef3000b-a36a-48b9-a2d4-806528d6648b service neutron] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 10:49:16.666972 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 731/1484] 10.222.0.25 () {60 vars in 1365 bytes} [Sun Aug 7 10:49:16 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 308 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 07 10:49:18.731901 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e240f9e2-7320-4835-bd44-c620ed80ce22 None None] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:49:18.732437 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e240f9e2-7320-4835-bd44-c620ed80ce22 None None] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:49:18.732437 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e240f9e2-7320-4835-bd44-c620ed80ce22 None None] PATH_INFO: `/` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:49:18.733756 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 754/1485] 10.222.0.25 () {54 vars in 927 bytes} [Sun Aug 7 10:49:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 07 10:49:18.739075 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eab368d-266f-4879-b0b6-86240f967c34 None None] REQUEST_METHOD: `POST` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:49:18.739524 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eab368d-266f-4879-b0b6-86240f967c34 None None] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:49:18.739640 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eab368d-266f-4879-b0b6-86240f967c34 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:49:18.807707 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.auth.core [None req-1eab368d-266f-4879-b0b6-86240f967c34 None None] MFA Rules not processed for user `bac4e6b76fe64217ac8fa44e71cffe6a`. Rule list: `[]` (Enabled: `True`). {{(pid=80143) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 10:49:18.808915 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-1eab368d-266f-4879-b0b6-86240f967c34 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 10:49:18.904407 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 732/1486] 10.222.0.25 () {58 vars in 1024 bytes} [Sun Aug 7 10:49:18 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 07 10:49:18.943358 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e240f9e2-7320-4835-bd44-c620ed80ce22 None None] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 10:49:18.944904 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.fernet_utils [None req-e240f9e2-7320-4835-bd44-c620ed80ce22 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80144) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 10:49:19.049692 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e240f9e2-7320-4835-bd44-c620ed80ce22 None None] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 10:49:19.121237 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=100 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 10:49:19.122884 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-470b5428-ced1-43eb-983c-09abeaff52cf service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be1283c9bf47424c9aa3ad27cab03027', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6de1f3cac02c4bcdb05dd2337176033d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 10:49:19.123804 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-470b5428-ced1-43eb-983c-09abeaff52cf service neutron] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:49:19.124084 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-470b5428-ced1-43eb-983c-09abeaff52cf service neutron] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:49:19.124416 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-470b5428-ced1-43eb-983c-09abeaff52cf service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:49:19.126396 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.fernet_utils [None req-470b5428-ced1-43eb-983c-09abeaff52cf service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80144) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 10:49:19.237338 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-470b5428-ced1-43eb-983c-09abeaff52cf service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 10:49:19.238756 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-470b5428-ced1-43eb-983c-09abeaff52cf service neutron] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 10:49:19.256889 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 755/1487] 10.222.0.25 () {60 vars in 1365 bytes} [Sun Aug 7 10:49:18 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 314 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 07 10:49:21.406278 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccdff3fa-9b1b-4b2b-8c0e-2d0fac766df8 None None] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:49:21.406774 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccdff3fa-9b1b-4b2b-8c0e-2d0fac766df8 None None] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:49:21.406897 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccdff3fa-9b1b-4b2b-8c0e-2d0fac766df8 None None] PATH_INFO: `/` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:49:21.408459 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 733/1488] 10.222.0.25 () {54 vars in 927 bytes} [Sun Aug 7 10:49:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 07 10:49:21.413810 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97b93f7b-9494-4653-ab45-6a56050ba4aa None None] REQUEST_METHOD: `POST` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:49:21.414241 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97b93f7b-9494-4653-ab45-6a56050ba4aa None None] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:49:21.414369 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97b93f7b-9494-4653-ab45-6a56050ba4aa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:49:21.481794 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.auth.core [None req-97b93f7b-9494-4653-ab45-6a56050ba4aa None None] MFA Rules not processed for user `bac4e6b76fe64217ac8fa44e71cffe6a`. Rule list: `[]` (Enabled: `True`). {{(pid=80144) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 10:49:21.483009 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.fernet_utils [None req-97b93f7b-9494-4653-ab45-6a56050ba4aa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80144) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 10:49:21.534556 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=100 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 10:49:21.579528 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 756/1489] 10.222.0.25 () {58 vars in 1024 bytes} [Sun Aug 7 10:49:21 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 07 10:49:21.620400 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccdff3fa-9b1b-4b2b-8c0e-2d0fac766df8 None None] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 10:49:21.621922 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-ccdff3fa-9b1b-4b2b-8c0e-2d0fac766df8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 10:49:21.657857 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=100 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 10:49:21.724656 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccdff3fa-9b1b-4b2b-8c0e-2d0fac766df8 None None] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 10:49:21.796692 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86905251-76fd-4075-bcc9-7feae36a02a8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be1283c9bf47424c9aa3ad27cab03027', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6de1f3cac02c4bcdb05dd2337176033d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 10:49:21.797295 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86905251-76fd-4075-bcc9-7feae36a02a8 service neutron] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:49:21.797566 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86905251-76fd-4075-bcc9-7feae36a02a8 service neutron] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:49:21.797832 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86905251-76fd-4075-bcc9-7feae36a02a8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:49:21.799421 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-86905251-76fd-4075-bcc9-7feae36a02a8 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 10:49:21.876722 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=100 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 10:49:21.905223 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86905251-76fd-4075-bcc9-7feae36a02a8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 10:49:21.906639 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86905251-76fd-4075-bcc9-7feae36a02a8 service neutron] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 10:49:21.924772 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 734/1490] 10.222.0.25 () {60 vars in 1365 bytes} [Sun Aug 7 10:49:21 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 305 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 07 10:49:24.192553 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5621a0da-7dc0-4416-bfc0-4d7cbcd172b8 None None] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:49:24.193081 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5621a0da-7dc0-4416-bfc0-4d7cbcd172b8 None None] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:49:24.193081 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5621a0da-7dc0-4416-bfc0-4d7cbcd172b8 None None] PATH_INFO: `/` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:49:24.194474 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 757/1491] 10.222.0.25 () {54 vars in 927 bytes} [Sun Aug 7 10:49:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 07 10:49:24.200348 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db494fb5-4f81-45f2-9321-3898ba6b9561 None None] REQUEST_METHOD: `POST` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:49:24.200808 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db494fb5-4f81-45f2-9321-3898ba6b9561 None None] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:49:24.200905 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db494fb5-4f81-45f2-9321-3898ba6b9561 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:49:24.266534 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.auth.core [None req-db494fb5-4f81-45f2-9321-3898ba6b9561 None None] MFA Rules not processed for user `bac4e6b76fe64217ac8fa44e71cffe6a`. Rule list: `[]` (Enabled: `True`). {{(pid=80143) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 10:49:24.267752 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-db494fb5-4f81-45f2-9321-3898ba6b9561 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 10:49:24.362939 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 735/1492] 10.222.0.25 () {58 vars in 1024 bytes} [Sun Aug 7 10:49:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 07 10:49:24.404186 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5621a0da-7dc0-4416-bfc0-4d7cbcd172b8 None None] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 10:49:24.405733 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.fernet_utils [None req-5621a0da-7dc0-4416-bfc0-4d7cbcd172b8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80144) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 10:49:24.511529 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5621a0da-7dc0-4416-bfc0-4d7cbcd172b8 None None] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 10:49:24.584112 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e84f1790-d768-4109-b0d4-3bf9c4116511 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be1283c9bf47424c9aa3ad27cab03027', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6de1f3cac02c4bcdb05dd2337176033d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 10:49:24.584672 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e84f1790-d768-4109-b0d4-3bf9c4116511 service neutron] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:49:24.584958 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e84f1790-d768-4109-b0d4-3bf9c4116511 service neutron] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:49:24.585230 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e84f1790-d768-4109-b0d4-3bf9c4116511 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:49:24.587153 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.fernet_utils [None req-e84f1790-d768-4109-b0d4-3bf9c4116511 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80144) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 10:49:24.596254 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=100 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 10:49:24.696950 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e84f1790-d768-4109-b0d4-3bf9c4116511 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 10:49:24.698367 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e84f1790-d768-4109-b0d4-3bf9c4116511 service neutron] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 10:49:24.716243 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 758/1493] 10.222.0.25 () {60 vars in 1365 bytes} [Sun Aug 7 10:49:24 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 313 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 07 10:49:34.360899 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=82 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 10:49:34.714038 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=50 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 10:50:41.916239 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc58f363-dbaa-420a-ae83-d3a96b28f753 None None] REQUEST_METHOD: `POST` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:50:41.916831 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc58f363-dbaa-420a-ae83-d3a96b28f753 None None] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:50:41.916831 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc58f363-dbaa-420a-ae83-d3a96b28f753 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:50:41.989568 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.auth.core [None req-dc58f363-dbaa-420a-ae83-d3a96b28f753 None None] MFA Rules not processed for user `bac4e6b76fe64217ac8fa44e71cffe6a`. Rule list: `[]` (Enabled: `True`). {{(pid=80143) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 10:50:41.990752 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-dc58f363-dbaa-420a-ae83-d3a96b28f753 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 10:50:42.085424 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 736/1494] 10.222.0.25 () {58 vars in 958 bytes} [Sun Aug 7 10:50:41 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 170 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 07 10:50:42.093609 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66479dc8-ce7b-4156-a23b-1727fb02c1d3 None None] REQUEST_METHOD: `POST` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:50:42.094100 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66479dc8-ce7b-4156-a23b-1727fb02c1d3 None None] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:50:42.094209 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66479dc8-ce7b-4156-a23b-1727fb02c1d3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:50:42.170365 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.auth.core [None req-66479dc8-ce7b-4156-a23b-1727fb02c1d3 None None] MFA Rules not processed for user `bac4e6b76fe64217ac8fa44e71cffe6a`. Rule list: `[]` (Enabled: `True`). {{(pid=80144) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 10:50:42.171507 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.fernet_utils [None req-66479dc8-ce7b-4156-a23b-1727fb02c1d3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80144) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 10:50:42.269252 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 759/1495] 10.222.0.25 () {58 vars in 958 bytes} [Sun Aug 7 10:50:42 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 177 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 07 10:50:42.274245 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc58f363-dbaa-420a-ae83-d3a96b28f753 None None] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 10:50:42.275473 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-dc58f363-dbaa-420a-ae83-d3a96b28f753 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 10:50:42.286952 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc58f363-dbaa-420a-ae83-d3a96b28f753 None None] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 10:50:42.351742 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=100 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 10:50:42.391953 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-704193e7-7640-47cd-a78f-cbeeb44ffcb1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5ac47e3b09f43d19a23a0f457fe9e23', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 10:50:42.392581 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-704193e7-7640-47cd-a78f-cbeeb44ffcb1 admin admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:50:42.392863 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-704193e7-7640-47cd-a78f-cbeeb44ffcb1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:50:42.393137 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-704193e7-7640-47cd-a78f-cbeeb44ffcb1 admin admin] PATH_INFO: `/v3/domains` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:50:42.393804 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-704193e7-7640-47cd-a78f-cbeeb44ffcb1 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80143) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 10:50:42.394108 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-704193e7-7640-47cd-a78f-cbeeb44ffcb1 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 10:50:42.395440 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: WARNING py.warnings [None req-704193e7-7640-47cd-a78f-cbeeb44ffcb1 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_domains": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Aug 07 10:50:42.395440 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: warnings.warn(msg) Aug 07 10:50:42.395440 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: Aug 07 10:50:42.396353 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-704193e7-7640-47cd-a78f-cbeeb44ffcb1 admin admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 10:50:42.408564 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 737/1496] 10.222.0.25 () {58 vars in 1153 bytes} [Sun Aug 7 10:50:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 10:50:42.416991 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66479dc8-ce7b-4156-a23b-1727fb02c1d3 None None] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 10:50:42.426775 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66479dc8-ce7b-4156-a23b-1727fb02c1d3 None None] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 10:50:42.487456 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=100 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 10:50:42.521814 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b839c859-2b08-4ff1-ac61-762a8f984399 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5ac47e3b09f43d19a23a0f457fe9e23', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 10:50:42.522419 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b839c859-2b08-4ff1-ac61-762a8f984399 admin admin] REQUEST_METHOD: `POST` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:50:42.522698 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b839c859-2b08-4ff1-ac61-762a8f984399 admin admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:50:42.522998 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b839c859-2b08-4ff1-ac61-762a8f984399 admin admin] PATH_INFO: `/v3/projects` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:50:42.523630 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b839c859-2b08-4ff1-ac61-762a8f984399 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 10:50:42.525341 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: WARNING py.warnings [None req-b839c859-2b08-4ff1-ac61-762a8f984399 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 07 10:50:42.525341 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: warnings.warn(msg) Aug 07 10:50:42.525341 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: Aug 07 10:50:42.526564 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b839c859-2b08-4ff1-ac61-762a8f984399 admin admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 10:50:42.581228 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 760/1497] 10.222.0.25 () {60 vars in 1153 bytes} [Sun Aug 7 10:50:42 2022] POST /identity/v3/projects => generated 379 bytes in 165 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 07 10:50:42.585675 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-704193e7-7640-47cd-a78f-cbeeb44ffcb1 admin admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 10:50:42.587237 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-704193e7-7640-47cd-a78f-cbeeb44ffcb1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 10:50:42.621824 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-704193e7-7640-47cd-a78f-cbeeb44ffcb1 admin admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 10:50:42.712013 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3131beed-557a-48b6-9bb9-beb9d355ed27 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5ac47e3b09f43d19a23a0f457fe9e23', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 10:50:42.712539 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3131beed-557a-48b6-9bb9-beb9d355ed27 admin admin] REQUEST_METHOD: `POST` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:50:42.712814 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3131beed-557a-48b6-9bb9-beb9d355ed27 admin admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:50:42.713084 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3131beed-557a-48b6-9bb9-beb9d355ed27 admin admin] PATH_INFO: `/v3/users` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:50:42.713691 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3131beed-557a-48b6-9bb9-beb9d355ed27 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 10:50:42.714684 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: WARNING py.warnings [None req-3131beed-557a-48b6-9bb9-beb9d355ed27 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 07 10:50:42.714684 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: warnings.warn(msg) Aug 07 10:50:42.714684 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: Aug 07 10:50:42.715919 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3131beed-557a-48b6-9bb9-beb9d355ed27 admin admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 10:50:42.758696 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 738/1498] 10.222.0.25 () {60 vars in 1144 bytes} [Sun Aug 7 10:50:42 2022] POST /identity/v3/users => generated 290 bytes in 174 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 07 10:50:42.762834 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b839c859-2b08-4ff1-ac61-762a8f984399 admin admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 10:50:42.764420 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.fernet_utils [None req-b839c859-2b08-4ff1-ac61-762a8f984399 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80144) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 10:50:42.825038 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b839c859-2b08-4ff1-ac61-762a8f984399 admin admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 10:50:42.908270 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 10:50:42.920289 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a94a27fe-ea56-4f57-b740-fedd952ebaaa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5ac47e3b09f43d19a23a0f457fe9e23', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 10:50:42.920901 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a94a27fe-ea56-4f57-b740-fedd952ebaaa admin admin] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:50:42.921179 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a94a27fe-ea56-4f57-b740-fedd952ebaaa admin admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:50:42.921449 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a94a27fe-ea56-4f57-b740-fedd952ebaaa admin admin] PATH_INFO: `/v3/roles` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:50:42.922115 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a94a27fe-ea56-4f57-b740-fedd952ebaaa admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 10:50:42.923136 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: WARNING py.warnings [None req-a94a27fe-ea56-4f57-b740-fedd952ebaaa admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_roles": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Aug 07 10:50:42.923136 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: warnings.warn(msg) Aug 07 10:50:42.923136 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: Aug 07 10:50:42.924019 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a94a27fe-ea56-4f57-b740-fedd952ebaaa admin admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 10:50:42.936771 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 761/1499] 10.222.0.25 () {58 vars in 1122 bytes} [Sun Aug 7 10:50:42 2022] GET /identity/v3/roles => generated 1420 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 07 10:50:42.940727 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3131beed-557a-48b6-9bb9-beb9d355ed27 admin admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 10:50:42.942280 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-3131beed-557a-48b6-9bb9-beb9d355ed27 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 10:50:42.953501 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3131beed-557a-48b6-9bb9-beb9d355ed27 admin admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 10:50:42.978664 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 10:50:43.047775 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b53fdb6-4d4c-4d81-877d-0ca750e2c560 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5ac47e3b09f43d19a23a0f457fe9e23', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 10:50:43.048381 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b53fdb6-4d4c-4d81-877d-0ca750e2c560 admin admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:50:43.048663 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b53fdb6-4d4c-4d81-877d-0ca750e2c560 admin admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:50:43.048935 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b53fdb6-4d4c-4d81-877d-0ca750e2c560 admin admin] PATH_INFO: `/v3/roles` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:50:43.049577 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b53fdb6-4d4c-4d81-877d-0ca750e2c560 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 10:50:43.050579 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: WARNING py.warnings [None req-2b53fdb6-4d4c-4d81-877d-0ca750e2c560 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_roles": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Aug 07 10:50:43.050579 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: warnings.warn(msg) Aug 07 10:50:43.050579 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: Aug 07 10:50:43.051530 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b53fdb6-4d4c-4d81-877d-0ca750e2c560 admin admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 10:50:43.063431 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 739/1500] 10.222.0.25 () {58 vars in 1122 bytes} [Sun Aug 7 10:50:42 2022] GET /identity/v3/roles => generated 1420 bytes in 123 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 07 10:50:43.067869 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a94a27fe-ea56-4f57-b740-fedd952ebaaa admin admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 10:50:43.069406 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.fernet_utils [None req-a94a27fe-ea56-4f57-b740-fedd952ebaaa admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80144) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 10:50:43.103461 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a94a27fe-ea56-4f57-b740-fedd952ebaaa admin admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 10:50:43.197670 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7164b1c7-8665-458a-9085-c03338e0535f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5ac47e3b09f43d19a23a0f457fe9e23', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 10:50:43.198327 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7164b1c7-8665-458a-9085-c03338e0535f admin admin] REQUEST_METHOD: `PUT` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:50:43.198607 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7164b1c7-8665-458a-9085-c03338e0535f admin admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:50:43.198882 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7164b1c7-8665-458a-9085-c03338e0535f admin admin] PATH_INFO: `/v3/projects/b526477e0a9d4206b23f083aa6596fb3/users/3d2120861c2b4b5f9c3b3ace1ffec245/roles/b98dfb3b9fee4644af2887a3ac3fd2e6` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:50:43.222080 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7164b1c7-8665-458a-9085-c03338e0535f admin admin] RBAC: Authorizing `identity:create_grant(project_id=b526477e0a9d4206b23f083aa6596fb3, user_id=3d2120861c2b4b5f9c3b3ace1ffec245, role_id=b98dfb3b9fee4644af2887a3ac3fd2e6)` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 10:50:43.223769 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: WARNING py.warnings [None req-7164b1c7-8665-458a-9085-c03338e0535f admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_grant": "(role:admin and system_scope:all) or ((role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.domain.id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.domain.id)s)) and (domain_id:%(target.role.domain_id)s or None:%(target.role.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 07 10:50:43.223769 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: warnings.warn(msg) Aug 07 10:50:43.223769 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: Aug 07 10:50:43.225673 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7164b1c7-8665-458a-9085-c03338e0535f admin admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 10:50:43.249523 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 762/1501] 10.222.0.25 () {60 vars in 1483 bytes} [Sun Aug 7 10:50:43 2022] PUT /identity/v3/projects/b526477e0a9d4206b23f083aa6596fb3/users/3d2120861c2b4b5f9c3b3ace1ffec245/roles/b98dfb3b9fee4644af2887a3ac3fd2e6 => generated 0 bytes in 182 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 07 10:50:43.261686 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da0a4d4a-5584-4c63-a73e-0810551da45f None None] REQUEST_METHOD: `POST` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:50:43.261981 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da0a4d4a-5584-4c63-a73e-0810551da45f None None] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:50:43.262279 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da0a4d4a-5584-4c63-a73e-0810551da45f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:50:43.318233 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.auth.core [None req-da0a4d4a-5584-4c63-a73e-0810551da45f None None] MFA Rules not processed for user `3d2120861c2b4b5f9c3b3ace1ffec245`. Rule list: `[]` (Enabled: `True`). {{(pid=80143) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 10:50:43.319504 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-da0a4d4a-5584-4c63-a73e-0810551da45f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 10:50:43.395375 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 740/1502] 10.222.0.25 () {58 vars in 958 bytes} [Sun Aug 7 10:50:43 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 135 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 07 10:50:43.408377 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bba3d055-0b62-4226-8e99-2536eff36d1c None None] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:50:43.408732 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bba3d055-0b62-4226-8e99-2536eff36d1c None None] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:50:43.409074 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bba3d055-0b62-4226-8e99-2536eff36d1c None None] PATH_INFO: `/` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:50:43.411098 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 763/1503] 10.222.0.25 () {54 vars in 964 bytes} [Sun Aug 7 10:50:43 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 07 10:50:43.416285 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8b9eafd-5a61-402a-a041-fcff4923690b None None] REQUEST_METHOD: `POST` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:50:43.416571 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8b9eafd-5a61-402a-a041-fcff4923690b None None] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:50:43.416844 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8b9eafd-5a61-402a-a041-fcff4923690b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:50:43.448688 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=100 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 10:50:43.493157 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.auth.core [None req-b8b9eafd-5a61-402a-a041-fcff4923690b None None] MFA Rules not processed for user `88dbd6dfd1164994a3e16da553561910`. Rule list: `[]` (Enabled: `True`). {{(pid=80143) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 10:50:43.494351 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-b8b9eafd-5a61-402a-a041-fcff4923690b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 10:50:43.568052 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 741/1504] 10.222.0.25 () {58 vars in 1061 bytes} [Sun Aug 7 10:50:43 2022] POST /identity/v3/auth/tokens => generated 3120 bytes in 153 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 07 10:50:43.575851 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bba3d055-0b62-4226-8e99-2536eff36d1c None None] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 10:50:43.577415 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.fernet_utils [None req-bba3d055-0b62-4226-8e99-2536eff36d1c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80144) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 10:50:43.627598 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 10:50:43.681212 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bba3d055-0b62-4226-8e99-2536eff36d1c None None] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 10:50:43.752374 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76cac2ab-a546-4f6a-8c1f-8e4097c9fca0 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88dbd6dfd1164994a3e16da553561910', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6de1f3cac02c4bcdb05dd2337176033d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 10:50:43.752971 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76cac2ab-a546-4f6a-8c1f-8e4097c9fca0 service cinder] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:50:43.753245 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76cac2ab-a546-4f6a-8c1f-8e4097c9fca0 service cinder] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:50:43.753512 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76cac2ab-a546-4f6a-8c1f-8e4097c9fca0 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:50:43.755270 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.fernet_utils [None req-76cac2ab-a546-4f6a-8c1f-8e4097c9fca0 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80144) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 10:50:43.852195 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76cac2ab-a546-4f6a-8c1f-8e4097c9fca0 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 10:50:43.853532 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76cac2ab-a546-4f6a-8c1f-8e4097c9fca0 service cinder] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 10:50:43.857012 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=100 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 10:50:43.872937 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 764/1505] 10.222.0.25 () {60 vars in 1365 bytes} [Sun Aug 7 10:50:43 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 298 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 07 10:50:43.920083 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24cc095f-8fbf-407a-8231-8bac9f1e22b1 None None] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:50:43.920367 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24cc095f-8fbf-407a-8231-8bac9f1e22b1 None None] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:50:43.920637 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24cc095f-8fbf-407a-8231-8bac9f1e22b1 None None] PATH_INFO: `/` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:50:43.921937 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 742/1506] 10.222.0.25 () {52 vars in 830 bytes} [Sun Aug 7 10:50:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 07 10:50:43.925539 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76cac2ab-a546-4f6a-8c1f-8e4097c9fca0 service cinder] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 10:50:43.927379 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.fernet_utils [None req-76cac2ab-a546-4f6a-8c1f-8e4097c9fca0 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80144) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 10:50:44.025539 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76cac2ab-a546-4f6a-8c1f-8e4097c9fca0 service cinder] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 10:50:44.120685 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5c24ff3-418f-4ef5-9ef4-f6d1ea4e7b9e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5ac47e3b09f43d19a23a0f457fe9e23', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 10:50:44.121290 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5c24ff3-418f-4ef5-9ef4-f6d1ea4e7b9e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:50:44.121509 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5c24ff3-418f-4ef5-9ef4-f6d1ea4e7b9e admin admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:50:44.121809 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5c24ff3-418f-4ef5-9ef4-f6d1ea4e7b9e admin admin] PATH_INFO: `/v3/users/3d2120861c2b4b5f9c3b3ace1ffec245` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:50:44.134275 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5c24ff3-418f-4ef5-9ef4-f6d1ea4e7b9e admin admin] RBAC: Authorizing `identity:delete_user(user_id=3d2120861c2b4b5f9c3b3ace1ffec245)` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 10:50:44.135334 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: WARNING py.warnings [None req-a5c24ff3-418f-4ef5-9ef4-f6d1ea4e7b9e admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 07 10:50:44.135334 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: warnings.warn(msg) Aug 07 10:50:44.135334 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: Aug 07 10:50:44.136546 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5c24ff3-418f-4ef5-9ef4-f6d1ea4e7b9e admin admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 10:50:44.146559 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=100 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 10:50:44.203699 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.notifications [None req-a5c24ff3-418f-4ef5-9ef4-f6d1ea4e7b9e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3d2120861c2b4b5f9c3b3ace1ffec245'} {{(pid=80144) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 10:50:44.217840 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.notifications [None req-a5c24ff3-418f-4ef5-9ef4-f6d1ea4e7b9e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3d2120861c2b4b5f9c3b3ace1ffec245'} {{(pid=80144) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 10:50:44.227387 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.notifications [None req-a5c24ff3-418f-4ef5-9ef4-f6d1ea4e7b9e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3d2120861c2b4b5f9c3b3ace1ffec245'} {{(pid=80144) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 10:50:44.227891 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.notifications [None req-a5c24ff3-418f-4ef5-9ef4-f6d1ea4e7b9e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3d2120861c2b4b5f9c3b3ace1ffec245'} {{(pid=80144) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 10:50:44.228311 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.notifications [None req-a5c24ff3-418f-4ef5-9ef4-f6d1ea4e7b9e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3d2120861c2b4b5f9c3b3ace1ffec245'} {{(pid=80144) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 10:50:44.256324 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 765/1507] 10.222.0.25 () {58 vars in 1224 bytes} [Sun Aug 7 10:50:43 2022] DELETE /identity/v3/users/3d2120861c2b4b5f9c3b3ace1ffec245 => generated 0 bytes in 331 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 07 10:50:44.260720 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52626996-b125-4c95-9469-b9a25cc5492a None None] REQUEST_METHOD: `POST` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:50:44.261207 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52626996-b125-4c95-9469-b9a25cc5492a None None] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:50:44.261306 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52626996-b125-4c95-9469-b9a25cc5492a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:50:44.322352 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.auth.core [None req-52626996-b125-4c95-9469-b9a25cc5492a None None] MFA Rules not processed for user `bac4e6b76fe64217ac8fa44e71cffe6a`. Rule list: `[]` (Enabled: `True`). {{(pid=80143) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 10:50:44.323581 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-52626996-b125-4c95-9469-b9a25cc5492a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 10:50:44.398984 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=100 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 10:50:44.412729 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 743/1508] 10.222.0.25 () {58 vars in 958 bytes} [Sun Aug 7 10:50:44 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 153 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 07 10:50:44.422267 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5c24ff3-418f-4ef5-9ef4-f6d1ea4e7b9e admin admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 10:50:44.423519 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.fernet_utils [None req-a5c24ff3-418f-4ef5-9ef4-f6d1ea4e7b9e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80144) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 10:50:44.524533 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5c24ff3-418f-4ef5-9ef4-f6d1ea4e7b9e admin admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 10:50:44.579032 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:DELETE=10,keystone:SELECT=89,keystone:INSERT=1 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 10:50:44.599902 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-848354f0-3830-4431-8d69-7ce708d6959a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be1283c9bf47424c9aa3ad27cab03027', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6de1f3cac02c4bcdb05dd2337176033d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 10:50:44.600514 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-848354f0-3830-4431-8d69-7ce708d6959a service neutron] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:50:44.600790 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-848354f0-3830-4431-8d69-7ce708d6959a service neutron] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:50:44.601063 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-848354f0-3830-4431-8d69-7ce708d6959a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:50:44.602644 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.fernet_utils [None req-848354f0-3830-4431-8d69-7ce708d6959a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80144) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 10:50:44.707956 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-848354f0-3830-4431-8d69-7ce708d6959a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 10:50:44.709271 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-848354f0-3830-4431-8d69-7ce708d6959a service neutron] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 10:50:44.727852 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 766/1509] 10.222.0.25 () {60 vars in 1365 bytes} [Sun Aug 7 10:50:44 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 306 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 07 10:50:45.081068 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52626996-b125-4c95-9469-b9a25cc5492a None None] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 10:50:45.083130 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-52626996-b125-4c95-9469-b9a25cc5492a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 10:50:45.094789 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52626996-b125-4c95-9469-b9a25cc5492a None None] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 10:50:45.193339 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b87f1db5-8147-46d8-a0b1-e409dc3d5604 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5ac47e3b09f43d19a23a0f457fe9e23', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 10:50:45.193974 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b87f1db5-8147-46d8-a0b1-e409dc3d5604 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:50:45.194254 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b87f1db5-8147-46d8-a0b1-e409dc3d5604 admin admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:50:45.194525 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b87f1db5-8147-46d8-a0b1-e409dc3d5604 admin admin] PATH_INFO: `/v3/projects/b526477e0a9d4206b23f083aa6596fb3` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:50:45.204650 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b87f1db5-8147-46d8-a0b1-e409dc3d5604 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b526477e0a9d4206b23f083aa6596fb3)` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 10:50:45.205836 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: WARNING py.warnings [None req-b87f1db5-8147-46d8-a0b1-e409dc3d5604 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 07 10:50:45.205836 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: warnings.warn(msg) Aug 07 10:50:45.205836 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: Aug 07 10:50:45.207659 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b87f1db5-8147-46d8-a0b1-e409dc3d5604 admin admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 10:50:45.265558 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-b87f1db5-8147-46d8-a0b1-e409dc3d5604 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b526477e0a9d4206b23f083aa6596fb3', 'request_id': 'req-b87f1db5-8147-46d8-a0b1-e409dc3d5604'} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 10:50:45.271878 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-b87f1db5-8147-46d8-a0b1-e409dc3d5604 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b526477e0a9d4206b23f083aa6596fb3', 'request_id': 'req-b87f1db5-8147-46d8-a0b1-e409dc3d5604'} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 10:50:45.285950 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-b87f1db5-8147-46d8-a0b1-e409dc3d5604 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b526477e0a9d4206b23f083aa6596fb3', 'request_id': 'req-b87f1db5-8147-46d8-a0b1-e409dc3d5604'} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 10:50:45.311894 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-b87f1db5-8147-46d8-a0b1-e409dc3d5604 admin admin] The token cache is being invalidate because project b526477e0a9d4206b23f083aa6596fb3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80143) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 07 10:50:45.312186 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-b87f1db5-8147-46d8-a0b1-e409dc3d5604 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 10:50:45.312615 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-b87f1db5-8147-46d8-a0b1-e409dc3d5604 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 10:50:45.313979 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 744/1510] 10.222.0.25 () {58 vars in 1233 bytes} [Sun Aug 7 10:50:45 2022] DELETE /identity/v3/projects/b526477e0a9d4206b23f083aa6596fb3 => generated 0 bytes in 234 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 07 10:50:54.725695 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=62 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 10:50:55.309908 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=84,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 10:52:11.280695 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c254615-8756-4dcf-874f-f51bd14176fd None None] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:52:11.281323 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c254615-8756-4dcf-874f-f51bd14176fd None None] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:52:11.281323 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c254615-8756-4dcf-874f-f51bd14176fd None None] PATH_INFO: `/` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:52:11.283241 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 767/1511] 127.0.0.1 () {54 vars in 901 bytes} [Sun Aug 7 10:52:11 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 07 10:52:11.288123 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2690470-44d8-4b9a-aeff-c633a285e3f7 None None] REQUEST_METHOD: `POST` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:52:11.288568 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2690470-44d8-4b9a-aeff-c633a285e3f7 None None] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:52:11.288684 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2690470-44d8-4b9a-aeff-c633a285e3f7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:52:11.369727 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.auth.core [None req-e2690470-44d8-4b9a-aeff-c633a285e3f7 None None] MFA Rules not processed for user `bac4e6b76fe64217ac8fa44e71cffe6a`. Rule list: `[]` (Enabled: `True`). {{(pid=80143) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 10:52:11.370916 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-e2690470-44d8-4b9a-aeff-c633a285e3f7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 10:52:11.472314 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 745/1512] 127.0.0.1 () {58 vars in 998 bytes} [Sun Aug 7 10:52:11 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 185 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 07 10:52:11.625088 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c254615-8756-4dcf-874f-f51bd14176fd None None] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 10:52:11.627310 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.fernet_utils [None req-1c254615-8756-4dcf-874f-f51bd14176fd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80144) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 10:52:11.747927 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c254615-8756-4dcf-874f-f51bd14176fd None None] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 10:52:11.753608 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb698b38-b40d-4b6f-bbf2-8c2cf124ca1d service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31d16d1b96104b4eb690398432b9829e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6de1f3cac02c4bcdb05dd2337176033d', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'reader', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 10:52:11.754208 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb698b38-b40d-4b6f-bbf2-8c2cf124ca1d service nova] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:52:11.754492 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb698b38-b40d-4b6f-bbf2-8c2cf124ca1d service nova] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:52:11.754770 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb698b38-b40d-4b6f-bbf2-8c2cf124ca1d service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:52:11.762704 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb698b38-b40d-4b6f-bbf2-8c2cf124ca1d service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 10:52:11.764305 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb698b38-b40d-4b6f-bbf2-8c2cf124ca1d service nova] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 10:52:11.774141 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 768/1513] 10.222.0.25 () {60 vars in 1365 bytes} [Sun Aug 7 10:52:11 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 149 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 07 10:52:12.597605 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0115fa42-7b63-402f-a483-2e5fb6fa1734 None None] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:52:12.598163 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0115fa42-7b63-402f-a483-2e5fb6fa1734 None None] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:52:12.598163 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0115fa42-7b63-402f-a483-2e5fb6fa1734 None None] PATH_INFO: `/` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:52:12.599557 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 746/1514] 127.0.0.1 () {54 vars in 901 bytes} [Sun Aug 7 10:52:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 07 10:52:12.604183 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73c50104-0635-41dc-a980-fa703532e307 None None] REQUEST_METHOD: `POST` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:52:12.604608 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73c50104-0635-41dc-a980-fa703532e307 None None] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:52:12.604726 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73c50104-0635-41dc-a980-fa703532e307 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:52:12.625219 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.auth.core [None req-73c50104-0635-41dc-a980-fa703532e307 None None] MFA Rules not processed for user `bac4e6b76fe64217ac8fa44e71cffe6a`. Rule list: `[]` (Enabled: `True`). {{(pid=80144) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 10:52:12.626915 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.fernet_utils [None req-73c50104-0635-41dc-a980-fa703532e307 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80144) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 10:52:12.635552 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 769/1515] 127.0.0.1 () {58 vars in 998 bytes} [Sun Aug 7 10:52:12 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 07 10:52:12.786260 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0115fa42-7b63-402f-a483-2e5fb6fa1734 None None] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 10:52:12.800802 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0115fa42-7b63-402f-a483-2e5fb6fa1734 None None] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 10:52:12.806043 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8b67e69-0356-4bf0-a3d0-0e261c0aab85 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31d16d1b96104b4eb690398432b9829e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6de1f3cac02c4bcdb05dd2337176033d', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'reader', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 10:52:12.806632 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8b67e69-0356-4bf0-a3d0-0e261c0aab85 service nova] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:52:12.806907 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8b67e69-0356-4bf0-a3d0-0e261c0aab85 service nova] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:52:12.807204 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8b67e69-0356-4bf0-a3d0-0e261c0aab85 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:52:12.815202 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8b67e69-0356-4bf0-a3d0-0e261c0aab85 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 10:52:12.816757 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8b67e69-0356-4bf0-a3d0-0e261c0aab85 service nova] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 10:52:12.825804 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 747/1516] 10.222.0.25 () {60 vars in 1365 bytes} [Sun Aug 7 10:52:12 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 07 10:52:13.083048 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-503de9d9-f5cd-4fed-9d84-f0b886fb737d None None] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:52:13.083255 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-503de9d9-f5cd-4fed-9d84-f0b886fb737d None None] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:52:13.083516 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-503de9d9-f5cd-4fed-9d84-f0b886fb737d None None] PATH_INFO: `/v3` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:52:13.084910 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 770/1517] 10.222.0.36 () {54 vars in 925 bytes} [Sun Aug 7 10:52:13 2022] GET /identity/v3 => generated 255 bytes in 3 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 10:52:13.091492 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9b3153c-88ea-491c-81e8-ff2b6920eda1 None None] REQUEST_METHOD: `POST` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:52:13.091782 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9b3153c-88ea-491c-81e8-ff2b6920eda1 None None] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:52:13.092062 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9b3153c-88ea-491c-81e8-ff2b6920eda1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:52:13.153105 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.auth.core [None req-c9b3153c-88ea-491c-81e8-ff2b6920eda1 None None] MFA Rules not processed for user `6ee2e4c08af04c219ed395d7ca7085a6`. Rule list: `[]` (Enabled: `True`). {{(pid=80143) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 10:52:13.154304 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-c9b3153c-88ea-491c-81e8-ff2b6920eda1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 10:52:13.171421 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=100 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 10:52:13.225915 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 748/1518] 10.222.0.36 () {58 vars in 1015 bytes} [Sun Aug 7 10:52:13 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 135 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 07 10:52:22.620665 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=56 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 10:52:23.223131 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=23 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 10:52:23.355040 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e44d516c-68f0-4f24-ab7f-ec4ab3cc4a0a None None] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:52:23.355333 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e44d516c-68f0-4f24-ab7f-ec4ab3cc4a0a None None] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:52:23.355642 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e44d516c-68f0-4f24-ab7f-ec4ab3cc4a0a None None] PATH_INFO: `/` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:52:23.356983 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 771/1519] 10.222.0.25 () {54 vars in 961 bytes} [Sun Aug 7 10:52:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 07 10:52:23.362317 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3337f77d-749d-4fce-8924-8bbbc3408f12 None None] REQUEST_METHOD: `POST` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:52:23.362611 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3337f77d-749d-4fce-8924-8bbbc3408f12 None None] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:52:23.362901 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3337f77d-749d-4fce-8924-8bbbc3408f12 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:52:23.383088 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.auth.core [None req-3337f77d-749d-4fce-8924-8bbbc3408f12 None None] MFA Rules not processed for user `6ee2e4c08af04c219ed395d7ca7085a6`. Rule list: `[]` (Enabled: `True`). {{(pid=80143) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 10:52:23.384289 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-3337f77d-749d-4fce-8924-8bbbc3408f12 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 10:52:23.393394 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 749/1520] 10.222.0.25 () {58 vars in 1058 bytes} [Sun Aug 7 10:52:23 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 07 10:52:23.400978 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e44d516c-68f0-4f24-ab7f-ec4ab3cc4a0a None None] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 10:52:23.414980 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e44d516c-68f0-4f24-ab7f-ec4ab3cc4a0a None None] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 10:52:23.422375 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-345d1807-f68d-4170-9771-9975f1be844a service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6ee2e4c08af04c219ed395d7ca7085a6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6de1f3cac02c4bcdb05dd2337176033d', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 10:52:23.423516 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-345d1807-f68d-4170-9771-9975f1be844a service placement] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:52:23.423797 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-345d1807-f68d-4170-9771-9975f1be844a service placement] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:52:23.424073 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-345d1807-f68d-4170-9771-9975f1be844a service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:52:23.432900 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-345d1807-f68d-4170-9771-9975f1be844a service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 10:52:23.434467 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-345d1807-f68d-4170-9771-9975f1be844a service placement] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 10:52:23.444991 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 772/1521] 10.222.0.25 () {60 vars in 1365 bytes} [Sun Aug 7 10:52:23 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 07 10:52:24.057902 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b93b837-7c9c-46d9-bdf7-21e36e1a1e1c None None] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:52:24.058267 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b93b837-7c9c-46d9-bdf7-21e36e1a1e1c None None] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:52:24.058549 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b93b837-7c9c-46d9-bdf7-21e36e1a1e1c None None] PATH_INFO: `/` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:52:24.060022 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 750/1522] 127.0.0.1 () {54 vars in 901 bytes} [Sun Aug 7 10:52:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 07 10:52:24.065415 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c95115c7-b5a7-4507-8e26-83503c291cc2 None None] REQUEST_METHOD: `POST` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:52:24.065885 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c95115c7-b5a7-4507-8e26-83503c291cc2 None None] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:52:24.065981 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c95115c7-b5a7-4507-8e26-83503c291cc2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:52:24.085022 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.auth.core [None req-c95115c7-b5a7-4507-8e26-83503c291cc2 None None] MFA Rules not processed for user `bac4e6b76fe64217ac8fa44e71cffe6a`. Rule list: `[]` (Enabled: `True`). {{(pid=80144) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 10:52:24.086207 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.fernet_utils [None req-c95115c7-b5a7-4507-8e26-83503c291cc2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80144) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 10:52:24.095243 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 773/1523] 127.0.0.1 () {58 vars in 998 bytes} [Sun Aug 7 10:52:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 07 10:52:24.253261 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b93b837-7c9c-46d9-bdf7-21e36e1a1e1c None None] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 10:52:24.269024 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b93b837-7c9c-46d9-bdf7-21e36e1a1e1c None None] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 10:52:24.274440 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53b25d5a-d192-4e5c-bc6e-ba91262c3717 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31d16d1b96104b4eb690398432b9829e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6de1f3cac02c4bcdb05dd2337176033d', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'reader', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 10:52:24.275069 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53b25d5a-d192-4e5c-bc6e-ba91262c3717 service nova] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:52:24.275351 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53b25d5a-d192-4e5c-bc6e-ba91262c3717 service nova] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:52:24.275624 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53b25d5a-d192-4e5c-bc6e-ba91262c3717 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:52:24.284485 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53b25d5a-d192-4e5c-bc6e-ba91262c3717 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 10:52:24.286058 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53b25d5a-d192-4e5c-bc6e-ba91262c3717 service nova] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 10:52:24.295263 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 751/1524] 10.222.0.25 () {60 vars in 1365 bytes} [Sun Aug 7 10:52:24 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 07 10:52:25.118731 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a975c871-9ce5-4e0e-8918-8416c2c393b2 None None] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:52:25.119291 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a975c871-9ce5-4e0e-8918-8416c2c393b2 None None] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:52:25.119403 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a975c871-9ce5-4e0e-8918-8416c2c393b2 None None] PATH_INFO: `/` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:52:25.120684 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 774/1525] 127.0.0.1 () {54 vars in 901 bytes} [Sun Aug 7 10:52:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 07 10:52:25.125545 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75a64d81-5308-47fb-bf5a-8615bb27c071 None None] REQUEST_METHOD: `POST` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:52:25.125861 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75a64d81-5308-47fb-bf5a-8615bb27c071 None None] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:52:25.126142 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75a64d81-5308-47fb-bf5a-8615bb27c071 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:52:25.145897 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.auth.core [None req-75a64d81-5308-47fb-bf5a-8615bb27c071 None None] MFA Rules not processed for user `bac4e6b76fe64217ac8fa44e71cffe6a`. Rule list: `[]` (Enabled: `True`). {{(pid=80143) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 10:52:25.147047 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-75a64d81-5308-47fb-bf5a-8615bb27c071 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 10:52:25.154787 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 752/1526] 127.0.0.1 () {58 vars in 998 bytes} [Sun Aug 7 10:52:25 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 07 10:52:25.305866 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a975c871-9ce5-4e0e-8918-8416c2c393b2 None None] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 10:52:25.320465 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a975c871-9ce5-4e0e-8918-8416c2c393b2 None None] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 10:52:25.325770 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a79683f-d8fe-48e8-8162-987fa67c8921 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31d16d1b96104b4eb690398432b9829e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6de1f3cac02c4bcdb05dd2337176033d', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'reader', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 10:52:25.326358 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a79683f-d8fe-48e8-8162-987fa67c8921 service nova] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:52:25.326636 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a79683f-d8fe-48e8-8162-987fa67c8921 service nova] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:52:25.326909 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a79683f-d8fe-48e8-8162-987fa67c8921 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:52:25.335577 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a79683f-d8fe-48e8-8162-987fa67c8921 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 10:52:25.337167 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a79683f-d8fe-48e8-8162-987fa67c8921 service nova] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 10:52:25.346602 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 775/1527] 10.222.0.25 () {60 vars in 1365 bytes} [Sun Aug 7 10:52:25 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 07 10:52:29.248352 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01d650c1-99f0-4982-b64d-9fac6f4697a7 None None] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:52:29.248941 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01d650c1-99f0-4982-b64d-9fac6f4697a7 None None] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:52:29.249133 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01d650c1-99f0-4982-b64d-9fac6f4697a7 None None] PATH_INFO: `/` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:52:29.250424 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 753/1528] 127.0.0.1 () {54 vars in 917 bytes} [Sun Aug 7 10:52:29 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 07 10:52:29.256399 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf8973f7-2c09-431a-a242-1aadff11f848 None None] REQUEST_METHOD: `POST` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:52:29.256811 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf8973f7-2c09-431a-a242-1aadff11f848 None None] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:52:29.256989 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf8973f7-2c09-431a-a242-1aadff11f848 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:52:29.276586 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.auth.core [None req-bf8973f7-2c09-431a-a242-1aadff11f848 None None] MFA Rules not processed for user `bac4e6b76fe64217ac8fa44e71cffe6a`. Rule list: `[]` (Enabled: `True`). {{(pid=80144) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 10:52:29.277771 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.fernet_utils [None req-bf8973f7-2c09-431a-a242-1aadff11f848 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80144) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 10:52:29.285657 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 776/1529] 127.0.0.1 () {58 vars in 1014 bytes} [Sun Aug 7 10:52:29 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 07 10:52:29.358006 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01d650c1-99f0-4982-b64d-9fac6f4697a7 None None] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 10:52:29.360160 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-01d650c1-99f0-4982-b64d-9fac6f4697a7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 10:52:29.441352 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01d650c1-99f0-4982-b64d-9fac6f4697a7 None None] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 10:52:29.444923 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9959664-0979-4e43-9f8d-2d69191d6af5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be1283c9bf47424c9aa3ad27cab03027', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6de1f3cac02c4bcdb05dd2337176033d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 10:52:29.445565 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9959664-0979-4e43-9f8d-2d69191d6af5 service neutron] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:52:29.445848 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9959664-0979-4e43-9f8d-2d69191d6af5 service neutron] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:52:29.446125 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9959664-0979-4e43-9f8d-2d69191d6af5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:52:29.455193 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9959664-0979-4e43-9f8d-2d69191d6af5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 10:52:29.456639 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9959664-0979-4e43-9f8d-2d69191d6af5 service neutron] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 10:52:29.466151 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 754/1530] 10.222.0.25 () {60 vars in 1365 bytes} [Sun Aug 7 10:52:29 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 109 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 07 10:52:31.568175 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cec5360-1235-4594-b357-b5a81cca6a46 None None] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:52:31.568704 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cec5360-1235-4594-b357-b5a81cca6a46 None None] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:52:31.568818 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cec5360-1235-4594-b357-b5a81cca6a46 None None] PATH_INFO: `/` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:52:31.570093 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 777/1531] 127.0.0.1 () {54 vars in 917 bytes} [Sun Aug 7 10:52:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 07 10:52:31.575338 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-846c1f93-6343-443f-9bce-368c5dedd85d None None] REQUEST_METHOD: `POST` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:52:31.575778 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-846c1f93-6343-443f-9bce-368c5dedd85d None None] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:52:31.575872 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-846c1f93-6343-443f-9bce-368c5dedd85d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:52:31.596435 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.auth.core [None req-846c1f93-6343-443f-9bce-368c5dedd85d None None] MFA Rules not processed for user `bac4e6b76fe64217ac8fa44e71cffe6a`. Rule list: `[]` (Enabled: `True`). {{(pid=80143) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 10:52:31.597634 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-846c1f93-6343-443f-9bce-368c5dedd85d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 10:52:31.605363 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 755/1532] 127.0.0.1 () {58 vars in 1014 bytes} [Sun Aug 7 10:52:31 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 07 10:52:31.642320 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cec5360-1235-4594-b357-b5a81cca6a46 None None] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 10:52:31.653134 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cec5360-1235-4594-b357-b5a81cca6a46 None None] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 10:52:31.656632 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cac94776-7ded-4612-a39c-16ab82fca622 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be1283c9bf47424c9aa3ad27cab03027', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6de1f3cac02c4bcdb05dd2337176033d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 10:52:31.657283 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cac94776-7ded-4612-a39c-16ab82fca622 service neutron] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:52:31.657565 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cac94776-7ded-4612-a39c-16ab82fca622 service neutron] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:52:31.657841 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cac94776-7ded-4612-a39c-16ab82fca622 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:52:31.666619 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cac94776-7ded-4612-a39c-16ab82fca622 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 10:52:31.668076 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cac94776-7ded-4612-a39c-16ab82fca622 service neutron] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 10:52:31.677936 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 778/1533] 10.222.0.25 () {60 vars in 1365 bytes} [Sun Aug 7 10:52:31 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 07 10:52:35.163749 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49d83fd7-73d0-4dc9-9736-27eff19c3c50 None None] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:52:35.164350 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49d83fd7-73d0-4dc9-9736-27eff19c3c50 None None] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:52:35.164350 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49d83fd7-73d0-4dc9-9736-27eff19c3c50 None None] PATH_INFO: `/` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:52:35.165682 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 756/1534] 127.0.0.1 () {54 vars in 917 bytes} [Sun Aug 7 10:52:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 07 10:52:35.171161 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ae3c3a2-e1e9-4616-8952-e15bf1e75d92 None None] REQUEST_METHOD: `POST` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:52:35.171571 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ae3c3a2-e1e9-4616-8952-e15bf1e75d92 None None] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:52:35.171716 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ae3c3a2-e1e9-4616-8952-e15bf1e75d92 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:52:35.191482 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.auth.core [None req-6ae3c3a2-e1e9-4616-8952-e15bf1e75d92 None None] MFA Rules not processed for user `bac4e6b76fe64217ac8fa44e71cffe6a`. Rule list: `[]` (Enabled: `True`). {{(pid=80144) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 10:52:35.192653 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.fernet_utils [None req-6ae3c3a2-e1e9-4616-8952-e15bf1e75d92 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80144) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 10:52:35.200404 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 779/1535] 127.0.0.1 () {58 vars in 1014 bytes} [Sun Aug 7 10:52:35 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 07 10:52:35.502502 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49d83fd7-73d0-4dc9-9736-27eff19c3c50 None None] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 10:52:35.504171 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-49d83fd7-73d0-4dc9-9736-27eff19c3c50 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 10:52:35.582069 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49d83fd7-73d0-4dc9-9736-27eff19c3c50 None None] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 10:52:35.585580 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0452302-1bdc-4912-afc9-3c8f2c2a65f6 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88dbd6dfd1164994a3e16da553561910', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6de1f3cac02c4bcdb05dd2337176033d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 10:52:35.586176 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0452302-1bdc-4912-afc9-3c8f2c2a65f6 service cinder] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:52:35.586453 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0452302-1bdc-4912-afc9-3c8f2c2a65f6 service cinder] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:52:35.586724 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0452302-1bdc-4912-afc9-3c8f2c2a65f6 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:52:35.596209 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0452302-1bdc-4912-afc9-3c8f2c2a65f6 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 10:52:35.597616 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0452302-1bdc-4912-afc9-3c8f2c2a65f6 service cinder] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 10:52:35.606708 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 757/1536] 10.222.0.25 () {60 vars in 1365 bytes} [Sun Aug 7 10:52:35 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 105 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 07 10:52:37.623055 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a23a3ac6-e4cf-4bf0-a466-1d3afd175473 None None] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:52:37.623590 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a23a3ac6-e4cf-4bf0-a466-1d3afd175473 None None] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:52:37.623590 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a23a3ac6-e4cf-4bf0-a466-1d3afd175473 None None] PATH_INFO: `/` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:52:37.624922 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 780/1537] 127.0.0.1 () {54 vars in 917 bytes} [Sun Aug 7 10:52:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 07 10:52:37.630752 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84c0250b-7c2a-45a4-85da-839977ed580f None None] REQUEST_METHOD: `POST` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:52:37.631212 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84c0250b-7c2a-45a4-85da-839977ed580f None None] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:52:37.631340 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84c0250b-7c2a-45a4-85da-839977ed580f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:52:37.651281 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.auth.core [None req-84c0250b-7c2a-45a4-85da-839977ed580f None None] MFA Rules not processed for user `bac4e6b76fe64217ac8fa44e71cffe6a`. Rule list: `[]` (Enabled: `True`). {{(pid=80143) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 10:52:37.651946 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-84c0250b-7c2a-45a4-85da-839977ed580f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 10:52:37.659745 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 758/1538] 127.0.0.1 () {58 vars in 1014 bytes} [Sun Aug 7 10:52:37 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 07 10:52:37.958435 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a23a3ac6-e4cf-4bf0-a466-1d3afd175473 None None] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 10:52:37.960115 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.fernet_utils [None req-a23a3ac6-e4cf-4bf0-a466-1d3afd175473 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80144) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 10:52:37.973382 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a23a3ac6-e4cf-4bf0-a466-1d3afd175473 None None] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 10:52:37.976849 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-968113cf-8303-41cc-898e-91b854abe2da service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88dbd6dfd1164994a3e16da553561910', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6de1f3cac02c4bcdb05dd2337176033d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 10:52:37.977422 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-968113cf-8303-41cc-898e-91b854abe2da service cinder] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:52:37.977699 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-968113cf-8303-41cc-898e-91b854abe2da service cinder] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:52:37.977968 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-968113cf-8303-41cc-898e-91b854abe2da service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:52:37.987016 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-968113cf-8303-41cc-898e-91b854abe2da service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 10:52:37.988416 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-968113cf-8303-41cc-898e-91b854abe2da service cinder] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 10:52:37.997610 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 781/1539] 10.222.0.25 () {60 vars in 1365 bytes} [Sun Aug 7 10:52:37 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 07 10:52:47.646186 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=88 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 10:52:47.993586 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=33 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 10:54:01.775997 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cda818a4-611a-4f85-8451-27522cdd2a39 None None] REQUEST_METHOD: `POST` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:54:01.777298 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cda818a4-611a-4f85-8451-27522cdd2a39 None None] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:54:01.777795 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cda818a4-611a-4f85-8451-27522cdd2a39 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:54:01.806243 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.auth.core [None req-cda818a4-611a-4f85-8451-27522cdd2a39 None None] MFA Rules not processed for user `bac4e6b76fe64217ac8fa44e71cffe6a`. Rule list: `[]` (Enabled: `True`). {{(pid=80143) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 10:54:01.808005 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-cda818a4-611a-4f85-8451-27522cdd2a39 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 10:54:01.823375 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 759/1540] 10.222.0.25 () {58 vars in 958 bytes} [Sun Aug 7 10:54:01 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 48 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 07 10:54:01.831679 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2ba2934-1cd8-4599-acf3-ff06b50db1db None None] REQUEST_METHOD: `POST` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:54:01.833086 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2ba2934-1cd8-4599-acf3-ff06b50db1db None None] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:54:01.833086 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2ba2934-1cd8-4599-acf3-ff06b50db1db None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:54:01.862611 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.auth.core [None req-a2ba2934-1cd8-4599-acf3-ff06b50db1db None None] MFA Rules not processed for user `bac4e6b76fe64217ac8fa44e71cffe6a`. Rule list: `[]` (Enabled: `True`). {{(pid=80144) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 10:54:01.864235 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.fernet_utils [None req-a2ba2934-1cd8-4599-acf3-ff06b50db1db None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80144) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 10:54:01.874358 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 782/1541] 10.222.0.25 () {58 vars in 958 bytes} [Sun Aug 7 10:54:01 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 44 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 07 10:54:01.879492 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cda818a4-611a-4f85-8451-27522cdd2a39 None None] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 10:54:01.890484 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cda818a4-611a-4f85-8451-27522cdd2a39 None None] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 10:54:01.899840 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14162097-4a04-4373-94f6-9476aad58dcd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5ac47e3b09f43d19a23a0f457fe9e23', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 10:54:01.900866 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14162097-4a04-4373-94f6-9476aad58dcd admin admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:54:01.901373 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14162097-4a04-4373-94f6-9476aad58dcd admin admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:54:01.901856 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14162097-4a04-4373-94f6-9476aad58dcd admin admin] PATH_INFO: `/v3/domains` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:54:01.902716 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14162097-4a04-4373-94f6-9476aad58dcd admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80143) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 10:54:01.903257 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14162097-4a04-4373-94f6-9476aad58dcd admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 10:54:01.905350 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14162097-4a04-4373-94f6-9476aad58dcd admin admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 10:54:01.923212 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 760/1542] 10.222.0.25 () {58 vars in 1153 bytes} [Sun Aug 7 10:54:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 10:54:01.931708 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2ba2934-1cd8-4599-acf3-ff06b50db1db None None] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 10:54:01.942315 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2ba2934-1cd8-4599-acf3-ff06b50db1db None None] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 10:54:01.950912 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a305bb1-db8a-401f-a6c1-22223065c181 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5ac47e3b09f43d19a23a0f457fe9e23', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 10:54:01.951909 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a305bb1-db8a-401f-a6c1-22223065c181 admin admin] REQUEST_METHOD: `POST` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:54:01.952418 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a305bb1-db8a-401f-a6c1-22223065c181 admin admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:54:01.952906 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a305bb1-db8a-401f-a6c1-22223065c181 admin admin] PATH_INFO: `/v3/projects` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:54:01.953817 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a305bb1-db8a-401f-a6c1-22223065c181 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 10:54:01.956594 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a305bb1-db8a-401f-a6c1-22223065c181 admin admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 10:54:01.978036 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6c4fd35-d24e-494f-afbb-386222059784 None None] REQUEST_METHOD: `POST` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:54:01.978697 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6c4fd35-d24e-494f-afbb-386222059784 None None] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:54:01.979242 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6c4fd35-d24e-494f-afbb-386222059784 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:54:02.020714 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.auth.core [None req-b6c4fd35-d24e-494f-afbb-386222059784 None None] MFA Rules not processed for user `bac4e6b76fe64217ac8fa44e71cffe6a`. Rule list: `[]` (Enabled: `True`). {{(pid=80143) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 10:54:02.022058 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-b6c4fd35-d24e-494f-afbb-386222059784 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 10:54:02.040621 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 783/1543] 10.222.0.25 () {60 vars in 1153 bytes} [Sun Aug 7 10:54:01 2022] POST /identity/v3/projects => generated 361 bytes in 109 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 07 10:54:02.042352 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3135b59d-1127-4be1-9780-31d3b16a342a None None] REQUEST_METHOD: `POST` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:54:02.042961 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3135b59d-1127-4be1-9780-31d3b16a342a None None] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:54:02.043768 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3135b59d-1127-4be1-9780-31d3b16a342a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:54:02.050960 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 761/1544] 10.222.0.25 () {58 vars in 958 bytes} [Sun Aug 7 10:54:01 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 72 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 07 10:54:02.050960 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6282536c-e5a8-448f-995f-bd6d9e595f51 None None] REQUEST_METHOD: `POST` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:54:02.051228 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6282536c-e5a8-448f-995f-bd6d9e595f51 None None] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:54:02.051320 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6282536c-e5a8-448f-995f-bd6d9e595f51 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:54:02.080010 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.auth.core [None req-3135b59d-1127-4be1-9780-31d3b16a342a None None] MFA Rules not processed for user `bac4e6b76fe64217ac8fa44e71cffe6a`. Rule list: `[]` (Enabled: `True`). {{(pid=80144) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 10:54:02.082088 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.fernet_utils [None req-3135b59d-1127-4be1-9780-31d3b16a342a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80144) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 10:54:02.099197 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.auth.core [None req-6282536c-e5a8-448f-995f-bd6d9e595f51 None None] MFA Rules not processed for user `bac4e6b76fe64217ac8fa44e71cffe6a`. Rule list: `[]` (Enabled: `True`). {{(pid=80143) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 10:54:02.100415 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-6282536c-e5a8-448f-995f-bd6d9e595f51 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 10:54:02.161510 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 784/1545] 10.222.0.25 () {58 vars in 958 bytes} [Sun Aug 7 10:54:02 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 07 10:54:02.162428 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3135b59d-1127-4be1-9780-31d3b16a342a None None] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 10:54:02.171856 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3135b59d-1127-4be1-9780-31d3b16a342a None None] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 10:54:02.176914 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a52d7bf-227c-4809-a2b9-1e3e252f073a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5ac47e3b09f43d19a23a0f457fe9e23', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 10:54:02.177509 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a52d7bf-227c-4809-a2b9-1e3e252f073a admin admin] REQUEST_METHOD: `POST` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:54:02.177794 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a52d7bf-227c-4809-a2b9-1e3e252f073a admin admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:54:02.178063 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a52d7bf-227c-4809-a2b9-1e3e252f073a admin admin] PATH_INFO: `/v3/users` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:54:02.178686 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a52d7bf-227c-4809-a2b9-1e3e252f073a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 10:54:02.179724 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: WARNING py.warnings [None req-2a52d7bf-227c-4809-a2b9-1e3e252f073a admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 07 10:54:02.179724 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: warnings.warn(msg) Aug 07 10:54:02.179724 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: Aug 07 10:54:02.187350 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 762/1546] 10.222.0.25 () {58 vars in 958 bytes} [Sun Aug 7 10:54:02 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 07 10:54:02.187523 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a52d7bf-227c-4809-a2b9-1e3e252f073a admin admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 10:54:02.187617 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-802a06e4-5e01-445a-942a-545215736600 None None] REQUEST_METHOD: `POST` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:54:02.187617 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-802a06e4-5e01-445a-942a-545215736600 None None] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:54:02.187617 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-802a06e4-5e01-445a-942a-545215736600 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:54:02.202197 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.auth.core [None req-802a06e4-5e01-445a-942a-545215736600 None None] MFA Rules not processed for user `bac4e6b76fe64217ac8fa44e71cffe6a`. Rule list: `[]` (Enabled: `True`). {{(pid=80143) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 10:54:02.202380 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 785/1547] 10.222.0.25 () {60 vars in 1144 bytes} [Sun Aug 7 10:54:02 2022] POST /identity/v3/users => generated 281 bytes in 40 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 07 10:54:02.203397 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-802a06e4-5e01-445a-942a-545215736600 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 10:54:02.203595 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd61d5de-c677-4713-913c-6561d37aa14a None None] REQUEST_METHOD: `POST` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:54:02.203884 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd61d5de-c677-4713-913c-6561d37aa14a None None] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:54:02.204220 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd61d5de-c677-4713-913c-6561d37aa14a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:54:02.211860 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 763/1548] 10.222.0.25 () {58 vars in 958 bytes} [Sun Aug 7 10:54:02 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 07 10:54:02.213041 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80121e31-73e1-4ca9-beb5-9f805b55eaa9 None None] REQUEST_METHOD: `POST` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:54:02.213336 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80121e31-73e1-4ca9-beb5-9f805b55eaa9 None None] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:54:02.213609 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80121e31-73e1-4ca9-beb5-9f805b55eaa9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:54:02.228474 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.auth.core [None req-fd61d5de-c677-4713-913c-6561d37aa14a None None] MFA Rules not processed for user `bac4e6b76fe64217ac8fa44e71cffe6a`. Rule list: `[]` (Enabled: `True`). {{(pid=80144) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 10:54:02.229682 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.fernet_utils [None req-fd61d5de-c677-4713-913c-6561d37aa14a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80144) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 10:54:02.232823 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.auth.core [None req-80121e31-73e1-4ca9-beb5-9f805b55eaa9 None None] MFA Rules not processed for user `bac4e6b76fe64217ac8fa44e71cffe6a`. Rule list: `[]` (Enabled: `True`). {{(pid=80143) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 10:54:02.234240 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-80121e31-73e1-4ca9-beb5-9f805b55eaa9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 10:54:02.237923 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 786/1549] 10.222.0.25 () {58 vars in 958 bytes} [Sun Aug 7 10:54:02 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 07 10:54:02.239202 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beb78bd2-896c-47c6-889d-846ccb76e25f None None] REQUEST_METHOD: `POST` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:54:02.239486 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beb78bd2-896c-47c6-889d-846ccb76e25f None None] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:54:02.239762 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beb78bd2-896c-47c6-889d-846ccb76e25f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:54:02.245545 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 764/1550] 10.222.0.25 () {58 vars in 958 bytes} [Sun Aug 7 10:54:02 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 07 10:54:02.245545 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-308fac32-1862-4dc8-87b6-826e00e80a9b None None] REQUEST_METHOD: `POST` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:54:02.245545 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-308fac32-1862-4dc8-87b6-826e00e80a9b None None] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:54:02.245545 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-308fac32-1862-4dc8-87b6-826e00e80a9b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:54:02.262536 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.auth.core [None req-beb78bd2-896c-47c6-889d-846ccb76e25f None None] MFA Rules not processed for user `bac4e6b76fe64217ac8fa44e71cffe6a`. Rule list: `[]` (Enabled: `True`). {{(pid=80144) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 10:54:02.263755 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.fernet_utils [None req-beb78bd2-896c-47c6-889d-846ccb76e25f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80144) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 10:54:02.265565 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.auth.core [None req-308fac32-1862-4dc8-87b6-826e00e80a9b None None] MFA Rules not processed for user `bac4e6b76fe64217ac8fa44e71cffe6a`. Rule list: `[]` (Enabled: `True`). {{(pid=80143) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 10:54:02.266723 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-308fac32-1862-4dc8-87b6-826e00e80a9b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 10:54:02.272305 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 787/1551] 10.222.0.25 () {58 vars in 958 bytes} [Sun Aug 7 10:54:02 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 07 10:54:02.273566 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c37c7b5e-cdd8-44a3-bac5-1c2ada84ac5a None None] REQUEST_METHOD: `POST` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:54:02.273850 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c37c7b5e-cdd8-44a3-bac5-1c2ada84ac5a None None] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:54:02.274124 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c37c7b5e-cdd8-44a3-bac5-1c2ada84ac5a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:54:02.280942 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 765/1552] 10.222.0.25 () {58 vars in 958 bytes} [Sun Aug 7 10:54:02 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 07 10:54:02.282244 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85f62e22-667b-4c02-a81a-ebbb4216da5f None None] REQUEST_METHOD: `POST` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:54:02.282531 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85f62e22-667b-4c02-a81a-ebbb4216da5f None None] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:54:02.282812 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85f62e22-667b-4c02-a81a-ebbb4216da5f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:54:02.294532 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.auth.core [None req-c37c7b5e-cdd8-44a3-bac5-1c2ada84ac5a None None] MFA Rules not processed for user `bac4e6b76fe64217ac8fa44e71cffe6a`. Rule list: `[]` (Enabled: `True`). {{(pid=80144) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 10:54:02.295760 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.fernet_utils [None req-c37c7b5e-cdd8-44a3-bac5-1c2ada84ac5a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80144) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 10:54:02.302517 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.auth.core [None req-85f62e22-667b-4c02-a81a-ebbb4216da5f None None] MFA Rules not processed for user `bac4e6b76fe64217ac8fa44e71cffe6a`. Rule list: `[]` (Enabled: `True`). {{(pid=80143) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 10:54:02.303711 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-85f62e22-667b-4c02-a81a-ebbb4216da5f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 10:54:02.304132 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 788/1553] 10.222.0.25 () {58 vars in 958 bytes} [Sun Aug 7 10:54:02 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 07 10:54:02.308064 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c37c7b5e-cdd8-44a3-bac5-1c2ada84ac5a None None] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 10:54:02.313889 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c37c7b5e-cdd8-44a3-bac5-1c2ada84ac5a None None] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 10:54:02.316039 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 766/1554] 10.222.0.25 () {58 vars in 958 bytes} [Sun Aug 7 10:54:02 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 07 10:54:02.316900 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85f62e22-667b-4c02-a81a-ebbb4216da5f None None] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 10:54:02.319202 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd30576f-c221-4cad-9972-b6d6e838e02b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5ac47e3b09f43d19a23a0f457fe9e23', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 10:54:02.319790 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd30576f-c221-4cad-9972-b6d6e838e02b admin admin] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:54:02.320066 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd30576f-c221-4cad-9972-b6d6e838e02b admin admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:54:02.320334 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd30576f-c221-4cad-9972-b6d6e838e02b admin admin] PATH_INFO: `/v3/roles` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:54:02.320965 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd30576f-c221-4cad-9972-b6d6e838e02b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 10:54:02.322528 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd30576f-c221-4cad-9972-b6d6e838e02b admin admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 10:54:02.326765 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85f62e22-667b-4c02-a81a-ebbb4216da5f None None] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 10:54:02.331455 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa1f3831-51a7-4932-ba99-90fd0f8e08b7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5ac47e3b09f43d19a23a0f457fe9e23', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 10:54:02.332064 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa1f3831-51a7-4932-ba99-90fd0f8e08b7 admin admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:54:02.332350 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa1f3831-51a7-4932-ba99-90fd0f8e08b7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:54:02.332621 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa1f3831-51a7-4932-ba99-90fd0f8e08b7 admin admin] PATH_INFO: `/v3/domains` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:54:02.332741 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 789/1555] 10.222.0.25 () {58 vars in 1122 bytes} [Sun Aug 7 10:54:02 2022] GET /identity/v3/roles => generated 1420 bytes in 28 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 07 10:54:02.333265 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa1f3831-51a7-4932-ba99-90fd0f8e08b7 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80143) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 10:54:02.333567 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa1f3831-51a7-4932-ba99-90fd0f8e08b7 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 10:54:02.333779 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d4672b7-3d14-4cc4-85b3-5b40ac1c05d1 None None] REQUEST_METHOD: `POST` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:54:02.334059 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d4672b7-3d14-4cc4-85b3-5b40ac1c05d1 None None] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:54:02.334328 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d4672b7-3d14-4cc4-85b3-5b40ac1c05d1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:54:02.335223 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa1f3831-51a7-4932-ba99-90fd0f8e08b7 admin admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 10:54:02.348074 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 767/1556] 10.222.0.25 () {58 vars in 1153 bytes} [Sun Aug 7 10:54:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 10:54:02.349359 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8ca6c7e-2fe8-4fe3-9f53-bc6ddbf48eed None None] REQUEST_METHOD: `POST` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:54:02.349624 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8ca6c7e-2fe8-4fe3-9f53-bc6ddbf48eed None None] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:54:02.349897 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8ca6c7e-2fe8-4fe3-9f53-bc6ddbf48eed None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:54:02.354432 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.auth.core [None req-1d4672b7-3d14-4cc4-85b3-5b40ac1c05d1 None None] MFA Rules not processed for user `bac4e6b76fe64217ac8fa44e71cffe6a`. Rule list: `[]` (Enabled: `True`). {{(pid=80144) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 10:54:02.355828 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.fernet_utils [None req-1d4672b7-3d14-4cc4-85b3-5b40ac1c05d1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80144) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 10:54:02.364804 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 790/1557] 10.222.0.25 () {58 vars in 958 bytes} [Sun Aug 7 10:54:02 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 07 10:54:02.366055 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37ced841-5c2a-46ca-8b46-e744ebafb7ed None None] REQUEST_METHOD: `POST` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:54:02.366338 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37ced841-5c2a-46ca-8b46-e744ebafb7ed None None] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:54:02.366608 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37ced841-5c2a-46ca-8b46-e744ebafb7ed None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:54:02.373920 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.auth.core [None req-d8ca6c7e-2fe8-4fe3-9f53-bc6ddbf48eed None None] MFA Rules not processed for user `bac4e6b76fe64217ac8fa44e71cffe6a`. Rule list: `[]` (Enabled: `True`). {{(pid=80143) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 10:54:02.375156 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-d8ca6c7e-2fe8-4fe3-9f53-bc6ddbf48eed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 10:54:02.383323 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 768/1558] 10.222.0.25 () {58 vars in 958 bytes} [Sun Aug 7 10:54:02 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 07 10:54:02.384505 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f44123f1-52b6-438a-b776-e1c2cd9c6a38 None None] REQUEST_METHOD: `POST` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:54:02.384900 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f44123f1-52b6-438a-b776-e1c2cd9c6a38 None None] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:54:02.385116 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f44123f1-52b6-438a-b776-e1c2cd9c6a38 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:54:02.385771 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.auth.core [None req-37ced841-5c2a-46ca-8b46-e744ebafb7ed None None] MFA Rules not processed for user `bac4e6b76fe64217ac8fa44e71cffe6a`. Rule list: `[]` (Enabled: `True`). {{(pid=80144) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 10:54:02.387114 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.fernet_utils [None req-37ced841-5c2a-46ca-8b46-e744ebafb7ed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80144) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 10:54:02.397172 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 791/1559] 10.222.0.25 () {58 vars in 958 bytes} [Sun Aug 7 10:54:02 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 07 10:54:02.398014 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37ced841-5c2a-46ca-8b46-e744ebafb7ed None None] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 10:54:02.406788 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.auth.core [None req-f44123f1-52b6-438a-b776-e1c2cd9c6a38 None None] MFA Rules not processed for user `bac4e6b76fe64217ac8fa44e71cffe6a`. Rule list: `[]` (Enabled: `True`). {{(pid=80143) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 10:54:02.408044 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-f44123f1-52b6-438a-b776-e1c2cd9c6a38 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 10:54:02.408168 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37ced841-5c2a-46ca-8b46-e744ebafb7ed None None] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 10:54:02.414040 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3377ae1-2006-47e0-a5d7-c8bef168bf22 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5ac47e3b09f43d19a23a0f457fe9e23', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 10:54:02.414639 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3377ae1-2006-47e0-a5d7-c8bef168bf22 admin admin] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:54:02.414965 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3377ae1-2006-47e0-a5d7-c8bef168bf22 admin admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:54:02.415226 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3377ae1-2006-47e0-a5d7-c8bef168bf22 admin admin] PATH_INFO: `/v3/domains` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:54:02.415870 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3377ae1-2006-47e0-a5d7-c8bef168bf22 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80144) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 10:54:02.416195 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3377ae1-2006-47e0-a5d7-c8bef168bf22 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 10:54:02.417156 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: WARNING py.warnings [None req-c3377ae1-2006-47e0-a5d7-c8bef168bf22 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_domains": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Aug 07 10:54:02.417156 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: warnings.warn(msg) Aug 07 10:54:02.417156 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: Aug 07 10:54:02.417722 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 769/1560] 10.222.0.25 () {58 vars in 958 bytes} [Sun Aug 7 10:54:02 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 07 10:54:02.418104 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3377ae1-2006-47e0-a5d7-c8bef168bf22 admin admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 10:54:02.418334 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f44123f1-52b6-438a-b776-e1c2cd9c6a38 None None] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 10:54:02.431642 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f44123f1-52b6-438a-b776-e1c2cd9c6a38 None None] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 10:54:02.436692 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 792/1561] 10.222.0.25 () {58 vars in 1153 bytes} [Sun Aug 7 10:54:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 10:54:02.436866 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f376748-b53b-4bca-9ea9-bcd132b8add0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5ac47e3b09f43d19a23a0f457fe9e23', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 10:54:02.436866 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f376748-b53b-4bca-9ea9-bcd132b8add0 admin admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:54:02.437056 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3377ae1-2006-47e0-a5d7-c8bef168bf22 admin admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 10:54:02.437145 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f376748-b53b-4bca-9ea9-bcd132b8add0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:54:02.437373 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f376748-b53b-4bca-9ea9-bcd132b8add0 admin admin] PATH_INFO: `/v3/domains` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:54:02.438011 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f376748-b53b-4bca-9ea9-bcd132b8add0 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80143) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 10:54:02.438315 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f376748-b53b-4bca-9ea9-bcd132b8add0 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 10:54:02.440064 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f376748-b53b-4bca-9ea9-bcd132b8add0 admin admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 10:54:02.452461 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3377ae1-2006-47e0-a5d7-c8bef168bf22 admin admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 10:54:02.460808 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 770/1562] 10.222.0.25 () {58 vars in 1153 bytes} [Sun Aug 7 10:54:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 10:54:02.460808 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f376748-b53b-4bca-9ea9-bcd132b8add0 admin admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 10:54:02.461053 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f622548-ff57-44bf-abff-cfbbcf442999 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5ac47e3b09f43d19a23a0f457fe9e23', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 10:54:02.461053 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f622548-ff57-44bf-abff-cfbbcf442999 admin admin] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:54:02.461053 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f622548-ff57-44bf-abff-cfbbcf442999 admin admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:54:02.461053 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f622548-ff57-44bf-abff-cfbbcf442999 admin admin] PATH_INFO: `/v3/roles` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:54:02.461428 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f622548-ff57-44bf-abff-cfbbcf442999 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 10:54:02.461428 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f622548-ff57-44bf-abff-cfbbcf442999 admin admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 10:54:02.465740 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f376748-b53b-4bca-9ea9-bcd132b8add0 admin admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 10:54:02.470362 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2ccb374-0aa9-4fbd-aad4-d8e78352cf7d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5ac47e3b09f43d19a23a0f457fe9e23', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 10:54:02.470977 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2ccb374-0aa9-4fbd-aad4-d8e78352cf7d admin admin] REQUEST_METHOD: `POST` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:54:02.471837 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2ccb374-0aa9-4fbd-aad4-d8e78352cf7d admin admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:54:02.472178 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2ccb374-0aa9-4fbd-aad4-d8e78352cf7d admin admin] PATH_INFO: `/v3/projects` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:54:02.475600 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 793/1563] 10.222.0.25 () {58 vars in 1122 bytes} [Sun Aug 7 10:54:02 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 07 10:54:02.475751 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2ccb374-0aa9-4fbd-aad4-d8e78352cf7d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 10:54:02.475851 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f622548-ff57-44bf-abff-cfbbcf442999 admin admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 10:54:02.475930 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: WARNING py.warnings [None req-b2ccb374-0aa9-4fbd-aad4-d8e78352cf7d admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 07 10:54:02.475930 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: warnings.warn(msg) Aug 07 10:54:02.475930 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: Aug 07 10:54:02.475930 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2ccb374-0aa9-4fbd-aad4-d8e78352cf7d admin admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 10:54:02.482967 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f622548-ff57-44bf-abff-cfbbcf442999 admin admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 10:54:02.488787 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d423d8df-e148-4da7-98d0-42af645e678c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5ac47e3b09f43d19a23a0f457fe9e23', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 10:54:02.489382 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d423d8df-e148-4da7-98d0-42af645e678c admin admin] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:54:02.489779 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d423d8df-e148-4da7-98d0-42af645e678c admin admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:54:02.490062 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d423d8df-e148-4da7-98d0-42af645e678c admin admin] PATH_INFO: `/v3/domains` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:54:02.490703 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d423d8df-e148-4da7-98d0-42af645e678c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80144) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 10:54:02.491044 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d423d8df-e148-4da7-98d0-42af645e678c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 10:54:02.492599 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d423d8df-e148-4da7-98d0-42af645e678c admin admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 10:54:02.503678 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 771/1564] 10.222.0.25 () {60 vars in 1153 bytes} [Sun Aug 7 10:54:02 2022] POST /identity/v3/projects => generated 385 bytes in 50 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 07 10:54:02.504542 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2ccb374-0aa9-4fbd-aad4-d8e78352cf7d admin admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 10:54:02.511242 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 794/1565] 10.222.0.25 () {58 vars in 1153 bytes} [Sun Aug 7 10:54:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 10:54:02.511242 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d423d8df-e148-4da7-98d0-42af645e678c admin admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 10:54:02.520846 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d423d8df-e148-4da7-98d0-42af645e678c admin admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 10:54:02.541267 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 10:54:02.559197 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 10:54:02.574471 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2ccb374-0aa9-4fbd-aad4-d8e78352cf7d admin admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 10:54:02.579493 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6c96e33-f630-49b4-ab80-e2aa6392289c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5ac47e3b09f43d19a23a0f457fe9e23', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 10:54:02.580099 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6c96e33-f630-49b4-ab80-e2aa6392289c admin admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:54:02.580368 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29fc72f8-1358-4a5b-a397-d9ddac55d058 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5ac47e3b09f43d19a23a0f457fe9e23', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 10:54:02.580592 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6c96e33-f630-49b4-ab80-e2aa6392289c admin admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:54:02.580731 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6c96e33-f630-49b4-ab80-e2aa6392289c admin admin] PATH_INFO: `/v3/domains` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:54:02.581244 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29fc72f8-1358-4a5b-a397-d9ddac55d058 admin admin] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:54:02.581342 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6c96e33-f630-49b4-ab80-e2aa6392289c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80143) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 10:54:02.581527 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29fc72f8-1358-4a5b-a397-d9ddac55d058 admin admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:54:02.581624 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6c96e33-f630-49b4-ab80-e2aa6392289c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 10:54:02.581805 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29fc72f8-1358-4a5b-a397-d9ddac55d058 admin admin] PATH_INFO: `/v3/domains` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:54:02.582448 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29fc72f8-1358-4a5b-a397-d9ddac55d058 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80144) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 10:54:02.582753 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29fc72f8-1358-4a5b-a397-d9ddac55d058 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 10:54:02.583196 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6c96e33-f630-49b4-ab80-e2aa6392289c admin admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 10:54:02.584367 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29fc72f8-1358-4a5b-a397-d9ddac55d058 admin admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 10:54:02.594856 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 772/1566] 10.222.0.25 () {58 vars in 1153 bytes} [Sun Aug 7 10:54:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 91 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 10:54:02.595597 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 795/1567] 10.222.0.25 () {58 vars in 1153 bytes} [Sun Aug 7 10:54:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 90 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 10:54:02.596602 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6c96e33-f630-49b4-ab80-e2aa6392289c admin admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 10:54:02.596877 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29fc72f8-1358-4a5b-a397-d9ddac55d058 admin admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 10:54:02.607993 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29fc72f8-1358-4a5b-a397-d9ddac55d058 admin admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 10:54:02.613435 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-532c8014-a408-443a-97f8-6846b6271582 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5ac47e3b09f43d19a23a0f457fe9e23', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 10:54:02.613590 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6c96e33-f630-49b4-ab80-e2aa6392289c admin admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 10:54:02.614029 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-532c8014-a408-443a-97f8-6846b6271582 admin admin] REQUEST_METHOD: `POST` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:54:02.614304 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-532c8014-a408-443a-97f8-6846b6271582 admin admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:54:02.614722 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-532c8014-a408-443a-97f8-6846b6271582 admin admin] PATH_INFO: `/v3/projects` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:54:02.615364 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-532c8014-a408-443a-97f8-6846b6271582 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 10:54:02.617261 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-532c8014-a408-443a-97f8-6846b6271582 admin admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 10:54:02.618363 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ea8e573-dd4e-4437-adb4-14bc6f5125e6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5ac47e3b09f43d19a23a0f457fe9e23', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 10:54:02.619059 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ea8e573-dd4e-4437-adb4-14bc6f5125e6 admin admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:54:02.619343 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ea8e573-dd4e-4437-adb4-14bc6f5125e6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:54:02.619620 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ea8e573-dd4e-4437-adb4-14bc6f5125e6 admin admin] PATH_INFO: `/v3/domains` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:54:02.620268 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ea8e573-dd4e-4437-adb4-14bc6f5125e6 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80143) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 10:54:02.620574 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ea8e573-dd4e-4437-adb4-14bc6f5125e6 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 10:54:02.622143 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ea8e573-dd4e-4437-adb4-14bc6f5125e6 admin admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 10:54:02.634361 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 773/1568] 10.222.0.25 () {58 vars in 1153 bytes} [Sun Aug 7 10:54:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 10:54:02.635372 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ea8e573-dd4e-4437-adb4-14bc6f5125e6 admin admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 10:54:02.646732 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ea8e573-dd4e-4437-adb4-14bc6f5125e6 admin admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 10:54:02.654238 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 796/1569] 10.222.0.25 () {60 vars in 1153 bytes} [Sun Aug 7 10:54:02 2022] POST /identity/v3/projects => generated 373 bytes in 55 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 07 10:54:02.654390 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-542e1d4b-c75d-464f-9dc8-be20d536a30a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5ac47e3b09f43d19a23a0f457fe9e23', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 10:54:02.654528 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-532c8014-a408-443a-97f8-6846b6271582 admin admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 10:54:02.654633 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-542e1d4b-c75d-464f-9dc8-be20d536a30a admin admin] REQUEST_METHOD: `POST` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:54:02.654633 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-542e1d4b-c75d-464f-9dc8-be20d536a30a admin admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:54:02.654633 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-542e1d4b-c75d-464f-9dc8-be20d536a30a admin admin] PATH_INFO: `/v3/projects` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:54:02.654633 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-542e1d4b-c75d-464f-9dc8-be20d536a30a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 10:54:02.655301 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-542e1d4b-c75d-464f-9dc8-be20d536a30a admin admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 10:54:02.686945 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 774/1570] 10.222.0.25 () {60 vars in 1153 bytes} [Sun Aug 7 10:54:02 2022] POST /identity/v3/projects => generated 379 bytes in 49 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 07 10:54:02.686945 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-542e1d4b-c75d-464f-9dc8-be20d536a30a admin admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 10:54:02.720402 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-532c8014-a408-443a-97f8-6846b6271582 admin admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 10:54:02.749230 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-542e1d4b-c75d-464f-9dc8-be20d536a30a admin admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 10:54:02.754018 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b82521c1-9e1e-463e-b0d2-6995f52372e2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5ac47e3b09f43d19a23a0f457fe9e23', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 10:54:02.754624 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b82521c1-9e1e-463e-b0d2-6995f52372e2 admin admin] REQUEST_METHOD: `POST` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:54:02.754915 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b82521c1-9e1e-463e-b0d2-6995f52372e2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:54:02.755217 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b82521c1-9e1e-463e-b0d2-6995f52372e2 admin admin] PATH_INFO: `/v3/users` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:54:02.755841 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b82521c1-9e1e-463e-b0d2-6995f52372e2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 10:54:02.757759 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b82521c1-9e1e-463e-b0d2-6995f52372e2 admin admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 10:54:02.778449 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fbf486d-3954-4505-9058-1a5d36fa748b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5ac47e3b09f43d19a23a0f457fe9e23', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 10:54:02.779153 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fbf486d-3954-4505-9058-1a5d36fa748b admin admin] REQUEST_METHOD: `PUT` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:54:02.779432 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fbf486d-3954-4505-9058-1a5d36fa748b admin admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:54:02.779761 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fbf486d-3954-4505-9058-1a5d36fa748b admin admin] PATH_INFO: `/v3/projects/e827ce1c53e1400c9a5af4924657100a/users/253a30f00d6f42d0af49f80acfeadcfa/roles/b98dfb3b9fee4644af2887a3ac3fd2e6` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:54:02.781349 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 775/1571] 10.222.0.25 () {60 vars in 1144 bytes} [Sun Aug 7 10:54:02 2022] POST /identity/v3/users => generated 293 bytes in 97 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 07 10:54:02.782407 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b82521c1-9e1e-463e-b0d2-6995f52372e2 admin admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 10:54:02.794409 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b82521c1-9e1e-463e-b0d2-6995f52372e2 admin admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 10:54:02.795292 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fbf486d-3954-4505-9058-1a5d36fa748b admin admin] RBAC: Authorizing `identity:create_grant(project_id=e827ce1c53e1400c9a5af4924657100a, user_id=253a30f00d6f42d0af49f80acfeadcfa, role_id=b98dfb3b9fee4644af2887a3ac3fd2e6)` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 10:54:02.797978 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fbf486d-3954-4505-9058-1a5d36fa748b admin admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 10:54:02.799133 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2ddc16f-dd91-4864-8c76-7cc1810e0f3b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5ac47e3b09f43d19a23a0f457fe9e23', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 10:54:02.800121 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2ddc16f-dd91-4864-8c76-7cc1810e0f3b admin admin] REQUEST_METHOD: `POST` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:54:02.800407 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2ddc16f-dd91-4864-8c76-7cc1810e0f3b admin admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:54:02.800677 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2ddc16f-dd91-4864-8c76-7cc1810e0f3b admin admin] PATH_INFO: `/v3/projects` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:54:02.801337 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2ddc16f-dd91-4864-8c76-7cc1810e0f3b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 10:54:02.803300 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2ddc16f-dd91-4864-8c76-7cc1810e0f3b admin admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 10:54:02.810737 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 797/1572] 10.222.0.25 () {60 vars in 1483 bytes} [Sun Aug 7 10:54:02 2022] PUT /identity/v3/projects/e827ce1c53e1400c9a5af4924657100a/users/253a30f00d6f42d0af49f80acfeadcfa/roles/b98dfb3b9fee4644af2887a3ac3fd2e6 => generated 0 bytes in 159 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 07 10:54:02.811645 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fbf486d-3954-4505-9058-1a5d36fa748b admin admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 10:54:02.842998 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 776/1573] 10.222.0.25 () {60 vars in 1153 bytes} [Sun Aug 7 10:54:02 2022] POST /identity/v3/projects => generated 365 bytes in 58 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 07 10:54:02.842998 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2ddc16f-dd91-4864-8c76-7cc1810e0f3b admin admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 10:54:02.844220 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 10:54:02.879111 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fbf486d-3954-4505-9058-1a5d36fa748b admin admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 10:54:02.909338 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2ddc16f-dd91-4864-8c76-7cc1810e0f3b admin admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 10:54:02.914281 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4ee4b39-52fd-488e-a44c-f1f6622a4dfd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5ac47e3b09f43d19a23a0f457fe9e23', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 10:54:02.914898 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4ee4b39-52fd-488e-a44c-f1f6622a4dfd admin admin] REQUEST_METHOD: `POST` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:54:02.915208 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4ee4b39-52fd-488e-a44c-f1f6622a4dfd admin admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:54:02.915484 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4ee4b39-52fd-488e-a44c-f1f6622a4dfd admin admin] PATH_INFO: `/v3/projects` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:54:02.916107 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4ee4b39-52fd-488e-a44c-f1f6622a4dfd admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 10:54:02.918008 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4ee4b39-52fd-488e-a44c-f1f6622a4dfd admin admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 10:54:02.936620 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a496e98-6ba9-4d4f-8c17-9ba4d724c9e4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5ac47e3b09f43d19a23a0f457fe9e23', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 10:54:02.937234 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a496e98-6ba9-4d4f-8c17-9ba4d724c9e4 admin admin] REQUEST_METHOD: `POST` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:54:02.937559 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a496e98-6ba9-4d4f-8c17-9ba4d724c9e4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:54:02.937797 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a496e98-6ba9-4d4f-8c17-9ba4d724c9e4 admin admin] PATH_INFO: `/v3/projects` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:54:02.938262 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 10:54:02.938405 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a496e98-6ba9-4d4f-8c17-9ba4d724c9e4 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 10:54:02.940801 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a496e98-6ba9-4d4f-8c17-9ba4d724c9e4 admin admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 10:54:02.946399 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 777/1574] 10.222.0.25 () {60 vars in 1153 bytes} [Sun Aug 7 10:54:02 2022] POST /identity/v3/projects => generated 377 bytes in 106 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 07 10:54:02.947219 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4ee4b39-52fd-488e-a44c-f1f6622a4dfd admin admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 10:54:02.969225 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 798/1575] 10.222.0.25 () {60 vars in 1153 bytes} [Sun Aug 7 10:54:02 2022] POST /identity/v3/projects => generated 393 bytes in 158 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 07 10:54:02.970284 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a496e98-6ba9-4d4f-8c17-9ba4d724c9e4 admin admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 10:54:03.009421 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4ee4b39-52fd-488e-a44c-f1f6622a4dfd admin admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 10:54:03.031974 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a496e98-6ba9-4d4f-8c17-9ba4d724c9e4 admin admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 10:54:03.037277 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45056662-da2d-4142-94b1-a569106ca045 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5ac47e3b09f43d19a23a0f457fe9e23', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 10:54:03.037884 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45056662-da2d-4142-94b1-a569106ca045 admin admin] REQUEST_METHOD: `POST` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:54:03.038167 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45056662-da2d-4142-94b1-a569106ca045 admin admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:54:03.038446 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45056662-da2d-4142-94b1-a569106ca045 admin admin] PATH_INFO: `/v3/users` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:54:03.039105 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45056662-da2d-4142-94b1-a569106ca045 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 10:54:03.041007 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45056662-da2d-4142-94b1-a569106ca045 admin admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 10:54:03.061896 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 799/1576] 10.222.0.25 () {60 vars in 1144 bytes} [Sun Aug 7 10:54:02 2022] POST /identity/v3/users => generated 287 bytes in 92 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 07 10:54:03.062772 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45056662-da2d-4142-94b1-a569106ca045 admin admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 10:54:03.069772 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f553d81a-0bfa-4c82-b471-66c0e03bb1f1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5ac47e3b09f43d19a23a0f457fe9e23', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 10:54:03.070387 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f553d81a-0bfa-4c82-b471-66c0e03bb1f1 admin admin] REQUEST_METHOD: `POST` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:54:03.070665 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f553d81a-0bfa-4c82-b471-66c0e03bb1f1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:54:03.071014 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f553d81a-0bfa-4c82-b471-66c0e03bb1f1 admin admin] PATH_INFO: `/v3/projects` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:54:03.071590 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f553d81a-0bfa-4c82-b471-66c0e03bb1f1 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 10:54:03.073879 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f553d81a-0bfa-4c82-b471-66c0e03bb1f1 admin admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 10:54:03.074724 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45056662-da2d-4142-94b1-a569106ca045 admin admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 10:54:03.079379 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a7ee514-338e-4a14-aba3-44081b865202 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5ac47e3b09f43d19a23a0f457fe9e23', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 10:54:03.080003 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a7ee514-338e-4a14-aba3-44081b865202 admin admin] REQUEST_METHOD: `POST` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:54:03.080306 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a7ee514-338e-4a14-aba3-44081b865202 admin admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:54:03.080789 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a7ee514-338e-4a14-aba3-44081b865202 admin admin] PATH_INFO: `/v3/users` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:54:03.081479 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a7ee514-338e-4a14-aba3-44081b865202 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 10:54:03.083423 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a7ee514-338e-4a14-aba3-44081b865202 admin admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 10:54:03.102488 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 778/1577] 10.222.0.25 () {60 vars in 1153 bytes} [Sun Aug 7 10:54:02 2022] POST /identity/v3/projects => generated 383 bytes in 156 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 07 10:54:03.103428 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f553d81a-0bfa-4c82-b471-66c0e03bb1f1 admin admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 10:54:03.106975 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 800/1578] 10.222.0.25 () {60 vars in 1144 bytes} [Sun Aug 7 10:54:03 2022] POST /identity/v3/users => generated 290 bytes in 44 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 07 10:54:03.108700 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f503a38-3d7f-4712-86d9-a61d5b314569 None None] REQUEST_METHOD: `POST` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:54:03.109090 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f503a38-3d7f-4712-86d9-a61d5b314569 None None] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:54:03.109380 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f503a38-3d7f-4712-86d9-a61d5b314569 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:54:03.138134 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.auth.core [None req-5f503a38-3d7f-4712-86d9-a61d5b314569 None None] MFA Rules not processed for user `253a30f00d6f42d0af49f80acfeadcfa`. Rule list: `[]` (Enabled: `True`). {{(pid=80144) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 10:54:03.139591 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.fernet_utils [None req-5f503a38-3d7f-4712-86d9-a61d5b314569 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80144) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 10:54:03.159855 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 10:54:03.166051 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f553d81a-0bfa-4c82-b471-66c0e03bb1f1 admin admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 10:54:03.171372 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3955c15-53bb-42ff-859d-1811953eb4cb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5ac47e3b09f43d19a23a0f457fe9e23', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 10:54:03.172292 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3955c15-53bb-42ff-859d-1811953eb4cb admin admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:54:03.172697 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3955c15-53bb-42ff-859d-1811953eb4cb admin admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:54:03.172978 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3955c15-53bb-42ff-859d-1811953eb4cb admin admin] PATH_INFO: `/v3/roles` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:54:03.173627 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3955c15-53bb-42ff-859d-1811953eb4cb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 10:54:03.175744 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3955c15-53bb-42ff-859d-1811953eb4cb admin admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 10:54:03.186581 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 779/1579] 10.222.0.25 () {58 vars in 1122 bytes} [Sun Aug 7 10:54:03 2022] GET /identity/v3/roles => generated 1420 bytes in 84 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 07 10:54:03.187577 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3955c15-53bb-42ff-859d-1811953eb4cb admin admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 10:54:03.198820 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3955c15-53bb-42ff-859d-1811953eb4cb admin admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 10:54:03.203598 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-687ecd74-8598-410a-9cdf-b8062821198d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5ac47e3b09f43d19a23a0f457fe9e23', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 10:54:03.204272 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-687ecd74-8598-410a-9cdf-b8062821198d admin admin] REQUEST_METHOD: `POST` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:54:03.204624 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-687ecd74-8598-410a-9cdf-b8062821198d admin admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:54:03.204969 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-687ecd74-8598-410a-9cdf-b8062821198d admin admin] PATH_INFO: `/v3/users` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:54:03.206135 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-687ecd74-8598-410a-9cdf-b8062821198d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 10:54:03.208090 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-687ecd74-8598-410a-9cdf-b8062821198d admin admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 10:54:03.214327 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 801/1580] 10.222.0.25 () {58 vars in 958 bytes} [Sun Aug 7 10:54:03 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 07 10:54:03.215175 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f503a38-3d7f-4712-86d9-a61d5b314569 None None] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 10:54:03.227139 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f503a38-3d7f-4712-86d9-a61d5b314569 None None] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 10:54:03.232257 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 780/1581] 10.222.0.25 () {60 vars in 1144 bytes} [Sun Aug 7 10:54:03 2022] POST /identity/v3/users => generated 283 bytes in 42 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 07 10:54:03.232257 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-687ecd74-8598-410a-9cdf-b8062821198d admin admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 10:54:03.235305 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cfa1079-940b-4234-9d71-66748134b529 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5ac47e3b09f43d19a23a0f457fe9e23', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 10:54:03.235913 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cfa1079-940b-4234-9d71-66748134b529 admin admin] REQUEST_METHOD: `POST` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:54:03.236195 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cfa1079-940b-4234-9d71-66748134b529 admin admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:54:03.236471 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cfa1079-940b-4234-9d71-66748134b529 admin admin] PATH_INFO: `/v3/users` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:54:03.237643 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cfa1079-940b-4234-9d71-66748134b529 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 10:54:03.239623 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cfa1079-940b-4234-9d71-66748134b529 admin admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 10:54:03.242068 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-687ecd74-8598-410a-9cdf-b8062821198d admin admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 10:54:03.246661 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a31638aa-2290-4dac-81c1-13ead556adce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5ac47e3b09f43d19a23a0f457fe9e23', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 10:54:03.247294 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a31638aa-2290-4dac-81c1-13ead556adce admin admin] REQUEST_METHOD: `POST` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:54:03.247579 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a31638aa-2290-4dac-81c1-13ead556adce admin admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:54:03.247855 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a31638aa-2290-4dac-81c1-13ead556adce admin admin] PATH_INFO: `/v3/users` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:54:03.248551 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a31638aa-2290-4dac-81c1-13ead556adce admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 10:54:03.250730 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a31638aa-2290-4dac-81c1-13ead556adce admin admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 10:54:03.261261 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 10:54:03.261605 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 802/1582] 10.222.0.25 () {60 vars in 1144 bytes} [Sun Aug 7 10:54:03 2022] POST /identity/v3/users => generated 289 bytes in 47 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 07 10:54:03.262567 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cfa1079-940b-4234-9d71-66748134b529 admin admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 10:54:03.274657 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cfa1079-940b-4234-9d71-66748134b529 admin admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 10:54:03.277380 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 781/1583] 10.222.0.25 () {60 vars in 1144 bytes} [Sun Aug 7 10:54:03 2022] POST /identity/v3/users => generated 297 bytes in 47 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 07 10:54:03.278192 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a31638aa-2290-4dac-81c1-13ead556adce admin admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 10:54:03.279346 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68eca491-7b3a-41b1-a441-d03fb66de9b5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5ac47e3b09f43d19a23a0f457fe9e23', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 10:54:03.280002 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68eca491-7b3a-41b1-a441-d03fb66de9b5 admin admin] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:54:03.280275 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68eca491-7b3a-41b1-a441-d03fb66de9b5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:54:03.280935 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68eca491-7b3a-41b1-a441-d03fb66de9b5 admin admin] PATH_INFO: `/v3/roles` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:54:03.281665 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68eca491-7b3a-41b1-a441-d03fb66de9b5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 10:54:03.283269 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68eca491-7b3a-41b1-a441-d03fb66de9b5 admin admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 10:54:03.288565 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a31638aa-2290-4dac-81c1-13ead556adce admin admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 10:54:03.293241 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aabc6d23-3763-4a83-b221-4a1d31c9cb1d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5ac47e3b09f43d19a23a0f457fe9e23', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 10:54:03.293831 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aabc6d23-3763-4a83-b221-4a1d31c9cb1d admin admin] REQUEST_METHOD: `POST` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:54:03.293967 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 803/1584] 10.222.0.25 () {58 vars in 1122 bytes} [Sun Aug 7 10:54:03 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 07 10:54:03.294118 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aabc6d23-3763-4a83-b221-4a1d31c9cb1d admin admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:54:03.294392 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aabc6d23-3763-4a83-b221-4a1d31c9cb1d admin admin] PATH_INFO: `/v3/users` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:54:03.294719 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68eca491-7b3a-41b1-a441-d03fb66de9b5 admin admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 10:54:03.295105 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aabc6d23-3763-4a83-b221-4a1d31c9cb1d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 10:54:03.297190 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aabc6d23-3763-4a83-b221-4a1d31c9cb1d admin admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 10:54:03.304304 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68eca491-7b3a-41b1-a441-d03fb66de9b5 admin admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 10:54:03.309206 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8594b300-7666-40b0-a635-6541ec13047c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5ac47e3b09f43d19a23a0f457fe9e23', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 10:54:03.309926 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8594b300-7666-40b0-a635-6541ec13047c admin admin] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:54:03.310242 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8594b300-7666-40b0-a635-6541ec13047c admin admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:54:03.310635 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8594b300-7666-40b0-a635-6541ec13047c admin admin] PATH_INFO: `/v3/roles` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:54:03.311420 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8594b300-7666-40b0-a635-6541ec13047c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 10:54:03.313452 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8594b300-7666-40b0-a635-6541ec13047c admin admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 10:54:03.320114 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 782/1585] 10.222.0.25 () {60 vars in 1144 bytes} [Sun Aug 7 10:54:03 2022] POST /identity/v3/users => generated 292 bytes in 42 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 07 10:54:03.320927 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aabc6d23-3763-4a83-b221-4a1d31c9cb1d admin admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 10:54:03.329746 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 804/1586] 10.222.0.25 () {58 vars in 1122 bytes} [Sun Aug 7 10:54:03 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 07 10:54:03.329746 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8594b300-7666-40b0-a635-6541ec13047c admin admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 10:54:03.330335 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aabc6d23-3763-4a83-b221-4a1d31c9cb1d admin admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 10:54:03.335501 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95722555-8a36-4776-a04b-84253f996ba7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5ac47e3b09f43d19a23a0f457fe9e23', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 10:54:03.336093 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95722555-8a36-4776-a04b-84253f996ba7 admin admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:54:03.336376 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95722555-8a36-4776-a04b-84253f996ba7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:54:03.336651 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95722555-8a36-4776-a04b-84253f996ba7 admin admin] PATH_INFO: `/v3/roles` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:54:03.337289 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95722555-8a36-4776-a04b-84253f996ba7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 10:54:03.338869 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95722555-8a36-4776-a04b-84253f996ba7 admin admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 10:54:03.339873 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8594b300-7666-40b0-a635-6541ec13047c admin admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 10:54:03.344635 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfa51e1f-6b33-4fdb-96a5-e95810a59ea1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5ac47e3b09f43d19a23a0f457fe9e23', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 10:54:03.345264 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfa51e1f-6b33-4fdb-96a5-e95810a59ea1 admin admin] REQUEST_METHOD: `POST` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:54:03.345547 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfa51e1f-6b33-4fdb-96a5-e95810a59ea1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:54:03.345826 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfa51e1f-6b33-4fdb-96a5-e95810a59ea1 admin admin] PATH_INFO: `/v3/projects` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:54:03.347042 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfa51e1f-6b33-4fdb-96a5-e95810a59ea1 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 10:54:03.348850 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 783/1587] 10.222.0.25 () {58 vars in 1122 bytes} [Sun Aug 7 10:54:03 2022] GET /identity/v3/roles => generated 1420 bytes in 28 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 07 10:54:03.348956 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfa51e1f-6b33-4fdb-96a5-e95810a59ea1 admin admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 10:54:03.349694 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95722555-8a36-4776-a04b-84253f996ba7 admin admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 10:54:03.359383 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95722555-8a36-4776-a04b-84253f996ba7 admin admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 10:54:03.363937 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bc65c4b-0360-450d-a8af-d5dd6ec982f6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5ac47e3b09f43d19a23a0f457fe9e23', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 10:54:03.364524 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bc65c4b-0360-450d-a8af-d5dd6ec982f6 admin admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:54:03.364807 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bc65c4b-0360-450d-a8af-d5dd6ec982f6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:54:03.365083 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bc65c4b-0360-450d-a8af-d5dd6ec982f6 admin admin] PATH_INFO: `/v3/roles` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:54:03.365714 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bc65c4b-0360-450d-a8af-d5dd6ec982f6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 10:54:03.367312 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bc65c4b-0360-450d-a8af-d5dd6ec982f6 admin admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 10:54:03.378826 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 805/1588] 10.222.0.25 () {60 vars in 1153 bytes} [Sun Aug 7 10:54:03 2022] POST /identity/v3/projects => generated 365 bytes in 51 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 07 10:54:03.379773 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfa51e1f-6b33-4fdb-96a5-e95810a59ea1 admin admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 10:54:03.382350 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 784/1589] 10.222.0.25 () {58 vars in 1122 bytes} [Sun Aug 7 10:54:03 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 07 10:54:03.382350 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bc65c4b-0360-450d-a8af-d5dd6ec982f6 admin admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 10:54:03.392146 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bc65c4b-0360-450d-a8af-d5dd6ec982f6 admin admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 10:54:03.443243 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfa51e1f-6b33-4fdb-96a5-e95810a59ea1 admin admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 10:54:03.448013 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c153611a-d8ab-4b21-8975-d9c36708a51b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5ac47e3b09f43d19a23a0f457fe9e23', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 10:54:03.448724 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c153611a-d8ab-4b21-8975-d9c36708a51b admin admin] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:54:03.449107 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c153611a-d8ab-4b21-8975-d9c36708a51b admin admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:54:03.449498 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c153611a-d8ab-4b21-8975-d9c36708a51b admin admin] PATH_INFO: `/v3/roles` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:54:03.450244 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c153611a-d8ab-4b21-8975-d9c36708a51b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 10:54:03.450656 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f441ef34-41f8-4503-8dfc-d2fda81d60b8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5ac47e3b09f43d19a23a0f457fe9e23', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 10:54:03.451297 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f441ef34-41f8-4503-8dfc-d2fda81d60b8 admin admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:54:03.451585 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f441ef34-41f8-4503-8dfc-d2fda81d60b8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:54:03.451858 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f441ef34-41f8-4503-8dfc-d2fda81d60b8 admin admin] PATH_INFO: `/v3/roles` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:54:03.452234 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c153611a-d8ab-4b21-8975-d9c36708a51b admin admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 10:54:03.452496 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f441ef34-41f8-4503-8dfc-d2fda81d60b8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 10:54:03.454111 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f441ef34-41f8-4503-8dfc-d2fda81d60b8 admin admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 10:54:03.462471 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 806/1590] 10.222.0.25 () {58 vars in 1122 bytes} [Sun Aug 7 10:54:03 2022] GET /identity/v3/roles => generated 1420 bytes in 83 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 07 10:54:03.463317 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c153611a-d8ab-4b21-8975-d9c36708a51b admin admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 10:54:03.465798 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 785/1591] 10.222.0.25 () {58 vars in 1122 bytes} [Sun Aug 7 10:54:03 2022] GET /identity/v3/roles => generated 1420 bytes in 84 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 07 10:54:03.466644 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f441ef34-41f8-4503-8dfc-d2fda81d60b8 admin admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 10:54:03.472726 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c153611a-d8ab-4b21-8975-d9c36708a51b admin admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 10:54:03.477123 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f441ef34-41f8-4503-8dfc-d2fda81d60b8 admin admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 10:54:03.477315 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0f11a9d-2386-4bf7-91a1-48974840a5bf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5ac47e3b09f43d19a23a0f457fe9e23', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 10:54:03.477905 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0f11a9d-2386-4bf7-91a1-48974840a5bf admin admin] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:54:03.478185 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0f11a9d-2386-4bf7-91a1-48974840a5bf admin admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:54:03.478463 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0f11a9d-2386-4bf7-91a1-48974840a5bf admin admin] PATH_INFO: `/v3/roles` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:54:03.479138 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0f11a9d-2386-4bf7-91a1-48974840a5bf admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 10:54:03.480701 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0f11a9d-2386-4bf7-91a1-48974840a5bf admin admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 10:54:03.481620 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b777221-b8e8-450d-8a19-a12cca94c903 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5ac47e3b09f43d19a23a0f457fe9e23', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 10:54:03.482194 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b777221-b8e8-450d-8a19-a12cca94c903 admin admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:54:03.482470 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b777221-b8e8-450d-8a19-a12cca94c903 admin admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:54:03.482740 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b777221-b8e8-450d-8a19-a12cca94c903 admin admin] PATH_INFO: `/v3/roles` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:54:03.483664 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b777221-b8e8-450d-8a19-a12cca94c903 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 10:54:03.485235 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b777221-b8e8-450d-8a19-a12cca94c903 admin admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 10:54:03.491526 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 807/1592] 10.222.0.25 () {58 vars in 1122 bytes} [Sun Aug 7 10:54:03 2022] GET /identity/v3/roles => generated 1420 bytes in 28 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 07 10:54:03.494856 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0f11a9d-2386-4bf7-91a1-48974840a5bf admin admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 10:54:03.498583 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 786/1593] 10.222.0.25 () {58 vars in 1122 bytes} [Sun Aug 7 10:54:03 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 07 10:54:03.498583 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b777221-b8e8-450d-8a19-a12cca94c903 admin admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 10:54:03.504955 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0f11a9d-2386-4bf7-91a1-48974840a5bf admin admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 10:54:03.506903 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b777221-b8e8-450d-8a19-a12cca94c903 admin admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 10:54:03.509948 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c27df4bb-bafb-440d-8e81-1b22df4896a7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5ac47e3b09f43d19a23a0f457fe9e23', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 10:54:03.510542 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c27df4bb-bafb-440d-8e81-1b22df4896a7 admin admin] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:54:03.510822 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c27df4bb-bafb-440d-8e81-1b22df4896a7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:54:03.511133 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c27df4bb-bafb-440d-8e81-1b22df4896a7 admin admin] PATH_INFO: `/v3/roles` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:54:03.511730 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62135bd5-0547-4f1c-91e4-e3263ea320ce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5ac47e3b09f43d19a23a0f457fe9e23', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 10:54:03.511847 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c27df4bb-bafb-440d-8e81-1b22df4896a7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 10:54:03.512323 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62135bd5-0547-4f1c-91e4-e3263ea320ce admin admin] REQUEST_METHOD: `PUT` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:54:03.512604 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62135bd5-0547-4f1c-91e4-e3263ea320ce admin admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:54:03.512882 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62135bd5-0547-4f1c-91e4-e3263ea320ce admin admin] PATH_INFO: `/v3/projects/1f23b23bf2f84fdf8401acf4d73e1f4b/users/2b757e3f47e44b9ba3645d12b20f7502/roles/b98dfb3b9fee4644af2887a3ac3fd2e6` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:54:03.513335 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c27df4bb-bafb-440d-8e81-1b22df4896a7 admin admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 10:54:03.524333 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 808/1594] 10.222.0.25 () {58 vars in 1122 bytes} [Sun Aug 7 10:54:03 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 07 10:54:03.525114 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c27df4bb-bafb-440d-8e81-1b22df4896a7 admin admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 10:54:03.529496 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62135bd5-0547-4f1c-91e4-e3263ea320ce admin admin] RBAC: Authorizing `identity:create_grant(project_id=1f23b23bf2f84fdf8401acf4d73e1f4b, user_id=2b757e3f47e44b9ba3645d12b20f7502, role_id=b98dfb3b9fee4644af2887a3ac3fd2e6)` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 10:54:03.530556 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: WARNING py.warnings [None req-62135bd5-0547-4f1c-91e4-e3263ea320ce admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_grant": "(role:admin and system_scope:all) or ((role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.domain.id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.domain.id)s)) and (domain_id:%(target.role.domain_id)s or None:%(target.role.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 07 10:54:03.530556 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: warnings.warn(msg) Aug 07 10:54:03.530556 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: Aug 07 10:54:03.532520 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62135bd5-0547-4f1c-91e4-e3263ea320ce admin admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 10:54:03.535122 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c27df4bb-bafb-440d-8e81-1b22df4896a7 admin admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 10:54:03.540257 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d2f8d7b-1cdf-4791-b02b-e3699c092aeb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5ac47e3b09f43d19a23a0f457fe9e23', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 10:54:03.540850 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d2f8d7b-1cdf-4791-b02b-e3699c092aeb admin admin] REQUEST_METHOD: `POST` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:54:03.541140 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d2f8d7b-1cdf-4791-b02b-e3699c092aeb admin admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:54:03.541432 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d2f8d7b-1cdf-4791-b02b-e3699c092aeb admin admin] PATH_INFO: `/v3/users` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:54:03.542041 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d2f8d7b-1cdf-4791-b02b-e3699c092aeb admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 10:54:03.544003 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d2f8d7b-1cdf-4791-b02b-e3699c092aeb admin admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 10:54:03.547320 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 787/1595] 10.222.0.25 () {60 vars in 1483 bytes} [Sun Aug 7 10:54:03 2022] PUT /identity/v3/projects/1f23b23bf2f84fdf8401acf4d73e1f4b/users/2b757e3f47e44b9ba3645d12b20f7502/roles/b98dfb3b9fee4644af2887a3ac3fd2e6 => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 07 10:54:03.548120 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62135bd5-0547-4f1c-91e4-e3263ea320ce admin admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 10:54:03.566565 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 809/1596] 10.222.0.25 () {60 vars in 1144 bytes} [Sun Aug 7 10:54:03 2022] POST /identity/v3/users => generated 289 bytes in 42 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 07 10:54:03.567738 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d2f8d7b-1cdf-4791-b02b-e3699c092aeb admin admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 10:54:03.577362 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d2f8d7b-1cdf-4791-b02b-e3699c092aeb admin admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 10:54:03.590810 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 10:54:03.613001 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62135bd5-0547-4f1c-91e4-e3263ea320ce admin admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 10:54:03.617613 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bebb366-eaf4-4bcf-bb00-5ceedcdfa08e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5ac47e3b09f43d19a23a0f457fe9e23', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 10:54:03.618218 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bebb366-eaf4-4bcf-bb00-5ceedcdfa08e admin admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:54:03.618500 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bebb366-eaf4-4bcf-bb00-5ceedcdfa08e admin admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:54:03.618775 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bebb366-eaf4-4bcf-bb00-5ceedcdfa08e admin admin] PATH_INFO: `/v3/roles` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:54:03.619453 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bebb366-eaf4-4bcf-bb00-5ceedcdfa08e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 10:54:03.621036 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bebb366-eaf4-4bcf-bb00-5ceedcdfa08e admin admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 10:54:03.631155 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 788/1597] 10.222.0.25 () {58 vars in 1122 bytes} [Sun Aug 7 10:54:03 2022] GET /identity/v3/roles => generated 1420 bytes in 83 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 07 10:54:03.632142 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bebb366-eaf4-4bcf-bb00-5ceedcdfa08e admin admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 10:54:03.638204 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-435adfbc-8433-4bc7-a05f-efbdbffecb1d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5ac47e3b09f43d19a23a0f457fe9e23', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 10:54:03.638825 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-435adfbc-8433-4bc7-a05f-efbdbffecb1d admin admin] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:54:03.639148 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-435adfbc-8433-4bc7-a05f-efbdbffecb1d admin admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:54:03.639544 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-435adfbc-8433-4bc7-a05f-efbdbffecb1d admin admin] PATH_INFO: `/v3/roles` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:54:03.640197 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-435adfbc-8433-4bc7-a05f-efbdbffecb1d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 10:54:03.641614 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bebb366-eaf4-4bcf-bb00-5ceedcdfa08e admin admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 10:54:03.641806 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-435adfbc-8433-4bc7-a05f-efbdbffecb1d admin admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 10:54:03.646698 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-910ab9f0-e1ba-4dd4-921a-a533a41f7bd1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5ac47e3b09f43d19a23a0f457fe9e23', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 10:54:03.647301 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-910ab9f0-e1ba-4dd4-921a-a533a41f7bd1 admin admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:54:03.647584 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-910ab9f0-e1ba-4dd4-921a-a533a41f7bd1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:54:03.647853 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-910ab9f0-e1ba-4dd4-921a-a533a41f7bd1 admin admin] PATH_INFO: `/v3/roles` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:54:03.648472 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-910ab9f0-e1ba-4dd4-921a-a533a41f7bd1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 10:54:03.650001 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-910ab9f0-e1ba-4dd4-921a-a533a41f7bd1 admin admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 10:54:03.652422 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:INSERT=6,keystone:SELECT=94 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 10:54:03.652669 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 810/1598] 10.222.0.25 () {58 vars in 1122 bytes} [Sun Aug 7 10:54:03 2022] GET /identity/v3/roles => generated 1420 bytes in 85 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 07 10:54:03.653495 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-435adfbc-8433-4bc7-a05f-efbdbffecb1d admin admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 10:54:03.666190 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-435adfbc-8433-4bc7-a05f-efbdbffecb1d admin admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 10:54:03.666504 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 789/1599] 10.222.0.25 () {58 vars in 1122 bytes} [Sun Aug 7 10:54:03 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 07 10:54:03.667396 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-910ab9f0-e1ba-4dd4-921a-a533a41f7bd1 admin admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 10:54:03.668349 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2113a92-fef7-445c-b52b-215c20881288 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5ac47e3b09f43d19a23a0f457fe9e23', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 10:54:03.669000 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2113a92-fef7-445c-b52b-215c20881288 admin admin] REQUEST_METHOD: `PUT` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:54:03.669279 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2113a92-fef7-445c-b52b-215c20881288 admin admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:54:03.669554 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2113a92-fef7-445c-b52b-215c20881288 admin admin] PATH_INFO: `/v3/projects/d16deb58b3724c8191e1e251a4e915f0/users/12de1ec12eb746bc9bb163bff5b464ed/roles/b98dfb3b9fee4644af2887a3ac3fd2e6` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:54:03.677360 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-910ab9f0-e1ba-4dd4-921a-a533a41f7bd1 admin admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 10:54:03.682059 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a02f13b1-16af-4d7d-9488-63c9463a8015 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5ac47e3b09f43d19a23a0f457fe9e23', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 10:54:03.682642 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a02f13b1-16af-4d7d-9488-63c9463a8015 admin admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:54:03.682931 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a02f13b1-16af-4d7d-9488-63c9463a8015 admin admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:54:03.683217 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a02f13b1-16af-4d7d-9488-63c9463a8015 admin admin] PATH_INFO: `/v3/roles` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:54:03.683855 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a02f13b1-16af-4d7d-9488-63c9463a8015 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 10:54:03.684353 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2113a92-fef7-445c-b52b-215c20881288 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d16deb58b3724c8191e1e251a4e915f0, user_id=12de1ec12eb746bc9bb163bff5b464ed, role_id=b98dfb3b9fee4644af2887a3ac3fd2e6)` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 10:54:03.685429 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a02f13b1-16af-4d7d-9488-63c9463a8015 admin admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 10:54:03.687002 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2113a92-fef7-445c-b52b-215c20881288 admin admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 10:54:03.696304 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 790/1600] 10.222.0.25 () {58 vars in 1122 bytes} [Sun Aug 7 10:54:03 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 07 10:54:03.697494 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a02f13b1-16af-4d7d-9488-63c9463a8015 admin admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 10:54:03.706375 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 811/1601] 10.222.0.25 () {60 vars in 1483 bytes} [Sun Aug 7 10:54:03 2022] PUT /identity/v3/projects/d16deb58b3724c8191e1e251a4e915f0/users/12de1ec12eb746bc9bb163bff5b464ed/roles/b98dfb3b9fee4644af2887a3ac3fd2e6 => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 07 10:54:03.707469 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a02f13b1-16af-4d7d-9488-63c9463a8015 admin admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 10:54:03.707855 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05c5e97c-e604-4d6e-9a43-352c01e1acce None None] REQUEST_METHOD: `POST` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:54:03.708237 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05c5e97c-e604-4d6e-9a43-352c01e1acce None None] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:54:03.708602 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05c5e97c-e604-4d6e-9a43-352c01e1acce None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:54:03.727673 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.auth.core [None req-05c5e97c-e604-4d6e-9a43-352c01e1acce None None] MFA Rules not processed for user `bac4e6b76fe64217ac8fa44e71cffe6a`. Rule list: `[]` (Enabled: `True`). {{(pid=80144) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 10:54:03.728863 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.fernet_utils [None req-05c5e97c-e604-4d6e-9a43-352c01e1acce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80144) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 10:54:03.763617 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cd6bfe9-30d5-44b4-8617-0263fb637e72 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5ac47e3b09f43d19a23a0f457fe9e23', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 10:54:03.764200 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cd6bfe9-30d5-44b4-8617-0263fb637e72 admin admin] REQUEST_METHOD: `PUT` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:54:03.764479 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cd6bfe9-30d5-44b4-8617-0263fb637e72 admin admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:54:03.764750 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cd6bfe9-30d5-44b4-8617-0263fb637e72 admin admin] PATH_INFO: `/v3/projects/2d2c614e07df4a29b2cd84c260fdc93a/users/616a497708664cca8bf7cfe93cbfd01f/roles/b98dfb3b9fee4644af2887a3ac3fd2e6` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:54:03.779394 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cd6bfe9-30d5-44b4-8617-0263fb637e72 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2d2c614e07df4a29b2cd84c260fdc93a, user_id=616a497708664cca8bf7cfe93cbfd01f, role_id=b98dfb3b9fee4644af2887a3ac3fd2e6)` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 10:54:03.782055 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cd6bfe9-30d5-44b4-8617-0263fb637e72 admin admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 10:54:03.789006 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 812/1602] 10.222.0.25 () {58 vars in 958 bytes} [Sun Aug 7 10:54:03 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 07 10:54:03.789962 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05c5e97c-e604-4d6e-9a43-352c01e1acce None None] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 10:54:03.797071 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 791/1603] 10.222.0.25 () {60 vars in 1483 bytes} [Sun Aug 7 10:54:03 2022] PUT /identity/v3/projects/2d2c614e07df4a29b2cd84c260fdc93a/users/616a497708664cca8bf7cfe93cbfd01f/roles/b98dfb3b9fee4644af2887a3ac3fd2e6 => generated 0 bytes in 100 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 07 10:54:03.797681 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cd6bfe9-30d5-44b4-8617-0263fb637e72 admin admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 10:54:03.799837 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05c5e97c-e604-4d6e-9a43-352c01e1acce None None] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 10:54:03.856862 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4062f28a-ebfb-4f5e-8c46-376a9803738a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5ac47e3b09f43d19a23a0f457fe9e23', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 10:54:03.857666 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4062f28a-ebfb-4f5e-8c46-376a9803738a admin admin] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:54:03.858024 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4062f28a-ebfb-4f5e-8c46-376a9803738a admin admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:54:03.858621 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4062f28a-ebfb-4f5e-8c46-376a9803738a admin admin] PATH_INFO: `/v3/roles` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:54:03.859365 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4062f28a-ebfb-4f5e-8c46-376a9803738a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 10:54:03.859798 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cd6bfe9-30d5-44b4-8617-0263fb637e72 admin admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 10:54:03.861087 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4062f28a-ebfb-4f5e-8c46-376a9803738a admin admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 10:54:03.865240 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92b657e0-d7c3-40f3-981b-ab82bdb02982 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5ac47e3b09f43d19a23a0f457fe9e23', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 10:54:03.865892 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92b657e0-d7c3-40f3-981b-ab82bdb02982 admin admin] REQUEST_METHOD: `PUT` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:54:03.866169 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92b657e0-d7c3-40f3-981b-ab82bdb02982 admin admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:54:03.866443 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92b657e0-d7c3-40f3-981b-ab82bdb02982 admin admin] PATH_INFO: `/v3/projects/8ac25ca8dc94488ba8404d09d27d034d/users/c9b20f3a7c394108a4b66fd2ade68cac/roles/b98dfb3b9fee4644af2887a3ac3fd2e6` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:54:03.874881 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 813/1604] 10.222.0.25 () {58 vars in 1122 bytes} [Sun Aug 7 10:54:03 2022] GET /identity/v3/roles => generated 1420 bytes in 82 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 07 10:54:03.874881 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4062f28a-ebfb-4f5e-8c46-376a9803738a admin admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 10:54:03.881416 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92b657e0-d7c3-40f3-981b-ab82bdb02982 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8ac25ca8dc94488ba8404d09d27d034d, user_id=c9b20f3a7c394108a4b66fd2ade68cac, role_id=b98dfb3b9fee4644af2887a3ac3fd2e6)` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 10:54:03.882086 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4062f28a-ebfb-4f5e-8c46-376a9803738a admin admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 10:54:03.884079 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92b657e0-d7c3-40f3-981b-ab82bdb02982 admin admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 10:54:03.886941 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-363c9d31-79d4-46eb-b160-b7f9b4706ea8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5ac47e3b09f43d19a23a0f457fe9e23', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 10:54:03.887681 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-363c9d31-79d4-46eb-b160-b7f9b4706ea8 admin admin] REQUEST_METHOD: `PUT` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:54:03.887890 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-363c9d31-79d4-46eb-b160-b7f9b4706ea8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:54:03.888168 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-363c9d31-79d4-46eb-b160-b7f9b4706ea8 admin admin] PATH_INFO: `/v3/projects/3c3be73ff916420f8c9eb418ef5f6716/users/2b64b05230b940eba992a063ea7cd778/roles/b98dfb3b9fee4644af2887a3ac3fd2e6` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:54:03.901704 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 792/1605] 10.222.0.25 () {60 vars in 1483 bytes} [Sun Aug 7 10:54:03 2022] PUT /identity/v3/projects/8ac25ca8dc94488ba8404d09d27d034d/users/c9b20f3a7c394108a4b66fd2ade68cac/roles/b98dfb3b9fee4644af2887a3ac3fd2e6 => generated 0 bytes in 104 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 07 10:54:03.902533 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92b657e0-d7c3-40f3-981b-ab82bdb02982 admin admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 10:54:03.907751 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-363c9d31-79d4-46eb-b160-b7f9b4706ea8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3c3be73ff916420f8c9eb418ef5f6716, user_id=2b64b05230b940eba992a063ea7cd778, role_id=b98dfb3b9fee4644af2887a3ac3fd2e6)` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 10:54:03.911020 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-363c9d31-79d4-46eb-b160-b7f9b4706ea8 admin admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 10:54:03.921590 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 10:54:03.928070 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 814/1606] 10.222.0.25 () {60 vars in 1483 bytes} [Sun Aug 7 10:54:03 2022] PUT /identity/v3/projects/3c3be73ff916420f8c9eb418ef5f6716/users/2b64b05230b940eba992a063ea7cd778/roles/b98dfb3b9fee4644af2887a3ac3fd2e6 => generated 0 bytes in 56 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 07 10:54:03.929182 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-363c9d31-79d4-46eb-b160-b7f9b4706ea8 admin admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 10:54:03.964671 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 10:54:03.978423 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92b657e0-d7c3-40f3-981b-ab82bdb02982 admin admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 10:54:03.992410 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-363c9d31-79d4-46eb-b160-b7f9b4706ea8 admin admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 10:54:03.997054 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dadcb2cd-af10-4431-b69e-fd8bad77f4fb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5ac47e3b09f43d19a23a0f457fe9e23', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 10:54:03.997712 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dadcb2cd-af10-4431-b69e-fd8bad77f4fb admin admin] REQUEST_METHOD: `PUT` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:54:03.997987 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dadcb2cd-af10-4431-b69e-fd8bad77f4fb admin admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:54:03.998257 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dadcb2cd-af10-4431-b69e-fd8bad77f4fb admin admin] PATH_INFO: `/v3/projects/639f69c020004f59b20d086d5f4149e5/users/60bf4ef3bca04a418ee55888f86384b1/roles/b98dfb3b9fee4644af2887a3ac3fd2e6` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:54:04.014415 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dadcb2cd-af10-4431-b69e-fd8bad77f4fb admin admin] RBAC: Authorizing `identity:create_grant(project_id=639f69c020004f59b20d086d5f4149e5, user_id=60bf4ef3bca04a418ee55888f86384b1, role_id=b98dfb3b9fee4644af2887a3ac3fd2e6)` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 10:54:04.017373 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dadcb2cd-af10-4431-b69e-fd8bad77f4fb admin admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 10:54:04.031583 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 815/1607] 10.222.0.25 () {60 vars in 1483 bytes} [Sun Aug 7 10:54:03 2022] PUT /identity/v3/projects/639f69c020004f59b20d086d5f4149e5/users/60bf4ef3bca04a418ee55888f86384b1/roles/b98dfb3b9fee4644af2887a3ac3fd2e6 => generated 0 bytes in 103 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 07 10:54:04.032796 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de213abe-4980-4ff0-94cf-8ce682bbadf3 None None] REQUEST_METHOD: `POST` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:54:04.033114 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de213abe-4980-4ff0-94cf-8ce682bbadf3 None None] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:54:04.033390 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de213abe-4980-4ff0-94cf-8ce682bbadf3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:54:04.036879 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb679dba-0801-4cd3-8dc9-208e4cf706b6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5ac47e3b09f43d19a23a0f457fe9e23', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 10:54:04.037669 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb679dba-0801-4cd3-8dc9-208e4cf706b6 admin admin] REQUEST_METHOD: `PUT` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:54:04.038057 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb679dba-0801-4cd3-8dc9-208e4cf706b6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:54:04.038497 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb679dba-0801-4cd3-8dc9-208e4cf706b6 admin admin] PATH_INFO: `/v3/projects/9df6ca365836428198f2d20fd9829feb/users/c3a12c3144844d32812e010550bb0921/roles/b98dfb3b9fee4644af2887a3ac3fd2e6` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:54:04.057597 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb679dba-0801-4cd3-8dc9-208e4cf706b6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9df6ca365836428198f2d20fd9829feb, user_id=c3a12c3144844d32812e010550bb0921, role_id=b98dfb3b9fee4644af2887a3ac3fd2e6)` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 10:54:04.060790 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb679dba-0801-4cd3-8dc9-208e4cf706b6 admin admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 10:54:04.061828 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.auth.core [None req-de213abe-4980-4ff0-94cf-8ce682bbadf3 None None] MFA Rules not processed for user `12de1ec12eb746bc9bb163bff5b464ed`. Rule list: `[]` (Enabled: `True`). {{(pid=80144) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 10:54:04.063064 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.fernet_utils [None req-de213abe-4980-4ff0-94cf-8ce682bbadf3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80144) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 10:54:04.074672 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 793/1608] 10.222.0.25 () {60 vars in 1483 bytes} [Sun Aug 7 10:54:03 2022] PUT /identity/v3/projects/9df6ca365836428198f2d20fd9829feb/users/c3a12c3144844d32812e010550bb0921/roles/b98dfb3b9fee4644af2887a3ac3fd2e6 => generated 0 bytes in 172 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 07 10:54:04.076332 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03281c80-2841-44d3-a1ca-cad5d466ed48 None None] REQUEST_METHOD: `POST` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:54:04.076618 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03281c80-2841-44d3-a1ca-cad5d466ed48 None None] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:54:04.076891 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03281c80-2841-44d3-a1ca-cad5d466ed48 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:54:04.095847 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.auth.core [None req-03281c80-2841-44d3-a1ca-cad5d466ed48 None None] MFA Rules not processed for user `bac4e6b76fe64217ac8fa44e71cffe6a`. Rule list: `[]` (Enabled: `True`). {{(pid=80143) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 10:54:04.097752 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-03281c80-2841-44d3-a1ca-cad5d466ed48 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 10:54:04.131542 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 816/1609] 10.222.0.25 () {58 vars in 958 bytes} [Sun Aug 7 10:54:04 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 07 10:54:04.136349 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de213abe-4980-4ff0-94cf-8ce682bbadf3 None None] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 10:54:04.160982 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 794/1610] 10.222.0.25 () {58 vars in 958 bytes} [Sun Aug 7 10:54:04 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 07 10:54:04.161648 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03281c80-2841-44d3-a1ca-cad5d466ed48 None None] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 10:54:04.173680 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03281c80-2841-44d3-a1ca-cad5d466ed48 None None] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 10:54:04.180034 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-564cdca0-c033-4baa-bc8a-cda6c549d1d5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5ac47e3b09f43d19a23a0f457fe9e23', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 10:54:04.180729 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-564cdca0-c033-4baa-bc8a-cda6c549d1d5 admin admin] REQUEST_METHOD: `PUT` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:54:04.181086 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-564cdca0-c033-4baa-bc8a-cda6c549d1d5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:54:04.181370 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-564cdca0-c033-4baa-bc8a-cda6c549d1d5 admin admin] PATH_INFO: `/v3/projects/5a16513a1528460cb138b3d11c58e815/users/fe6827bf0a3e4dad816e42f9bb9e0657/roles/81b1f1bcee2a473088fbbd327c1c5b23` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:54:04.196872 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-564cdca0-c033-4baa-bc8a-cda6c549d1d5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5a16513a1528460cb138b3d11c58e815, user_id=fe6827bf0a3e4dad816e42f9bb9e0657, role_id=81b1f1bcee2a473088fbbd327c1c5b23)` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 10:54:04.199088 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de213abe-4980-4ff0-94cf-8ce682bbadf3 None None] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 10:54:04.199621 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-564cdca0-c033-4baa-bc8a-cda6c549d1d5 admin admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 10:54:04.212902 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 795/1611] 10.222.0.25 () {60 vars in 1483 bytes} [Sun Aug 7 10:54:04 2022] PUT /identity/v3/projects/5a16513a1528460cb138b3d11c58e815/users/fe6827bf0a3e4dad816e42f9bb9e0657/roles/81b1f1bcee2a473088fbbd327c1c5b23 => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 07 10:54:04.214224 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ec102ef-384b-4c99-9797-608d1a5316a5 None None] REQUEST_METHOD: `POST` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:54:04.214513 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ec102ef-384b-4c99-9797-608d1a5316a5 None None] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:54:04.214789 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ec102ef-384b-4c99-9797-608d1a5316a5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:54:04.227243 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 10:54:04.243109 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.auth.core [None req-1ec102ef-384b-4c99-9797-608d1a5316a5 None None] MFA Rules not processed for user `c9b20f3a7c394108a4b66fd2ade68cac`. Rule list: `[]` (Enabled: `True`). {{(pid=80143) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 10:54:04.244420 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-1ec102ef-384b-4c99-9797-608d1a5316a5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 10:54:04.252352 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-506cdfcd-e24f-49b7-8ac4-1ec7b247bb48 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be1283c9bf47424c9aa3ad27cab03027', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6de1f3cac02c4bcdb05dd2337176033d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 10:54:04.252997 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-506cdfcd-e24f-49b7-8ac4-1ec7b247bb48 service neutron] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:54:04.253250 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-506cdfcd-e24f-49b7-8ac4-1ec7b247bb48 service neutron] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:54:04.253528 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-506cdfcd-e24f-49b7-8ac4-1ec7b247bb48 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:54:04.262559 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-506cdfcd-e24f-49b7-8ac4-1ec7b247bb48 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 10:54:04.264008 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-506cdfcd-e24f-49b7-8ac4-1ec7b247bb48 service neutron] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 10:54:04.274159 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 817/1612] 10.222.0.25 () {60 vars in 1365 bytes} [Sun Aug 7 10:54:04 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 142 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 07 10:54:04.275165 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e94c4665-3f22-4220-a9eb-a8f4fc03f9c7 None None] REQUEST_METHOD: `POST` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:54:04.275458 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e94c4665-3f22-4220-a9eb-a8f4fc03f9c7 None None] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:54:04.275736 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e94c4665-3f22-4220-a9eb-a8f4fc03f9c7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:54:04.293661 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 10:54:04.296862 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.auth.core [None req-e94c4665-3f22-4220-a9eb-a8f4fc03f9c7 None None] MFA Rules not processed for user `bac4e6b76fe64217ac8fa44e71cffe6a`. Rule list: `[]` (Enabled: `True`). {{(pid=80144) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 10:54:04.298174 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.fernet_utils [None req-e94c4665-3f22-4220-a9eb-a8f4fc03f9c7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80144) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 10:54:04.316729 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 796/1613] 10.222.0.25 () {58 vars in 958 bytes} [Sun Aug 7 10:54:04 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 07 10:54:04.320249 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c712fa11-ce42-4a87-9894-23673cd9ee89 None None] REQUEST_METHOD: `POST` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:54:04.320543 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c712fa11-ce42-4a87-9894-23673cd9ee89 None None] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:54:04.322335 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c712fa11-ce42-4a87-9894-23673cd9ee89 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:54:04.350852 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.auth.core [None req-c712fa11-ce42-4a87-9894-23673cd9ee89 None None] MFA Rules not processed for user `60bf4ef3bca04a418ee55888f86384b1`. Rule list: `[]` (Enabled: `True`). {{(pid=80143) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 10:54:04.352222 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-c712fa11-ce42-4a87-9894-23673cd9ee89 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 10:54:04.368148 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 818/1614] 10.222.0.25 () {58 vars in 958 bytes} [Sun Aug 7 10:54:04 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 07 10:54:04.372771 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-effdce89-5185-4137-8ee0-f9738bc09aad None None] REQUEST_METHOD: `POST` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:54:04.372934 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-effdce89-5185-4137-8ee0-f9738bc09aad None None] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:54:04.373174 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-effdce89-5185-4137-8ee0-f9738bc09aad None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:54:04.394684 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.auth.core [None req-effdce89-5185-4137-8ee0-f9738bc09aad None None] MFA Rules not processed for user `bac4e6b76fe64217ac8fa44e71cffe6a`. Rule list: `[]` (Enabled: `True`). {{(pid=80144) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 10:54:04.395698 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.fernet_utils [None req-effdce89-5185-4137-8ee0-f9738bc09aad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80144) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 10:54:04.403395 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 819/1615] 10.222.0.25 () {58 vars in 958 bytes} [Sun Aug 7 10:54:04 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 07 10:54:04.404605 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbff1c55-6b6d-42ba-be58-ba4bfe300297 None None] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:54:04.405119 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbff1c55-6b6d-42ba-be58-ba4bfe300297 None None] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:54:04.405119 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbff1c55-6b6d-42ba-be58-ba4bfe300297 None None] PATH_INFO: `/` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:54:04.411498 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 820/1616] 10.222.0.25 () {54 vars in 962 bytes} [Sun Aug 7 10:54:04 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 07 10:54:04.412394 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbff1c55-6b6d-42ba-be58-ba4bfe300297 None None] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 10:54:04.441477 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 797/1617] 10.222.0.25 () {58 vars in 958 bytes} [Sun Aug 7 10:54:04 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 07 10:54:04.445541 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c712fa11-ce42-4a87-9894-23673cd9ee89 None None] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 10:54:04.459090 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c712fa11-ce42-4a87-9894-23673cd9ee89 None None] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 10:54:04.464693 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0380cf22-3e6d-41ca-84d7-cd4f913d3edc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5ac47e3b09f43d19a23a0f457fe9e23', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 10:54:04.465409 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0380cf22-3e6d-41ca-84d7-cd4f913d3edc admin admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:54:04.465751 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0380cf22-3e6d-41ca-84d7-cd4f913d3edc admin admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:54:04.466127 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0380cf22-3e6d-41ca-84d7-cd4f913d3edc admin admin] PATH_INFO: `/v3/roles` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:54:04.466839 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0380cf22-3e6d-41ca-84d7-cd4f913d3edc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 10:54:04.468914 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0380cf22-3e6d-41ca-84d7-cd4f913d3edc admin admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 10:54:04.485332 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 798/1618] 10.222.0.25 () {58 vars in 1122 bytes} [Sun Aug 7 10:54:04 2022] GET /identity/v3/roles => generated 1420 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 07 10:54:04.486196 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0380cf22-3e6d-41ca-84d7-cd4f913d3edc admin admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 10:54:04.497827 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0380cf22-3e6d-41ca-84d7-cd4f913d3edc admin admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 10:54:04.498010 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbff1c55-6b6d-42ba-be58-ba4bfe300297 None None] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 10:54:04.504522 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e669bcbf-bb4a-4bd1-8e36-52b54329ad00 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be1283c9bf47424c9aa3ad27cab03027', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6de1f3cac02c4bcdb05dd2337176033d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 10:54:04.505327 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e669bcbf-bb4a-4bd1-8e36-52b54329ad00 service neutron] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:54:04.505693 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e669bcbf-bb4a-4bd1-8e36-52b54329ad00 service neutron] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:54:04.505878 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e669bcbf-bb4a-4bd1-8e36-52b54329ad00 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:54:04.506098 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2078d53c-f46c-4178-b248-aab55b253f39 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5ac47e3b09f43d19a23a0f457fe9e23', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 10:54:04.506768 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2078d53c-f46c-4178-b248-aab55b253f39 admin admin] REQUEST_METHOD: `POST` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:54:04.507093 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2078d53c-f46c-4178-b248-aab55b253f39 admin admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:54:04.507630 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2078d53c-f46c-4178-b248-aab55b253f39 admin admin] PATH_INFO: `/v3/projects` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:54:04.508272 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2078d53c-f46c-4178-b248-aab55b253f39 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 10:54:04.510331 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2078d53c-f46c-4178-b248-aab55b253f39 admin admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 10:54:04.517709 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e669bcbf-bb4a-4bd1-8e36-52b54329ad00 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 10:54:04.519307 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e669bcbf-bb4a-4bd1-8e36-52b54329ad00 service neutron] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 10:54:04.529889 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 821/1619] 10.222.0.25 () {60 vars in 1365 bytes} [Sun Aug 7 10:54:04 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 118 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 07 10:54:04.530762 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e669bcbf-bb4a-4bd1-8e36-52b54329ad00 service neutron] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 10:54:04.539632 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e669bcbf-bb4a-4bd1-8e36-52b54329ad00 service neutron] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 10:54:04.545958 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 799/1620] 10.222.0.25 () {60 vars in 1153 bytes} [Sun Aug 7 10:54:04 2022] POST /identity/v3/projects => generated 363 bytes in 57 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 07 10:54:04.546119 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ee13be0-c3d2-4450-b7b3-a13631ac4f21 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be1283c9bf47424c9aa3ad27cab03027', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6de1f3cac02c4bcdb05dd2337176033d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 10:54:04.546119 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ee13be0-c3d2-4450-b7b3-a13631ac4f21 service neutron] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:54:04.546119 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ee13be0-c3d2-4450-b7b3-a13631ac4f21 service neutron] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:54:04.546377 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b553a897-aa11-4a91-a082-c67e8028a50e None None] REQUEST_METHOD: `POST` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:54:04.546454 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ee13be0-c3d2-4450-b7b3-a13631ac4f21 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:54:04.546535 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b553a897-aa11-4a91-a082-c67e8028a50e None None] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:54:04.546535 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b553a897-aa11-4a91-a082-c67e8028a50e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:54:04.557893 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ee13be0-c3d2-4450-b7b3-a13631ac4f21 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 10:54:04.560344 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ee13be0-c3d2-4450-b7b3-a13631ac4f21 service neutron] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 10:54:04.571475 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 822/1621] 10.222.0.25 () {60 vars in 1365 bytes} [Sun Aug 7 10:54:04 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 07 10:54:04.572393 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ee13be0-c3d2-4450-b7b3-a13631ac4f21 service neutron] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 10:54:04.573778 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.fernet_utils [None req-3ee13be0-c3d2-4450-b7b3-a13631ac4f21 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80144) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 10:54:04.585586 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ee13be0-c3d2-4450-b7b3-a13631ac4f21 service neutron] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 10:54:04.588245 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.auth.core [None req-b553a897-aa11-4a91-a082-c67e8028a50e None None] MFA Rules not processed for user `31d16d1b96104b4eb690398432b9829e`. Rule list: `[]` (Enabled: `True`). {{(pid=80143) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 10:54:04.589525 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-b553a897-aa11-4a91-a082-c67e8028a50e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 10:54:04.646524 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6976ca05-23a2-476b-94f7-4b03fce0b388 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be1283c9bf47424c9aa3ad27cab03027', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6de1f3cac02c4bcdb05dd2337176033d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 10:54:04.647175 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6976ca05-23a2-476b-94f7-4b03fce0b388 service neutron] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:54:04.647463 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6976ca05-23a2-476b-94f7-4b03fce0b388 service neutron] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:54:04.647740 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6976ca05-23a2-476b-94f7-4b03fce0b388 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:54:04.657913 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=100 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 10:54:04.675118 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 800/1622] 10.222.0.25 () {58 vars in 1059 bytes} [Sun Aug 7 10:54:04 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 07 10:54:04.677429 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20f64a7e-4ff5-4313-a64a-0c30fa93d2a6 None None] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:54:04.678030 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20f64a7e-4ff5-4313-a64a-0c30fa93d2a6 None None] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:54:04.678607 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20f64a7e-4ff5-4313-a64a-0c30fa93d2a6 None None] PATH_INFO: `/` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:54:04.680309 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 801/1623] 10.222.0.25 () {54 vars in 962 bytes} [Sun Aug 7 10:54:04 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 07 10:54:04.681010 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20f64a7e-4ff5-4313-a64a-0c30fa93d2a6 None None] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 10:54:04.720357 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 10:54:04.732149 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6976ca05-23a2-476b-94f7-4b03fce0b388 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 10:54:04.734062 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6976ca05-23a2-476b-94f7-4b03fce0b388 service neutron] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 10:54:04.746105 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 823/1624] 10.222.0.25 () {60 vars in 1365 bytes} [Sun Aug 7 10:54:04 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 174 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 07 10:54:04.748284 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6976ca05-23a2-476b-94f7-4b03fce0b388 service neutron] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 10:54:04.771142 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6976ca05-23a2-476b-94f7-4b03fce0b388 service neutron] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 10:54:04.773052 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20f64a7e-4ff5-4313-a64a-0c30fa93d2a6 None None] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 10:54:04.779197 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e4cff74-38c8-4fb3-9310-270026c0b21d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5ac47e3b09f43d19a23a0f457fe9e23', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 10:54:04.780071 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e4cff74-38c8-4fb3-9310-270026c0b21d admin admin] REQUEST_METHOD: `PUT` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:54:04.780857 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e4cff74-38c8-4fb3-9310-270026c0b21d admin admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:54:04.781212 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e4cff74-38c8-4fb3-9310-270026c0b21d admin admin] PATH_INFO: `/v3/projects/5a16513a1528460cb138b3d11c58e815/users/fe6827bf0a3e4dad816e42f9bb9e0657/roles/81b1f1bcee2a473088fbbd327c1c5b23` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:54:04.782384 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93a900b5-92b0-4e82-bbbd-8aef5e620745 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5ac47e3b09f43d19a23a0f457fe9e23', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 10:54:04.782853 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93a900b5-92b0-4e82-bbbd-8aef5e620745 admin admin] REQUEST_METHOD: `POST` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:54:04.783222 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93a900b5-92b0-4e82-bbbd-8aef5e620745 admin admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:54:04.787251 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93a900b5-92b0-4e82-bbbd-8aef5e620745 admin admin] PATH_INFO: `/v3/users` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:54:04.787510 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e4cff74-38c8-4fb3-9310-270026c0b21d admin admin] RBAC: Authorizing `identity:create_grant(project_id=5a16513a1528460cb138b3d11c58e815, user_id=fe6827bf0a3e4dad816e42f9bb9e0657, role_id=81b1f1bcee2a473088fbbd327c1c5b23)` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 10:54:04.787928 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93a900b5-92b0-4e82-bbbd-8aef5e620745 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 10:54:04.788418 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e4cff74-38c8-4fb3-9310-270026c0b21d admin admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 10:54:04.789918 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93a900b5-92b0-4e82-bbbd-8aef5e620745 admin admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 10:54:04.809717 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 802/1625] 10.222.0.25 () {60 vars in 1483 bytes} [Sun Aug 7 10:54:04 2022] PUT /identity/v3/projects/5a16513a1528460cb138b3d11c58e815/users/fe6827bf0a3e4dad816e42f9bb9e0657/roles/81b1f1bcee2a473088fbbd327c1c5b23 => generated 0 bytes in 120 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 07 10:54:04.809717 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e4cff74-38c8-4fb3-9310-270026c0b21d admin admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 10:54:04.840104 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 824/1626] 10.222.0.25 () {60 vars in 1144 bytes} [Sun Aug 7 10:54:04 2022] POST /identity/v3/users => generated 288 bytes in 90 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 07 10:54:04.840759 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03c6737c-edfd-4ea8-b9ad-2dbc6e043ed6 None None] REQUEST_METHOD: `POST` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:54:04.841061 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03c6737c-edfd-4ea8-b9ad-2dbc6e043ed6 None None] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:54:04.841430 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03c6737c-edfd-4ea8-b9ad-2dbc6e043ed6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:54:04.876778 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.auth.core [None req-03c6737c-edfd-4ea8-b9ad-2dbc6e043ed6 None None] MFA Rules not processed for user `31d16d1b96104b4eb690398432b9829e`. Rule list: `[]` (Enabled: `True`). {{(pid=80144) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 10:54:04.879705 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.fernet_utils [None req-03c6737c-edfd-4ea8-b9ad-2dbc6e043ed6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80144) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 10:54:04.937565 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e4cff74-38c8-4fb3-9310-270026c0b21d admin admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 10:54:04.946916 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6b5cd25-8002-4134-be52-3e6a756420d9 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31d16d1b96104b4eb690398432b9829e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6de1f3cac02c4bcdb05dd2337176033d', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'reader', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 10:54:04.948022 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6b5cd25-8002-4134-be52-3e6a756420d9 service nova] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:54:04.948480 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6b5cd25-8002-4134-be52-3e6a756420d9 service nova] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:54:04.948699 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6b5cd25-8002-4134-be52-3e6a756420d9 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:54:04.985320 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 825/1627] 10.222.0.25 () {58 vars in 1059 bytes} [Sun Aug 7 10:54:04 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 147 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 07 10:54:04.987226 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4158920-8d24-4032-ad60-b5eff459b607 None None] REQUEST_METHOD: `POST` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:54:04.987789 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4158920-8d24-4032-ad60-b5eff459b607 None None] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:54:04.988195 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4158920-8d24-4032-ad60-b5eff459b607 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:54:05.021880 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.auth.core [None req-b4158920-8d24-4032-ad60-b5eff459b607 None None] MFA Rules not processed for user `fe6827bf0a3e4dad816e42f9bb9e0657`. Rule list: `[]` (Enabled: `True`). {{(pid=80144) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 10:54:05.023710 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.fernet_utils [None req-b4158920-8d24-4032-ad60-b5eff459b607 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80144) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 10:54:05.039014 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6b5cd25-8002-4134-be52-3e6a756420d9 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 10:54:05.041446 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6b5cd25-8002-4134-be52-3e6a756420d9 service nova] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 10:54:05.054479 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 803/1628] 10.222.0.25 () {60 vars in 1365 bytes} [Sun Aug 7 10:54:04 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 252 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 07 10:54:05.054915 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6b5cd25-8002-4134-be52-3e6a756420d9 service nova] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 10:54:05.193581 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 826/1629] 10.222.0.25 () {58 vars in 958 bytes} [Sun Aug 7 10:54:04 2022] POST /identity/v3/auth/tokens => generated 3309 bytes in 208 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 07 10:54:05.194598 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4158920-8d24-4032-ad60-b5eff459b607 None None] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 10:54:05.206097 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 10:54:05.210104 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6b5cd25-8002-4134-be52-3e6a756420d9 service nova] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 10:54:05.213664 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 10:54:05.223470 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77835694-6729-422b-af62-f3dcfd70cfdd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5ac47e3b09f43d19a23a0f457fe9e23', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 10:54:05.225279 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77835694-6729-422b-af62-f3dcfd70cfdd admin admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:54:05.225729 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77835694-6729-422b-af62-f3dcfd70cfdd admin admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:54:05.227315 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77835694-6729-422b-af62-f3dcfd70cfdd admin admin] PATH_INFO: `/v3/roles` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:54:05.228350 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4158920-8d24-4032-ad60-b5eff459b607 None None] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 10:54:05.229104 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77835694-6729-422b-af62-f3dcfd70cfdd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 10:54:05.258443 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77835694-6729-422b-af62-f3dcfd70cfdd admin admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 10:54:05.273852 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27c53995-da11-448d-8e5a-0bf05c516e75 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31d16d1b96104b4eb690398432b9829e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6de1f3cac02c4bcdb05dd2337176033d', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'reader', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 10:54:05.275594 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27c53995-da11-448d-8e5a-0bf05c516e75 service nova] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:54:05.277329 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27c53995-da11-448d-8e5a-0bf05c516e75 service nova] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:54:05.278708 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27c53995-da11-448d-8e5a-0bf05c516e75 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:54:05.294420 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 804/1630] 10.222.0.25 () {58 vars in 1122 bytes} [Sun Aug 7 10:54:05 2022] GET /identity/v3/roles => generated 1420 bytes in 239 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 07 10:54:05.296087 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77835694-6729-422b-af62-f3dcfd70cfdd admin admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 10:54:05.443958 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27c53995-da11-448d-8e5a-0bf05c516e75 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 10:54:05.448902 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27c53995-da11-448d-8e5a-0bf05c516e75 service nova] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 10:54:05.465030 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77835694-6729-422b-af62-f3dcfd70cfdd admin admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 10:54:05.474951 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 827/1631] 10.222.0.25 () {60 vars in 1365 bytes} [Sun Aug 7 10:54:05 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 280 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 07 10:54:05.476591 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c619794a-7f4c-43c2-9044-3a7c52af2c4d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88dbd6dfd1164994a3e16da553561910', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6de1f3cac02c4bcdb05dd2337176033d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 10:54:05.476969 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27c53995-da11-448d-8e5a-0bf05c516e75 service nova] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 10:54:05.477672 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c619794a-7f4c-43c2-9044-3a7c52af2c4d service cinder] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:54:05.478441 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c619794a-7f4c-43c2-9044-3a7c52af2c4d service cinder] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:54:05.479731 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c619794a-7f4c-43c2-9044-3a7c52af2c4d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:54:05.513891 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c619794a-7f4c-43c2-9044-3a7c52af2c4d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 10:54:05.515440 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27c53995-da11-448d-8e5a-0bf05c516e75 service nova] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 10:54:05.521362 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c619794a-7f4c-43c2-9044-3a7c52af2c4d service cinder] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 10:54:05.522607 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d72ed6c0-3a7b-4645-a4c0-59b0ee803b6f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5ac47e3b09f43d19a23a0f457fe9e23', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 10:54:05.523407 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d72ed6c0-3a7b-4645-a4c0-59b0ee803b6f admin admin] REQUEST_METHOD: `PUT` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:54:05.524302 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d72ed6c0-3a7b-4645-a4c0-59b0ee803b6f admin admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:54:05.525164 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d72ed6c0-3a7b-4645-a4c0-59b0ee803b6f admin admin] PATH_INFO: `/v3/projects/7819c84c6c844cbfb9b334ed8026ccb3/users/54b0402fadff4656aad6761d9c58f457/roles/81b1f1bcee2a473088fbbd327c1c5b23` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:54:05.537445 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 805/1632] 10.222.0.25 () {60 vars in 1365 bytes} [Sun Aug 7 10:54:05 2022] GET /identity/v3/auth/tokens => generated 3309 bytes in 243 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 07 10:54:05.539838 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91b9fb54-5989-4a98-bf6a-315ecc5d66dd None None] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:54:05.540368 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91b9fb54-5989-4a98-bf6a-315ecc5d66dd None None] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:54:05.540917 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91b9fb54-5989-4a98-bf6a-315ecc5d66dd None None] PATH_INFO: `/` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:54:05.543677 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 806/1633] 10.222.0.25 () {54 vars in 912 bytes} [Sun Aug 7 10:54:05 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 07 10:54:05.545905 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d72ed6c0-3a7b-4645-a4c0-59b0ee803b6f admin admin] RBAC: Authorizing `identity:create_grant(project_id=7819c84c6c844cbfb9b334ed8026ccb3, user_id=54b0402fadff4656aad6761d9c58f457, role_id=81b1f1bcee2a473088fbbd327c1c5b23)` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 10:54:05.548769 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d72ed6c0-3a7b-4645-a4c0-59b0ee803b6f admin admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 10:54:05.553783 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73429c37-cf32-4b1e-9ae9-7dae863df5df None None] REQUEST_METHOD: `POST` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:54:05.554155 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73429c37-cf32-4b1e-9ae9-7dae863df5df None None] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:54:05.554738 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73429c37-cf32-4b1e-9ae9-7dae863df5df None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:54:05.568348 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 828/1634] 10.222.0.25 () {60 vars in 1483 bytes} [Sun Aug 7 10:54:05 2022] PUT /identity/v3/projects/7819c84c6c844cbfb9b334ed8026ccb3/users/54b0402fadff4656aad6761d9c58f457/roles/81b1f1bcee2a473088fbbd327c1c5b23 => generated 0 bytes in 93 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 07 10:54:05.573056 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d72ed6c0-3a7b-4645-a4c0-59b0ee803b6f admin admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 10:54:05.587945 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.auth.core [None req-73429c37-cf32-4b1e-9ae9-7dae863df5df None None] MFA Rules not processed for user `31d16d1b96104b4eb690398432b9829e`. Rule list: `[]` (Enabled: `True`). {{(pid=80143) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 10:54:05.589232 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-73429c37-cf32-4b1e-9ae9-7dae863df5df None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 10:54:05.692818 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d72ed6c0-3a7b-4645-a4c0-59b0ee803b6f admin admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 10:54:05.710722 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ed426b1-24db-4f0d-86ca-dbcd619d0adc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5ac47e3b09f43d19a23a0f457fe9e23', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 10:54:05.711682 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ed426b1-24db-4f0d-86ca-dbcd619d0adc admin admin] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:54:05.712198 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ed426b1-24db-4f0d-86ca-dbcd619d0adc admin admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:54:05.712784 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ed426b1-24db-4f0d-86ca-dbcd619d0adc admin admin] PATH_INFO: `/v3/roles` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:54:05.715482 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ed426b1-24db-4f0d-86ca-dbcd619d0adc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 10:54:05.719305 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ed426b1-24db-4f0d-86ca-dbcd619d0adc admin admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 10:54:05.744896 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 829/1635] 10.222.0.25 () {58 vars in 1122 bytes} [Sun Aug 7 10:54:05 2022] GET /identity/v3/roles => generated 1420 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 07 10:54:05.747134 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ed426b1-24db-4f0d-86ca-dbcd619d0adc admin admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 10:54:05.760833 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 807/1636] 10.222.0.25 () {58 vars in 1009 bytes} [Sun Aug 7 10:54:05 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 209 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 07 10:54:05.761782 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73429c37-cf32-4b1e-9ae9-7dae863df5df None None] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 10:54:05.794324 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73429c37-cf32-4b1e-9ae9-7dae863df5df None None] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 10:54:05.809476 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bcf0aff-99af-4533-891d-02005b6b7d5d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5ac47e3b09f43d19a23a0f457fe9e23', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 10:54:05.810093 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bcf0aff-99af-4533-891d-02005b6b7d5d admin admin] REQUEST_METHOD: `PUT` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:54:05.810378 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bcf0aff-99af-4533-891d-02005b6b7d5d admin admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:54:05.811276 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bcf0aff-99af-4533-891d-02005b6b7d5d admin admin] PATH_INFO: `/v3/projects/7819c84c6c844cbfb9b334ed8026ccb3/users/54b0402fadff4656aad6761d9c58f457/roles/81b1f1bcee2a473088fbbd327c1c5b23` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:54:05.819492 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bcf0aff-99af-4533-891d-02005b6b7d5d admin admin] RBAC: Authorizing `identity:create_grant(project_id=7819c84c6c844cbfb9b334ed8026ccb3, user_id=54b0402fadff4656aad6761d9c58f457, role_id=81b1f1bcee2a473088fbbd327c1c5b23)` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 10:54:05.824359 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bcf0aff-99af-4533-891d-02005b6b7d5d admin admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 10:54:05.851895 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 808/1637] 10.222.0.25 () {60 vars in 1483 bytes} [Sun Aug 7 10:54:05 2022] PUT /identity/v3/projects/7819c84c6c844cbfb9b334ed8026ccb3/users/54b0402fadff4656aad6761d9c58f457/roles/81b1f1bcee2a473088fbbd327c1c5b23 => generated 0 bytes in 90 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 07 10:54:05.854907 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bcf0aff-99af-4533-891d-02005b6b7d5d admin admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 10:54:05.869551 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ed426b1-24db-4f0d-86ca-dbcd619d0adc admin admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 10:54:05.911971 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 10:54:05.938558 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-092447c8-2c86-43fb-ba7a-851d41850f0e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88dbd6dfd1164994a3e16da553561910', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6de1f3cac02c4bcdb05dd2337176033d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 10:54:05.939477 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-092447c8-2c86-43fb-ba7a-851d41850f0e service cinder] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:54:05.940144 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-092447c8-2c86-43fb-ba7a-851d41850f0e service cinder] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:54:05.940577 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-092447c8-2c86-43fb-ba7a-851d41850f0e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:54:05.989939 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bcf0aff-99af-4533-891d-02005b6b7d5d admin admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 10:54:06.000843 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9260db2f-9e45-45e7-b960-60267facdccd service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88dbd6dfd1164994a3e16da553561910', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6de1f3cac02c4bcdb05dd2337176033d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 10:54:06.001481 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9260db2f-9e45-45e7-b960-60267facdccd service cinder] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:54:06.001765 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9260db2f-9e45-45e7-b960-60267facdccd service cinder] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:54:06.002407 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9260db2f-9e45-45e7-b960-60267facdccd service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:54:06.008248 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-092447c8-2c86-43fb-ba7a-851d41850f0e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 10:54:06.009986 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-092447c8-2c86-43fb-ba7a-851d41850f0e service cinder] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 10:54:06.021114 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 830/1638] 10.222.0.25 () {60 vars in 1365 bytes} [Sun Aug 7 10:54:05 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 275 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 07 10:54:06.035158 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b49dae0a-5805-42d9-af86-c1654f8ef32d None None] REQUEST_METHOD: `POST` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:54:06.035462 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b49dae0a-5805-42d9-af86-c1654f8ef32d None None] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:54:06.035748 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b49dae0a-5805-42d9-af86-c1654f8ef32d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:54:06.062460 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 10:54:06.103535 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.auth.core [None req-b49dae0a-5805-42d9-af86-c1654f8ef32d None None] MFA Rules not processed for user `54b0402fadff4656aad6761d9c58f457`. Rule list: `[]` (Enabled: `True`). {{(pid=80144) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 10:54:06.104743 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.fernet_utils [None req-b49dae0a-5805-42d9-af86-c1654f8ef32d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80144) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 10:54:06.164004 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9260db2f-9e45-45e7-b960-60267facdccd service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 10:54:06.165481 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9260db2f-9e45-45e7-b960-60267facdccd service cinder] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 10:54:06.187835 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 809/1639] 10.222.0.25 () {60 vars in 1365 bytes} [Sun Aug 7 10:54:05 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 335 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 07 10:54:06.189319 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b5e351b-6120-42de-94a4-8348abd3467f None None] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:54:06.189592 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b5e351b-6120-42de-94a4-8348abd3467f None None] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:54:06.190096 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b5e351b-6120-42de-94a4-8348abd3467f None None] PATH_INFO: `/` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:54:06.191734 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 810/1640] 10.222.0.25 () {54 vars in 912 bytes} [Sun Aug 7 10:54:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 07 10:54:06.199120 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c89c8976-140e-4a6f-841d-535b73f5a1ed None None] REQUEST_METHOD: `POST` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:54:06.199412 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c89c8976-140e-4a6f-841d-535b73f5a1ed None None] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:54:06.199747 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c89c8976-140e-4a6f-841d-535b73f5a1ed None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:54:06.250733 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.auth.core [None req-c89c8976-140e-4a6f-841d-535b73f5a1ed None None] MFA Rules not processed for user `31d16d1b96104b4eb690398432b9829e`. Rule list: `[]` (Enabled: `True`). {{(pid=80143) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 10:54:06.255096 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-c89c8976-140e-4a6f-841d-535b73f5a1ed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 10:54:06.309082 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 811/1641] 10.222.0.25 () {58 vars in 1009 bytes} [Sun Aug 7 10:54:06 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 07 10:54:06.317367 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 831/1642] 10.222.0.25 () {58 vars in 958 bytes} [Sun Aug 7 10:54:06 2022] POST /identity/v3/auth/tokens => generated 3307 bytes in 286 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 07 10:54:06.427694 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c89c8976-140e-4a6f-841d-535b73f5a1ed None None] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 10:54:06.441079 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c89c8976-140e-4a6f-841d-535b73f5a1ed None None] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 10:54:06.453044 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35b3c0e4-8d15-452b-859b-2c1a41baa35f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88dbd6dfd1164994a3e16da553561910', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6de1f3cac02c4bcdb05dd2337176033d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 10:54:06.454608 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35b3c0e4-8d15-452b-859b-2c1a41baa35f service cinder] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:54:06.457153 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35b3c0e4-8d15-452b-859b-2c1a41baa35f service cinder] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:54:06.457893 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35b3c0e4-8d15-452b-859b-2c1a41baa35f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:54:06.482904 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35b3c0e4-8d15-452b-859b-2c1a41baa35f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 10:54:06.490811 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35b3c0e4-8d15-452b-859b-2c1a41baa35f service cinder] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 10:54:06.525986 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 812/1643] 10.222.0.25 () {60 vars in 1365 bytes} [Sun Aug 7 10:54:06 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 99 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 07 10:54:07.121988 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b49dae0a-5805-42d9-af86-c1654f8ef32d None None] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 10:54:07.137074 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b49dae0a-5805-42d9-af86-c1654f8ef32d None None] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 10:54:07.142312 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e06007d-a113-4781-afe9-5f351eaaff50 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88dbd6dfd1164994a3e16da553561910', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6de1f3cac02c4bcdb05dd2337176033d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 10:54:07.143005 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e06007d-a113-4781-afe9-5f351eaaff50 service cinder] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:54:07.143354 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e06007d-a113-4781-afe9-5f351eaaff50 service cinder] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:54:07.143691 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e06007d-a113-4781-afe9-5f351eaaff50 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:54:07.172557 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e06007d-a113-4781-afe9-5f351eaaff50 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 10:54:07.174137 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e06007d-a113-4781-afe9-5f351eaaff50 service cinder] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 10:54:07.201922 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 832/1644] 10.222.0.25 () {60 vars in 1365 bytes} [Sun Aug 7 10:54:07 2022] GET /identity/v3/auth/tokens => generated 3307 bytes in 80 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 07 10:54:07.652436 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35b3c0e4-8d15-452b-859b-2c1a41baa35f service cinder] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 10:54:07.662268 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35b3c0e4-8d15-452b-859b-2c1a41baa35f service cinder] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 10:54:07.673546 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-475746b7-52c2-4590-b5d8-8ac46ce1ea42 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88dbd6dfd1164994a3e16da553561910', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6de1f3cac02c4bcdb05dd2337176033d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 10:54:07.676809 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-475746b7-52c2-4590-b5d8-8ac46ce1ea42 service cinder] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:54:07.677489 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-475746b7-52c2-4590-b5d8-8ac46ce1ea42 service cinder] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:54:07.677919 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-475746b7-52c2-4590-b5d8-8ac46ce1ea42 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:54:07.841549 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-475746b7-52c2-4590-b5d8-8ac46ce1ea42 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 10:54:07.843040 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-475746b7-52c2-4590-b5d8-8ac46ce1ea42 service cinder] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 10:54:07.868541 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 813/1645] 10.222.0.25 () {60 vars in 1365 bytes} [Sun Aug 7 10:54:07 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 216 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 07 10:54:11.847655 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e06007d-a113-4781-afe9-5f351eaaff50 service cinder] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 10:54:11.955892 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=100 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 10:54:11.962696 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e06007d-a113-4781-afe9-5f351eaaff50 service cinder] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 10:54:11.969382 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8230c213-5926-4c81-aca1-7c439b1ac5d3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5ac47e3b09f43d19a23a0f457fe9e23', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 10:54:11.970671 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8230c213-5926-4c81-aca1-7c439b1ac5d3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:54:11.971384 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8230c213-5926-4c81-aca1-7c439b1ac5d3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:54:11.972181 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8230c213-5926-4c81-aca1-7c439b1ac5d3 admin admin] PATH_INFO: `/v3/users/12de1ec12eb746bc9bb163bff5b464ed` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:54:11.973713 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8230c213-5926-4c81-aca1-7c439b1ac5d3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=12de1ec12eb746bc9bb163bff5b464ed)` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 10:54:11.976075 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8230c213-5926-4c81-aca1-7c439b1ac5d3 admin admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 10:54:12.071782 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.notifications [None req-8230c213-5926-4c81-aca1-7c439b1ac5d3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '12de1ec12eb746bc9bb163bff5b464ed'} {{(pid=80144) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 10:54:12.097904 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.notifications [None req-8230c213-5926-4c81-aca1-7c439b1ac5d3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '12de1ec12eb746bc9bb163bff5b464ed'} {{(pid=80144) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 10:54:12.115986 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.notifications [None req-8230c213-5926-4c81-aca1-7c439b1ac5d3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '12de1ec12eb746bc9bb163bff5b464ed'} {{(pid=80144) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 10:54:12.117225 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.notifications [None req-8230c213-5926-4c81-aca1-7c439b1ac5d3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '12de1ec12eb746bc9bb163bff5b464ed'} {{(pid=80144) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 10:54:12.118545 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.notifications [None req-8230c213-5926-4c81-aca1-7c439b1ac5d3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '12de1ec12eb746bc9bb163bff5b464ed'} {{(pid=80144) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 10:54:12.194963 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 833/1646] 10.222.0.25 () {58 vars in 1224 bytes} [Sun Aug 7 10:54:11 2022] DELETE /identity/v3/users/12de1ec12eb746bc9bb163bff5b464ed => generated 0 bytes in 349 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 07 10:54:12.199490 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4786af3-d290-4f05-9c1f-955d00e18420 None None] REQUEST_METHOD: `POST` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:54:12.200029 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4786af3-d290-4f05-9c1f-955d00e18420 None None] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:54:12.200029 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4786af3-d290-4f05-9c1f-955d00e18420 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:54:12.250426 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.auth.core [None req-b4786af3-d290-4f05-9c1f-955d00e18420 None None] MFA Rules not processed for user `bac4e6b76fe64217ac8fa44e71cffe6a`. Rule list: `[]` (Enabled: `True`). {{(pid=80143) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 10:54:12.251780 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-b4786af3-d290-4f05-9c1f-955d00e18420 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 10:54:12.455974 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 814/1647] 10.222.0.25 () {58 vars in 958 bytes} [Sun Aug 7 10:54:12 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 258 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 07 10:54:12.518423 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8230c213-5926-4c81-aca1-7c439b1ac5d3 admin admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 10:54:12.519278 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4786af3-d290-4f05-9c1f-955d00e18420 None None] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 10:54:12.520467 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.fernet_utils [None req-8230c213-5926-4c81-aca1-7c439b1ac5d3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80144) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 10:54:12.521147 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-b4786af3-d290-4f05-9c1f-955d00e18420 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 10:54:12.538263 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4786af3-d290-4f05-9c1f-955d00e18420 None None] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 10:54:12.551557 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a605c709-bd37-4ae2-a98c-a669ba2c2cae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5ac47e3b09f43d19a23a0f457fe9e23', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 10:54:12.554235 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a605c709-bd37-4ae2-a98c-a669ba2c2cae admin admin] REQUEST_METHOD: `DELETE` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:54:12.554948 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a605c709-bd37-4ae2-a98c-a669ba2c2cae admin admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:54:12.556363 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a605c709-bd37-4ae2-a98c-a669ba2c2cae admin admin] PATH_INFO: `/v3/users/c9b20f3a7c394108a4b66fd2ade68cac` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:54:12.559853 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a605c709-bd37-4ae2-a98c-a669ba2c2cae admin admin] RBAC: Authorizing `identity:delete_user(user_id=c9b20f3a7c394108a4b66fd2ade68cac)` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 10:54:12.562728 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: WARNING py.warnings [None req-a605c709-bd37-4ae2-a98c-a669ba2c2cae admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 07 10:54:12.562728 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: warnings.warn(msg) Aug 07 10:54:12.562728 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: Aug 07 10:54:12.567464 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a605c709-bd37-4ae2-a98c-a669ba2c2cae admin admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 10:54:12.658708 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=95,keystone:DELETE=5 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 10:54:12.664291 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8230c213-5926-4c81-aca1-7c439b1ac5d3 admin admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 10:54:12.673664 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4144c11a-c8b6-453b-9f32-75cbf74133ce service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be1283c9bf47424c9aa3ad27cab03027', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6de1f3cac02c4bcdb05dd2337176033d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 10:54:12.675583 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4144c11a-c8b6-453b-9f32-75cbf74133ce service neutron] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:54:12.677371 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4144c11a-c8b6-453b-9f32-75cbf74133ce service neutron] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:54:12.680353 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4144c11a-c8b6-453b-9f32-75cbf74133ce service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:54:12.700533 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-a605c709-bd37-4ae2-a98c-a669ba2c2cae admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c9b20f3a7c394108a4b66fd2ade68cac'} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 10:54:12.702414 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4144c11a-c8b6-453b-9f32-75cbf74133ce service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 10:54:12.707424 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4144c11a-c8b6-453b-9f32-75cbf74133ce service neutron] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 10:54:12.727861 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-a605c709-bd37-4ae2-a98c-a669ba2c2cae admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c9b20f3a7c394108a4b66fd2ade68cac'} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 10:54:12.737664 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 834/1648] 10.222.0.25 () {60 vars in 1365 bytes} [Sun Aug 7 10:54:12 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 220 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 07 10:54:12.757040 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-a605c709-bd37-4ae2-a98c-a669ba2c2cae admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c9b20f3a7c394108a4b66fd2ade68cac'} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 10:54:12.758062 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-a605c709-bd37-4ae2-a98c-a669ba2c2cae admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c9b20f3a7c394108a4b66fd2ade68cac'} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 10:54:12.772564 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-a605c709-bd37-4ae2-a98c-a669ba2c2cae admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c9b20f3a7c394108a4b66fd2ade68cac'} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 10:54:12.791559 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 815/1649] 10.222.0.25 () {58 vars in 1224 bytes} [Sun Aug 7 10:54:12 2022] DELETE /identity/v3/users/c9b20f3a7c394108a4b66fd2ade68cac => generated 0 bytes in 274 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 07 10:54:12.795839 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e650496-1118-40b4-ae39-476a520eb4e6 None None] REQUEST_METHOD: `POST` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:54:12.796238 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e650496-1118-40b4-ae39-476a520eb4e6 None None] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:54:12.796605 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e650496-1118-40b4-ae39-476a520eb4e6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:54:12.821343 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.auth.core [None req-0e650496-1118-40b4-ae39-476a520eb4e6 None None] MFA Rules not processed for user `bac4e6b76fe64217ac8fa44e71cffe6a`. Rule list: `[]` (Enabled: `True`). {{(pid=80144) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 10:54:12.823042 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.fernet_utils [None req-0e650496-1118-40b4-ae39-476a520eb4e6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80144) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 10:54:12.915063 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 835/1650] 10.222.0.25 () {58 vars in 958 bytes} [Sun Aug 7 10:54:12 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 07 10:54:12.946871 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a605c709-bd37-4ae2-a98c-a669ba2c2cae admin admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 10:54:12.948559 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-a605c709-bd37-4ae2-a98c-a669ba2c2cae admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 10:54:13.046555 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a605c709-bd37-4ae2-a98c-a669ba2c2cae admin admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 10:54:13.051530 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afc58655-1433-43d1-8307-023fef6ea104 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be1283c9bf47424c9aa3ad27cab03027', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6de1f3cac02c4bcdb05dd2337176033d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 10:54:13.052226 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afc58655-1433-43d1-8307-023fef6ea104 service neutron] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:54:13.052516 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afc58655-1433-43d1-8307-023fef6ea104 service neutron] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:54:13.053166 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afc58655-1433-43d1-8307-023fef6ea104 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:54:13.067048 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afc58655-1433-43d1-8307-023fef6ea104 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 10:54:13.068523 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afc58655-1433-43d1-8307-023fef6ea104 service neutron] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 10:54:13.083985 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 816/1651] 10.222.0.25 () {60 vars in 1365 bytes} [Sun Aug 7 10:54:12 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 142 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 07 10:54:13.925847 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55d10267-51c1-4a76-a715-dbc2629b2d97 None None] REQUEST_METHOD: `POST` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:54:13.927236 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55d10267-51c1-4a76-a715-dbc2629b2d97 None None] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:54:13.927663 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55d10267-51c1-4a76-a715-dbc2629b2d97 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:54:13.987139 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.auth.core [None req-55d10267-51c1-4a76-a715-dbc2629b2d97 None None] MFA Rules not processed for user `c3a12c3144844d32812e010550bb0921`. Rule list: `[]` (Enabled: `True`). {{(pid=80144) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 10:54:13.988560 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.fernet_utils [None req-55d10267-51c1-4a76-a715-dbc2629b2d97 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80144) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 10:54:14.008672 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 10:54:14.159367 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 836/1652] 10.222.0.25 () {58 vars in 958 bytes} [Sun Aug 7 10:54:13 2022] POST /identity/v3/auth/tokens => generated 3270 bytes in 235 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 07 10:54:14.170045 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8401552-e5b9-4523-b14d-31d3e05ebb44 None None] REQUEST_METHOD: `POST` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:54:14.170911 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8401552-e5b9-4523-b14d-31d3e05ebb44 None None] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:54:14.171352 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8401552-e5b9-4523-b14d-31d3e05ebb44 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:54:14.195658 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.auth.core [None req-a8401552-e5b9-4523-b14d-31d3e05ebb44 None None] MFA Rules not processed for user `c3a12c3144844d32812e010550bb0921`. Rule list: `[]` (Enabled: `True`). {{(pid=80143) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 10:54:14.197696 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-a8401552-e5b9-4523-b14d-31d3e05ebb44 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 10:54:14.210904 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 817/1653] 10.222.0.25 () {58 vars in 958 bytes} [Sun Aug 7 10:54:14 2022] POST /identity/v3/auth/tokens => generated 3270 bytes in 42 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 07 10:54:14.220160 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55d10267-51c1-4a76-a715-dbc2629b2d97 None None] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 10:54:14.222393 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.fernet_utils [None req-55d10267-51c1-4a76-a715-dbc2629b2d97 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80144) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 10:54:14.347362 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8401552-e5b9-4523-b14d-31d3e05ebb44 None None] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 10:54:14.348893 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-a8401552-e5b9-4523-b14d-31d3e05ebb44 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 10:54:14.381281 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8401552-e5b9-4523-b14d-31d3e05ebb44 None None] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 10:54:14.395691 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f036781-0978-469c-a984-fc877ea2874f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5ac47e3b09f43d19a23a0f457fe9e23', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 10:54:14.396309 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f036781-0978-469c-a984-fc877ea2874f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:54:14.396595 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f036781-0978-469c-a984-fc877ea2874f admin admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:54:14.396875 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f036781-0978-469c-a984-fc877ea2874f admin admin] PATH_INFO: `/v3/projects/d16deb58b3724c8191e1e251a4e915f0` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:54:14.398693 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f036781-0978-469c-a984-fc877ea2874f admin admin] RBAC: Authorizing `identity:delete_project(project_id=d16deb58b3724c8191e1e251a4e915f0)` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 10:54:14.400786 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f036781-0978-469c-a984-fc877ea2874f admin admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 10:54:14.407972 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55d10267-51c1-4a76-a715-dbc2629b2d97 None None] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 10:54:14.420915 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f87f33a9-f162-4b11-b5d9-2c2b999156ce service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31d16d1b96104b4eb690398432b9829e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6de1f3cac02c4bcdb05dd2337176033d', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'reader', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 10:54:14.421944 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f87f33a9-f162-4b11-b5d9-2c2b999156ce service nova] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:54:14.421944 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f87f33a9-f162-4b11-b5d9-2c2b999156ce service nova] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:54:14.422754 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f87f33a9-f162-4b11-b5d9-2c2b999156ce service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:54:14.440284 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f87f33a9-f162-4b11-b5d9-2c2b999156ce service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 10:54:14.445558 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f87f33a9-f162-4b11-b5d9-2c2b999156ce service nova] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 10:54:14.466791 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 837/1654] 10.222.0.25 () {60 vars in 1365 bytes} [Sun Aug 7 10:54:14 2022] GET /identity/v3/auth/tokens => generated 3270 bytes in 247 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 07 10:54:14.509887 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-1f036781-0978-469c-a984-fc877ea2874f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd16deb58b3724c8191e1e251a4e915f0', 'request_id': 'req-1f036781-0978-469c-a984-fc877ea2874f'} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 10:54:14.520163 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-1f036781-0978-469c-a984-fc877ea2874f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd16deb58b3724c8191e1e251a4e915f0', 'request_id': 'req-1f036781-0978-469c-a984-fc877ea2874f'} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 10:54:14.541599 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-1f036781-0978-469c-a984-fc877ea2874f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd16deb58b3724c8191e1e251a4e915f0', 'request_id': 'req-1f036781-0978-469c-a984-fc877ea2874f'} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 10:54:14.569719 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-1f036781-0978-469c-a984-fc877ea2874f admin admin] The token cache is being invalidate because project d16deb58b3724c8191e1e251a4e915f0 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80143) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 07 10:54:14.570165 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-1f036781-0978-469c-a984-fc877ea2874f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 10:54:14.573074 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-1f036781-0978-469c-a984-fc877ea2874f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 10:54:14.580156 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 818/1655] 10.222.0.25 () {58 vars in 1233 bytes} [Sun Aug 7 10:54:14 2022] DELETE /identity/v3/projects/d16deb58b3724c8191e1e251a4e915f0 => generated 0 bytes in 231 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 07 10:54:14.581894 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a2011d9-e49c-4b25-a936-b04127f40e91 None None] REQUEST_METHOD: `POST` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:54:14.582450 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a2011d9-e49c-4b25-a936-b04127f40e91 None None] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:54:14.582740 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a2011d9-e49c-4b25-a936-b04127f40e91 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:54:14.633095 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.auth.core [None req-0a2011d9-e49c-4b25-a936-b04127f40e91 None None] MFA Rules not processed for user `bac4e6b76fe64217ac8fa44e71cffe6a`. Rule list: `[]` (Enabled: `True`). {{(pid=80144) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 10:54:14.636510 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.fernet_utils [None req-0a2011d9-e49c-4b25-a936-b04127f40e91 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80144) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 10:54:14.746366 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 838/1656] 10.222.0.25 () {58 vars in 958 bytes} [Sun Aug 7 10:54:14 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 07 10:54:14.752877 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2428c8ef-54e0-4496-9dcc-7348af6cb418 None None] REQUEST_METHOD: `POST` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:54:14.753057 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2428c8ef-54e0-4496-9dcc-7348af6cb418 None None] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:54:14.753332 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2428c8ef-54e0-4496-9dcc-7348af6cb418 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:54:14.786066 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.auth.core [None req-2428c8ef-54e0-4496-9dcc-7348af6cb418 None None] MFA Rules not processed for user `bac4e6b76fe64217ac8fa44e71cffe6a`. Rule list: `[]` (Enabled: `True`). {{(pid=80143) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 10:54:14.787466 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-2428c8ef-54e0-4496-9dcc-7348af6cb418 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 10:54:14.806014 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 819/1657] 10.222.0.25 () {58 vars in 958 bytes} [Sun Aug 7 10:54:14 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 54 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 07 10:54:14.824369 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a2011d9-e49c-4b25-a936-b04127f40e91 None None] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 10:54:14.849669 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a2011d9-e49c-4b25-a936-b04127f40e91 None None] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 10:54:14.851291 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2428c8ef-54e0-4496-9dcc-7348af6cb418 None None] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 10:54:14.852621 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-2428c8ef-54e0-4496-9dcc-7348af6cb418 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 10:54:14.870670 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-078c625d-c876-40f1-8146-232fcfecac39 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5ac47e3b09f43d19a23a0f457fe9e23', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 10:54:14.871331 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-078c625d-c876-40f1-8146-232fcfecac39 admin admin] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:54:14.871609 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-078c625d-c876-40f1-8146-232fcfecac39 admin admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:54:14.871879 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-078c625d-c876-40f1-8146-232fcfecac39 admin admin] PATH_INFO: `/v3/domains` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:54:14.872527 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-078c625d-c876-40f1-8146-232fcfecac39 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80144) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 10:54:14.872830 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-078c625d-c876-40f1-8146-232fcfecac39 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 10:54:14.874412 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-078c625d-c876-40f1-8146-232fcfecac39 admin admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 10:54:14.887932 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2428c8ef-54e0-4496-9dcc-7348af6cb418 None None] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 10:54:14.909060 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31f44deb-c324-417d-a35f-6dee3159243b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5ac47e3b09f43d19a23a0f457fe9e23', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 10:54:14.909730 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31f44deb-c324-417d-a35f-6dee3159243b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:54:14.910012 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31f44deb-c324-417d-a35f-6dee3159243b admin admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:54:14.910290 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31f44deb-c324-417d-a35f-6dee3159243b admin admin] PATH_INFO: `/v3/projects/8ac25ca8dc94488ba8404d09d27d034d` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:54:14.911941 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31f44deb-c324-417d-a35f-6dee3159243b admin admin] RBAC: Authorizing `identity:delete_project(project_id=8ac25ca8dc94488ba8404d09d27d034d)` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 10:54:14.913872 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31f44deb-c324-417d-a35f-6dee3159243b admin admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 10:54:14.915318 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 839/1658] 10.222.0.25 () {58 vars in 1153 bytes} [Sun Aug 7 10:54:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 100 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 10:54:14.916254 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-078c625d-c876-40f1-8146-232fcfecac39 admin admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 10:54:14.941341 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-078c625d-c876-40f1-8146-232fcfecac39 admin admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 10:54:14.958064 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=86,keystone:DELETE=12,keystone:INSERT=2 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 10:54:14.967380 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bccaa90d-b4a3-4a27-bd2c-fc18d3735e6d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5ac47e3b09f43d19a23a0f457fe9e23', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 10:54:14.970634 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bccaa90d-b4a3-4a27-bd2c-fc18d3735e6d admin admin] REQUEST_METHOD: `POST` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:54:14.972724 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bccaa90d-b4a3-4a27-bd2c-fc18d3735e6d admin admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:54:14.973170 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bccaa90d-b4a3-4a27-bd2c-fc18d3735e6d admin admin] PATH_INFO: `/v3/projects` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:54:14.975381 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bccaa90d-b4a3-4a27-bd2c-fc18d3735e6d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 10:54:14.982760 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bccaa90d-b4a3-4a27-bd2c-fc18d3735e6d admin admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 10:54:15.028185 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 10:54:15.044283 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 840/1659] 10.222.0.25 () {60 vars in 1153 bytes} [Sun Aug 7 10:54:14 2022] POST /identity/v3/projects => generated 363 bytes in 128 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 07 10:54:15.050223 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bccaa90d-b4a3-4a27-bd2c-fc18d3735e6d admin admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 10:54:15.064198 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-31f44deb-c324-417d-a35f-6dee3159243b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8ac25ca8dc94488ba8404d09d27d034d', 'request_id': 'req-31f44deb-c324-417d-a35f-6dee3159243b'} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 10:54:15.088310 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-31f44deb-c324-417d-a35f-6dee3159243b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8ac25ca8dc94488ba8404d09d27d034d', 'request_id': 'req-31f44deb-c324-417d-a35f-6dee3159243b'} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 10:54:15.108435 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-31f44deb-c324-417d-a35f-6dee3159243b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8ac25ca8dc94488ba8404d09d27d034d', 'request_id': 'req-31f44deb-c324-417d-a35f-6dee3159243b'} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 10:54:15.134063 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-31f44deb-c324-417d-a35f-6dee3159243b admin admin] The token cache is being invalidate because project 8ac25ca8dc94488ba8404d09d27d034d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80143) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 07 10:54:15.134588 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-31f44deb-c324-417d-a35f-6dee3159243b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 10:54:15.136852 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-31f44deb-c324-417d-a35f-6dee3159243b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 10:54:15.139228 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 820/1660] 10.222.0.25 () {58 vars in 1233 bytes} [Sun Aug 7 10:54:14 2022] DELETE /identity/v3/projects/8ac25ca8dc94488ba8404d09d27d034d => generated 0 bytes in 288 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 07 10:54:15.144007 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31f44deb-c324-417d-a35f-6dee3159243b admin admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 10:54:15.148528 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-31f44deb-c324-417d-a35f-6dee3159243b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 10:54:15.179914 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bccaa90d-b4a3-4a27-bd2c-fc18d3735e6d admin admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 10:54:15.186330 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64949247-c987-465f-9248-1dd4cdacc6d8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5ac47e3b09f43d19a23a0f457fe9e23', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 10:54:15.187159 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64949247-c987-465f-9248-1dd4cdacc6d8 admin admin] REQUEST_METHOD: `POST` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:54:15.187548 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64949247-c987-465f-9248-1dd4cdacc6d8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:54:15.188069 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64949247-c987-465f-9248-1dd4cdacc6d8 admin admin] PATH_INFO: `/v3/users` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:54:15.188820 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64949247-c987-465f-9248-1dd4cdacc6d8 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 10:54:15.191148 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64949247-c987-465f-9248-1dd4cdacc6d8 admin admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 10:54:15.223622 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 841/1661] 10.222.0.25 () {60 vars in 1144 bytes} [Sun Aug 7 10:54:15 2022] POST /identity/v3/users => generated 282 bytes in 175 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 07 10:54:15.230371 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64949247-c987-465f-9248-1dd4cdacc6d8 admin admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 10:54:15.233907 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.fernet_utils [None req-64949247-c987-465f-9248-1dd4cdacc6d8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80144) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 10:54:15.261898 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64949247-c987-465f-9248-1dd4cdacc6d8 admin admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 10:54:15.273138 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e17ff0a7-177e-496a-805a-b7124762298a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5ac47e3b09f43d19a23a0f457fe9e23', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 10:54:15.274033 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e17ff0a7-177e-496a-805a-b7124762298a admin admin] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:54:15.274511 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e17ff0a7-177e-496a-805a-b7124762298a admin admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:54:15.274902 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e17ff0a7-177e-496a-805a-b7124762298a admin admin] PATH_INFO: `/v3/roles` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:54:15.275701 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e17ff0a7-177e-496a-805a-b7124762298a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 10:54:15.277690 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e17ff0a7-177e-496a-805a-b7124762298a admin admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 10:54:15.301819 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 842/1662] 10.222.0.25 () {58 vars in 1122 bytes} [Sun Aug 7 10:54:15 2022] GET /identity/v3/roles => generated 1420 bytes in 71 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 07 10:54:15.305584 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e17ff0a7-177e-496a-805a-b7124762298a admin admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 10:54:15.307137 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31f44deb-c324-417d-a35f-6dee3159243b admin admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 10:54:15.324059 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a3c7dc2-356d-4074-a93f-b04163c6f8ca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5ac47e3b09f43d19a23a0f457fe9e23', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 10:54:15.324990 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a3c7dc2-356d-4074-a93f-b04163c6f8ca admin admin] REQUEST_METHOD: `DELETE` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:54:15.325397 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a3c7dc2-356d-4074-a93f-b04163c6f8ca admin admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:54:15.325820 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a3c7dc2-356d-4074-a93f-b04163c6f8ca admin admin] PATH_INFO: `/v3/users/54b0402fadff4656aad6761d9c58f457` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:54:15.327212 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a3c7dc2-356d-4074-a93f-b04163c6f8ca admin admin] RBAC: Authorizing `identity:delete_user(user_id=54b0402fadff4656aad6761d9c58f457)` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 10:54:15.329127 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a3c7dc2-356d-4074-a93f-b04163c6f8ca admin admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 10:54:15.329600 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e17ff0a7-177e-496a-805a-b7124762298a admin admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 10:54:15.339350 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ff30ec1-6684-4c6c-b998-afe83c789922 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5ac47e3b09f43d19a23a0f457fe9e23', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 10:54:15.340091 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ff30ec1-6684-4c6c-b998-afe83c789922 admin admin] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:54:15.340483 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ff30ec1-6684-4c6c-b998-afe83c789922 admin admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:54:15.341186 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ff30ec1-6684-4c6c-b998-afe83c789922 admin admin] PATH_INFO: `/v3/roles` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:54:15.343917 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ff30ec1-6684-4c6c-b998-afe83c789922 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 10:54:15.348712 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ff30ec1-6684-4c6c-b998-afe83c789922 admin admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 10:54:15.372880 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 843/1663] 10.222.0.25 () {58 vars in 1122 bytes} [Sun Aug 7 10:54:15 2022] GET /identity/v3/roles => generated 1420 bytes in 66 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 07 10:54:15.373470 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57394e7d-3469-4bd4-8118-de1e9b81cd7c None None] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:54:15.373871 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57394e7d-3469-4bd4-8118-de1e9b81cd7c None None] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:54:15.374706 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57394e7d-3469-4bd4-8118-de1e9b81cd7c None None] PATH_INFO: `/` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:54:15.376496 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 844/1664] 10.222.0.25 () {54 vars in 912 bytes} [Sun Aug 7 10:54:15 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 07 10:54:15.377769 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57394e7d-3469-4bd4-8118-de1e9b81cd7c None None] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 10:54:15.395760 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57394e7d-3469-4bd4-8118-de1e9b81cd7c None None] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 10:54:15.401674 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d53a05e5-ff23-44cc-9aa5-bf7046b795f5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5ac47e3b09f43d19a23a0f457fe9e23', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 10:54:15.402991 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d53a05e5-ff23-44cc-9aa5-bf7046b795f5 admin admin] REQUEST_METHOD: `PUT` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:54:15.403554 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d53a05e5-ff23-44cc-9aa5-bf7046b795f5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:54:15.403991 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d53a05e5-ff23-44cc-9aa5-bf7046b795f5 admin admin] PATH_INFO: `/v3/projects/599c100ce4e7439bb77596d7f4cbdd35/users/76b81660953e460fae209814b208eff7/roles/b98dfb3b9fee4644af2887a3ac3fd2e6` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:54:15.445086 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d53a05e5-ff23-44cc-9aa5-bf7046b795f5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=599c100ce4e7439bb77596d7f4cbdd35, user_id=76b81660953e460fae209814b208eff7, role_id=b98dfb3b9fee4644af2887a3ac3fd2e6)` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 10:54:15.448972 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d53a05e5-ff23-44cc-9aa5-bf7046b795f5 admin admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 10:54:15.449604 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-5a3c7dc2-356d-4074-a93f-b04163c6f8ca admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '54b0402fadff4656aad6761d9c58f457'} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 10:54:15.463018 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-5a3c7dc2-356d-4074-a93f-b04163c6f8ca admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '54b0402fadff4656aad6761d9c58f457'} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 10:54:15.479480 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-5a3c7dc2-356d-4074-a93f-b04163c6f8ca admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '54b0402fadff4656aad6761d9c58f457'} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 10:54:15.484449 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-5a3c7dc2-356d-4074-a93f-b04163c6f8ca admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '54b0402fadff4656aad6761d9c58f457'} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 10:54:15.486033 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 845/1665] 10.222.0.25 () {60 vars in 1483 bytes} [Sun Aug 7 10:54:15 2022] PUT /identity/v3/projects/599c100ce4e7439bb77596d7f4cbdd35/users/76b81660953e460fae209814b208eff7/roles/b98dfb3b9fee4644af2887a3ac3fd2e6 => generated 0 bytes in 98 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 07 10:54:15.488534 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5b112ac-e54a-46c7-b80a-f4c372ba35b8 None None] REQUEST_METHOD: `POST` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:54:15.488822 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5b112ac-e54a-46c7-b80a-f4c372ba35b8 None None] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:54:15.489097 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5b112ac-e54a-46c7-b80a-f4c372ba35b8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:54:15.498500 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-5a3c7dc2-356d-4074-a93f-b04163c6f8ca admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '54b0402fadff4656aad6761d9c58f457'} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 10:54:15.519188 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 821/1666] 10.222.0.25 () {58 vars in 1224 bytes} [Sun Aug 7 10:54:15 2022] DELETE /identity/v3/users/54b0402fadff4656aad6761d9c58f457 => generated 0 bytes in 368 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 07 10:54:15.531923 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e43088db-968b-4bbe-8ced-61675053e5ce None None] REQUEST_METHOD: `POST` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:54:15.532690 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e43088db-968b-4bbe-8ced-61675053e5ce None None] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:54:15.533147 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e43088db-968b-4bbe-8ced-61675053e5ce None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:54:15.541611 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.auth.core [None req-e5b112ac-e54a-46c7-b80a-f4c372ba35b8 None None] MFA Rules not processed for user `be1283c9bf47424c9aa3ad27cab03027`. Rule list: `[]` (Enabled: `True`). {{(pid=80144) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 10:54:15.542843 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.fernet_utils [None req-e5b112ac-e54a-46c7-b80a-f4c372ba35b8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80144) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 10:54:15.588710 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.auth.core [None req-e43088db-968b-4bbe-8ced-61675053e5ce None None] MFA Rules not processed for user `76b81660953e460fae209814b208eff7`. Rule list: `[]` (Enabled: `True`). {{(pid=80143) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 10:54:15.590982 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-e43088db-968b-4bbe-8ced-61675053e5ce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 10:54:15.605178 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 10:54:15.722526 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 846/1667] 10.222.0.25 () {58 vars in 1009 bytes} [Sun Aug 7 10:54:15 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 236 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 07 10:54:15.743406 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5b112ac-e54a-46c7-b80a-f4c372ba35b8 None None] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 10:54:15.745233 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.fernet_utils [None req-e5b112ac-e54a-46c7-b80a-f4c372ba35b8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80144) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 10:54:15.761240 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5b112ac-e54a-46c7-b80a-f4c372ba35b8 None None] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 10:54:15.773377 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47339e17-e0a2-48fb-9de1-ba44a89963ee service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be1283c9bf47424c9aa3ad27cab03027', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6de1f3cac02c4bcdb05dd2337176033d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 10:54:15.776905 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47339e17-e0a2-48fb-9de1-ba44a89963ee service neutron] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:54:15.777512 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47339e17-e0a2-48fb-9de1-ba44a89963ee service neutron] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:54:15.778119 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47339e17-e0a2-48fb-9de1-ba44a89963ee service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:54:15.810885 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47339e17-e0a2-48fb-9de1-ba44a89963ee service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 10:54:15.812928 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47339e17-e0a2-48fb-9de1-ba44a89963ee service neutron] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 10:54:15.824564 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 847/1668] 10.222.0.25 () {60 vars in 1365 bytes} [Sun Aug 7 10:54:15 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 82 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 07 10:54:15.837637 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 822/1669] 10.222.0.25 () {58 vars in 958 bytes} [Sun Aug 7 10:54:15 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 318 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 07 10:54:15.838997 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47339e17-e0a2-48fb-9de1-ba44a89963ee service neutron] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 10:54:15.840897 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.fernet_utils [None req-47339e17-e0a2-48fb-9de1-ba44a89963ee service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80144) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 10:54:15.894462 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 10:54:15.997831 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47339e17-e0a2-48fb-9de1-ba44a89963ee service neutron] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 10:54:16.004379 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4353aa1-ad06-4df8-b689-0d5fabe579da admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5ac47e3b09f43d19a23a0f457fe9e23', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 10:54:16.005163 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4353aa1-ad06-4df8-b689-0d5fabe579da admin admin] REQUEST_METHOD: `POST` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:54:16.005578 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4353aa1-ad06-4df8-b689-0d5fabe579da admin admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:54:16.005970 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4353aa1-ad06-4df8-b689-0d5fabe579da admin admin] PATH_INFO: `/v3/projects` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:54:16.008084 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4353aa1-ad06-4df8-b689-0d5fabe579da admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 10:54:16.016934 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4353aa1-ad06-4df8-b689-0d5fabe579da admin admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 10:54:16.129130 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 848/1670] 10.222.0.25 () {60 vars in 1153 bytes} [Sun Aug 7 10:54:15 2022] POST /identity/v3/projects => generated 365 bytes in 290 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 07 10:54:16.133179 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e43088db-968b-4bbe-8ced-61675053e5ce None None] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 10:54:16.156399 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e43088db-968b-4bbe-8ced-61675053e5ce None None] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 10:54:16.354199 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-918bf4b6-1a2b-4c70-a316-9ec57dc11969 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5ac47e3b09f43d19a23a0f457fe9e23', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 10:54:16.354874 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-918bf4b6-1a2b-4c70-a316-9ec57dc11969 admin admin] REQUEST_METHOD: `POST` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:54:16.362045 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-918bf4b6-1a2b-4c70-a316-9ec57dc11969 admin admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:54:16.362447 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-918bf4b6-1a2b-4c70-a316-9ec57dc11969 admin admin] PATH_INFO: `/v3/users` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:54:16.372359 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-918bf4b6-1a2b-4c70-a316-9ec57dc11969 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 10:54:16.376423 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-918bf4b6-1a2b-4c70-a316-9ec57dc11969 admin admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 10:54:16.438336 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 823/1671] 10.222.0.25 () {60 vars in 1144 bytes} [Sun Aug 7 10:54:16 2022] POST /identity/v3/users => generated 289 bytes in 308 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 07 10:54:16.441711 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4353aa1-ad06-4df8-b689-0d5fabe579da admin admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 10:54:16.457587 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4353aa1-ad06-4df8-b689-0d5fabe579da admin admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 10:54:16.463457 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54bc8b19-a18c-4d8f-984b-7668b5cea436 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5ac47e3b09f43d19a23a0f457fe9e23', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 10:54:16.464506 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54bc8b19-a18c-4d8f-984b-7668b5cea436 admin admin] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:54:16.464987 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54bc8b19-a18c-4d8f-984b-7668b5cea436 admin admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:54:16.465466 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54bc8b19-a18c-4d8f-984b-7668b5cea436 admin admin] PATH_INFO: `/v3/roles` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:54:16.466636 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54bc8b19-a18c-4d8f-984b-7668b5cea436 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 10:54:16.468477 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54bc8b19-a18c-4d8f-984b-7668b5cea436 admin admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 10:54:16.510408 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 849/1672] 10.222.0.25 () {58 vars in 1122 bytes} [Sun Aug 7 10:54:16 2022] GET /identity/v3/roles => generated 1420 bytes in 70 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 07 10:54:16.514555 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-918bf4b6-1a2b-4c70-a316-9ec57dc11969 admin admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 10:54:16.529356 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-918bf4b6-1a2b-4c70-a316-9ec57dc11969 admin admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 10:54:16.535934 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2447f830-7941-4f53-b89d-bc65781f4cc9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5ac47e3b09f43d19a23a0f457fe9e23', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 10:54:16.536771 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2447f830-7941-4f53-b89d-bc65781f4cc9 admin admin] REQUEST_METHOD: `PUT` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:54:16.537156 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2447f830-7941-4f53-b89d-bc65781f4cc9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:54:16.537538 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2447f830-7941-4f53-b89d-bc65781f4cc9 admin admin] PATH_INFO: `/v3/projects/f392209c49534024a25a0a839234bba0/users/d98015c8dd7f46cdb9f4155f15de5018/roles/81b1f1bcee2a473088fbbd327c1c5b23` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:54:16.563117 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2447f830-7941-4f53-b89d-bc65781f4cc9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f392209c49534024a25a0a839234bba0, user_id=d98015c8dd7f46cdb9f4155f15de5018, role_id=81b1f1bcee2a473088fbbd327c1c5b23)` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 10:54:16.566972 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2447f830-7941-4f53-b89d-bc65781f4cc9 admin admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 10:54:16.592224 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 824/1673] 10.222.0.25 () {60 vars in 1483 bytes} [Sun Aug 7 10:54:16 2022] PUT /identity/v3/projects/f392209c49534024a25a0a839234bba0/users/d98015c8dd7f46cdb9f4155f15de5018/roles/81b1f1bcee2a473088fbbd327c1c5b23 => generated 0 bytes in 78 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 07 10:54:16.597348 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54bc8b19-a18c-4d8f-984b-7668b5cea436 admin admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 10:54:16.619985 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54bc8b19-a18c-4d8f-984b-7668b5cea436 admin admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 10:54:16.703816 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed5e079b-51cb-4961-b26f-9e4902f5f049 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5ac47e3b09f43d19a23a0f457fe9e23', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 10:54:16.706324 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed5e079b-51cb-4961-b26f-9e4902f5f049 admin admin] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:54:16.706695 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed5e079b-51cb-4961-b26f-9e4902f5f049 admin admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:54:16.707070 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed5e079b-51cb-4961-b26f-9e4902f5f049 admin admin] PATH_INFO: `/v3/roles` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:54:16.713611 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed5e079b-51cb-4961-b26f-9e4902f5f049 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 10:54:16.718288 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed5e079b-51cb-4961-b26f-9e4902f5f049 admin admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 10:54:16.742318 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 850/1674] 10.222.0.25 () {58 vars in 1122 bytes} [Sun Aug 7 10:54:16 2022] GET /identity/v3/roles => generated 1420 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 07 10:54:16.745621 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2447f830-7941-4f53-b89d-bc65781f4cc9 admin admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 10:54:16.758779 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2447f830-7941-4f53-b89d-bc65781f4cc9 admin admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 10:54:16.773982 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8c4f520-c465-4816-8e5c-de13aa78a2aa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5ac47e3b09f43d19a23a0f457fe9e23', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 10:54:16.775870 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8c4f520-c465-4816-8e5c-de13aa78a2aa admin admin] REQUEST_METHOD: `PUT` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:54:16.775870 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8c4f520-c465-4816-8e5c-de13aa78a2aa admin admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:54:16.775870 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8c4f520-c465-4816-8e5c-de13aa78a2aa admin admin] PATH_INFO: `/v3/projects/f392209c49534024a25a0a839234bba0/users/d98015c8dd7f46cdb9f4155f15de5018/roles/81b1f1bcee2a473088fbbd327c1c5b23` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:54:16.777791 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8c4f520-c465-4816-8e5c-de13aa78a2aa admin admin] RBAC: Authorizing `identity:create_grant(project_id=f392209c49534024a25a0a839234bba0, user_id=d98015c8dd7f46cdb9f4155f15de5018, role_id=81b1f1bcee2a473088fbbd327c1c5b23)` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 10:54:16.780473 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8c4f520-c465-4816-8e5c-de13aa78a2aa admin admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 10:54:16.795454 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 825/1675] 10.222.0.25 () {60 vars in 1483 bytes} [Sun Aug 7 10:54:16 2022] PUT /identity/v3/projects/f392209c49534024a25a0a839234bba0/users/d98015c8dd7f46cdb9f4155f15de5018/roles/81b1f1bcee2a473088fbbd327c1c5b23 => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 07 10:54:16.809028 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7d84e7a-ed61-46a4-8dbf-8f8c64c7869f None None] REQUEST_METHOD: `POST` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:54:16.809545 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7d84e7a-ed61-46a4-8dbf-8f8c64c7869f None None] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:54:16.809949 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7d84e7a-ed61-46a4-8dbf-8f8c64c7869f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:54:16.919235 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8c4f520-c465-4816-8e5c-de13aa78a2aa admin admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 10:54:16.921148 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-f8c4f520-c465-4816-8e5c-de13aa78a2aa admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 10:54:16.948106 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.auth.core [None req-c7d84e7a-ed61-46a4-8dbf-8f8c64c7869f None None] MFA Rules not processed for user `d98015c8dd7f46cdb9f4155f15de5018`. Rule list: `[]` (Enabled: `True`). {{(pid=80144) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 10:54:16.960217 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.fernet_utils [None req-c7d84e7a-ed61-46a4-8dbf-8f8c64c7869f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80144) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 10:54:17.155762 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8c4f520-c465-4816-8e5c-de13aa78a2aa admin admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 10:54:17.163448 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0bc5e44-c513-45e7-9d9e-c05c61164be6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5ac47e3b09f43d19a23a0f457fe9e23', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 10:54:17.163778 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0bc5e44-c513-45e7-9d9e-c05c61164be6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:54:17.164060 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0bc5e44-c513-45e7-9d9e-c05c61164be6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:54:17.164327 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0bc5e44-c513-45e7-9d9e-c05c61164be6 admin admin] PATH_INFO: `/v3/projects/7819c84c6c844cbfb9b334ed8026ccb3` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:54:17.165485 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0bc5e44-c513-45e7-9d9e-c05c61164be6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=7819c84c6c844cbfb9b334ed8026ccb3)` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 10:54:17.167952 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0bc5e44-c513-45e7-9d9e-c05c61164be6 admin admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 10:54:17.171237 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 10:54:17.260248 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 851/1676] 10.222.0.25 () {58 vars in 958 bytes} [Sun Aug 7 10:54:16 2022] POST /identity/v3/auth/tokens => generated 3309 bytes in 452 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 07 10:54:17.273699 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7d84e7a-ed61-46a4-8dbf-8f8c64c7869f None None] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 10:54:17.275328 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.fernet_utils [None req-c7d84e7a-ed61-46a4-8dbf-8f8c64c7869f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80144) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 10:54:17.335311 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 10:54:17.355783 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-d0bc5e44-c513-45e7-9d9e-c05c61164be6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7819c84c6c844cbfb9b334ed8026ccb3', 'request_id': 'req-d0bc5e44-c513-45e7-9d9e-c05c61164be6'} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 10:54:17.373114 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-d0bc5e44-c513-45e7-9d9e-c05c61164be6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7819c84c6c844cbfb9b334ed8026ccb3', 'request_id': 'req-d0bc5e44-c513-45e7-9d9e-c05c61164be6'} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 10:54:17.389150 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-d0bc5e44-c513-45e7-9d9e-c05c61164be6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7819c84c6c844cbfb9b334ed8026ccb3', 'request_id': 'req-d0bc5e44-c513-45e7-9d9e-c05c61164be6'} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 10:54:17.433373 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-d0bc5e44-c513-45e7-9d9e-c05c61164be6 admin admin] The token cache is being invalidate because project 7819c84c6c844cbfb9b334ed8026ccb3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80143) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 07 10:54:17.433677 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-d0bc5e44-c513-45e7-9d9e-c05c61164be6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 10:54:17.441456 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-d0bc5e44-c513-45e7-9d9e-c05c61164be6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 10:54:17.443727 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7d84e7a-ed61-46a4-8dbf-8f8c64c7869f None None] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 10:54:17.447062 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 826/1677] 10.222.0.25 () {58 vars in 1233 bytes} [Sun Aug 7 10:54:16 2022] DELETE /identity/v3/projects/7819c84c6c844cbfb9b334ed8026ccb3 => generated 0 bytes in 529 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 07 10:54:17.452138 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f1221d7-3594-4b78-a2e8-12b3e7c7118f None None] REQUEST_METHOD: `POST` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:54:17.452718 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f1221d7-3594-4b78-a2e8-12b3e7c7118f None None] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:54:17.453290 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f1221d7-3594-4b78-a2e8-12b3e7c7118f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:54:17.505500 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.auth.core [None req-2f1221d7-3594-4b78-a2e8-12b3e7c7118f None None] MFA Rules not processed for user `bac4e6b76fe64217ac8fa44e71cffe6a`. Rule list: `[]` (Enabled: `True`). {{(pid=80143) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 10:54:17.510489 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-2f1221d7-3594-4b78-a2e8-12b3e7c7118f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 10:54:17.599986 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b86d225d-92e7-4233-aee6-db7ffc6d568b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88dbd6dfd1164994a3e16da553561910', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6de1f3cac02c4bcdb05dd2337176033d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 10:54:17.600738 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b86d225d-92e7-4233-aee6-db7ffc6d568b service cinder] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:54:17.601228 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b86d225d-92e7-4233-aee6-db7ffc6d568b service cinder] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:54:17.601564 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b86d225d-92e7-4233-aee6-db7ffc6d568b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:54:17.606311 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.fernet_utils [None req-b86d225d-92e7-4233-aee6-db7ffc6d568b service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80144) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 10:54:17.715830 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 827/1678] 10.222.0.25 () {58 vars in 958 bytes} [Sun Aug 7 10:54:17 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 265 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 07 10:54:17.721023 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d959f394-3838-4242-9481-8aa88491c70b None None] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:54:17.721818 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d959f394-3838-4242-9481-8aa88491c70b None None] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:54:17.723157 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d959f394-3838-4242-9481-8aa88491c70b None None] PATH_INFO: `/` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:54:17.728038 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 828/1679] 10.222.0.25 () {54 vars in 961 bytes} [Sun Aug 7 10:54:17 2022] GET /identity => generated 270 bytes in 11 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 07 10:54:17.735033 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f043b9c-d0ff-42a8-bbf2-5b298fd2215c None None] REQUEST_METHOD: `POST` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:54:17.735673 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f043b9c-d0ff-42a8-bbf2-5b298fd2215c None None] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:54:17.736678 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f043b9c-d0ff-42a8-bbf2-5b298fd2215c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:54:17.819704 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.auth.core [None req-8f043b9c-d0ff-42a8-bbf2-5b298fd2215c None None] MFA Rules not processed for user `bac4e6b76fe64217ac8fa44e71cffe6a`. Rule list: `[]` (Enabled: `True`). {{(pid=80143) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 10:54:17.821116 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-8f043b9c-d0ff-42a8-bbf2-5b298fd2215c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 10:54:17.838623 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 829/1680] 10.222.0.25 () {58 vars in 958 bytes} [Sun Aug 7 10:54:17 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 07 10:54:17.839987 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70c640d8-b277-44e8-b841-a2c7fe1cd0ae None None] REQUEST_METHOD: `POST` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:54:17.840453 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70c640d8-b277-44e8-b841-a2c7fe1cd0ae None None] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:54:17.840742 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70c640d8-b277-44e8-b841-a2c7fe1cd0ae None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:54:17.855253 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b86d225d-92e7-4233-aee6-db7ffc6d568b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 10:54:17.856652 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b86d225d-92e7-4233-aee6-db7ffc6d568b service cinder] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 10:54:17.872263 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 852/1681] 10.222.0.25 () {60 vars in 1365 bytes} [Sun Aug 7 10:54:17 2022] GET /identity/v3/auth/tokens => generated 3309 bytes in 599 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 07 10:54:17.872723 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.auth.core [None req-70c640d8-b277-44e8-b841-a2c7fe1cd0ae None None] MFA Rules not processed for user `6ee2e4c08af04c219ed395d7ca7085a6`. Rule list: `[]` (Enabled: `True`). {{(pid=80143) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 10:54:17.873617 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80ff6c51-e1a7-483b-a20a-07e8697c8bde None None] REQUEST_METHOD: `POST` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:54:17.873912 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80ff6c51-e1a7-483b-a20a-07e8697c8bde None None] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:54:17.874020 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-70c640d8-b277-44e8-b841-a2c7fe1cd0ae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 10:54:17.874194 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80ff6c51-e1a7-483b-a20a-07e8697c8bde None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:54:17.947974 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.auth.core [None req-80ff6c51-e1a7-483b-a20a-07e8697c8bde None None] MFA Rules not processed for user `c3a12c3144844d32812e010550bb0921`. Rule list: `[]` (Enabled: `True`). {{(pid=80144) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 10:54:17.951506 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.fernet_utils [None req-80ff6c51-e1a7-483b-a20a-07e8697c8bde None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80144) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 10:54:18.038017 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 830/1682] 10.222.0.25 () {58 vars in 1058 bytes} [Sun Aug 7 10:54:17 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 199 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 07 10:54:18.039838 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70c640d8-b277-44e8-b841-a2c7fe1cd0ae None None] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 10:54:18.072452 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70c640d8-b277-44e8-b841-a2c7fe1cd0ae None None] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 10:54:18.077950 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f2b18d0-ef8b-4cdb-9d8d-920304a8c513 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5ac47e3b09f43d19a23a0f457fe9e23', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 10:54:18.078815 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f2b18d0-ef8b-4cdb-9d8d-920304a8c513 admin admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:54:18.079534 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f2b18d0-ef8b-4cdb-9d8d-920304a8c513 admin admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:54:18.080522 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f2b18d0-ef8b-4cdb-9d8d-920304a8c513 admin admin] PATH_INFO: `/v3/domains` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:54:18.081393 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f2b18d0-ef8b-4cdb-9d8d-920304a8c513 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80143) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 10:54:18.081856 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f2b18d0-ef8b-4cdb-9d8d-920304a8c513 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 10:54:18.083996 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f2b18d0-ef8b-4cdb-9d8d-920304a8c513 admin admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 10:54:18.096874 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 853/1683] 10.222.0.25 () {58 vars in 958 bytes} [Sun Aug 7 10:54:17 2022] POST /identity/v3/auth/tokens => generated 3270 bytes in 220 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 07 10:54:18.096874 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80ff6c51-e1a7-483b-a20a-07e8697c8bde None None] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 10:54:18.105267 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 831/1684] 10.222.0.25 () {58 vars in 1153 bytes} [Sun Aug 7 10:54:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 67 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 10:54:18.110395 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f2b18d0-ef8b-4cdb-9d8d-920304a8c513 admin admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 10:54:18.135252 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f2b18d0-ef8b-4cdb-9d8d-920304a8c513 admin admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 10:54:18.145161 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80ff6c51-e1a7-483b-a20a-07e8697c8bde None None] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 10:54:18.153381 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3605570e-7f8c-40c5-ba10-0fecc33585c9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5ac47e3b09f43d19a23a0f457fe9e23', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 10:54:18.154937 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3605570e-7f8c-40c5-ba10-0fecc33585c9 admin admin] REQUEST_METHOD: `POST` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:54:18.157826 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3605570e-7f8c-40c5-ba10-0fecc33585c9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:54:18.160599 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3605570e-7f8c-40c5-ba10-0fecc33585c9 admin admin] PATH_INFO: `/v3/projects` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:54:18.163806 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e450ee0a-912b-4fd0-889d-39da9b7fc37c service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6ee2e4c08af04c219ed395d7ca7085a6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6de1f3cac02c4bcdb05dd2337176033d', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 10:54:18.164444 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e450ee0a-912b-4fd0-889d-39da9b7fc37c service placement] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:54:18.165281 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e450ee0a-912b-4fd0-889d-39da9b7fc37c service placement] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:54:18.165281 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e450ee0a-912b-4fd0-889d-39da9b7fc37c service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:54:18.166790 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.fernet_utils [None req-e450ee0a-912b-4fd0-889d-39da9b7fc37c service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80144) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 10:54:18.175898 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3605570e-7f8c-40c5-ba10-0fecc33585c9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 10:54:18.180999 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3605570e-7f8c-40c5-ba10-0fecc33585c9 admin admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 10:54:18.190874 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e450ee0a-912b-4fd0-889d-39da9b7fc37c service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 10:54:18.194047 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e450ee0a-912b-4fd0-889d-39da9b7fc37c service placement] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 10:54:18.211548 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 10:54:18.273122 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 832/1685] 10.222.0.25 () {60 vars in 1153 bytes} [Sun Aug 7 10:54:18 2022] POST /identity/v3/projects => generated 367 bytes in 163 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 07 10:54:18.277659 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 854/1686] 10.222.0.25 () {60 vars in 1365 bytes} [Sun Aug 7 10:54:18 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 184 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 07 10:54:18.279318 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3605570e-7f8c-40c5-ba10-0fecc33585c9 admin admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 10:54:18.547538 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3605570e-7f8c-40c5-ba10-0fecc33585c9 admin admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 10:54:18.565725 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90d934d4-8a9d-46a6-80ac-af4702fae0d9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5ac47e3b09f43d19a23a0f457fe9e23', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 10:54:18.567665 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90d934d4-8a9d-46a6-80ac-af4702fae0d9 admin admin] REQUEST_METHOD: `POST` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:54:18.568027 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90d934d4-8a9d-46a6-80ac-af4702fae0d9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:54:18.568409 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90d934d4-8a9d-46a6-80ac-af4702fae0d9 admin admin] PATH_INFO: `/v3/users` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:54:18.581630 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90d934d4-8a9d-46a6-80ac-af4702fae0d9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 10:54:18.590349 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90d934d4-8a9d-46a6-80ac-af4702fae0d9 admin admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 10:54:18.708676 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 833/1687] 10.222.0.25 () {60 vars in 1144 bytes} [Sun Aug 7 10:54:18 2022] POST /identity/v3/users => generated 284 bytes in 429 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 07 10:54:18.710044 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e450ee0a-912b-4fd0-889d-39da9b7fc37c service placement] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 10:54:18.745428 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e450ee0a-912b-4fd0-889d-39da9b7fc37c service placement] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 10:54:18.797826 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37806a67-0a92-4057-85fd-72d582d0e44b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5ac47e3b09f43d19a23a0f457fe9e23', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 10:54:18.799258 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37806a67-0a92-4057-85fd-72d582d0e44b admin admin] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:54:18.800799 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37806a67-0a92-4057-85fd-72d582d0e44b admin admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:54:18.801057 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37806a67-0a92-4057-85fd-72d582d0e44b admin admin] PATH_INFO: `/v3/roles` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:54:18.803629 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37806a67-0a92-4057-85fd-72d582d0e44b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 10:54:18.812429 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37806a67-0a92-4057-85fd-72d582d0e44b admin admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 10:54:18.854300 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 855/1688] 10.222.0.25 () {58 vars in 1122 bytes} [Sun Aug 7 10:54:18 2022] GET /identity/v3/roles => generated 1420 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 07 10:54:18.855632 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90d934d4-8a9d-46a6-80ac-af4702fae0d9 admin admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 10:54:18.887101 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90d934d4-8a9d-46a6-80ac-af4702fae0d9 admin admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 10:54:18.899623 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64c743aa-d605-4dfb-9e36-618fc4cef574 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5ac47e3b09f43d19a23a0f457fe9e23', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 10:54:18.900448 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64c743aa-d605-4dfb-9e36-618fc4cef574 admin admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:54:18.900736 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64c743aa-d605-4dfb-9e36-618fc4cef574 admin admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:54:18.901069 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64c743aa-d605-4dfb-9e36-618fc4cef574 admin admin] PATH_INFO: `/v3/roles` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:54:18.901679 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64c743aa-d605-4dfb-9e36-618fc4cef574 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 10:54:18.903332 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64c743aa-d605-4dfb-9e36-618fc4cef574 admin admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 10:54:18.919633 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 834/1689] 10.222.0.25 () {58 vars in 1122 bytes} [Sun Aug 7 10:54:18 2022] GET /identity/v3/roles => generated 1420 bytes in 64 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 07 10:54:18.920444 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37806a67-0a92-4057-85fd-72d582d0e44b admin admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 10:54:18.963252 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37806a67-0a92-4057-85fd-72d582d0e44b admin admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 10:54:18.976902 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15f1e822-5df2-4dda-b6ba-6418308144d9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5ac47e3b09f43d19a23a0f457fe9e23', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 10:54:18.977916 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15f1e822-5df2-4dda-b6ba-6418308144d9 admin admin] REQUEST_METHOD: `PUT` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:54:18.979273 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15f1e822-5df2-4dda-b6ba-6418308144d9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:54:18.979931 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15f1e822-5df2-4dda-b6ba-6418308144d9 admin admin] PATH_INFO: `/v3/projects/e9bdee4b7e6e4f3fb532ed213bcd5eeb/users/fc3101c81e904918aeef0b7f1fa7644e/roles/b98dfb3b9fee4644af2887a3ac3fd2e6` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:54:19.007937 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=100 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 10:54:19.054828 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15f1e822-5df2-4dda-b6ba-6418308144d9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e9bdee4b7e6e4f3fb532ed213bcd5eeb, user_id=fc3101c81e904918aeef0b7f1fa7644e, role_id=b98dfb3b9fee4644af2887a3ac3fd2e6)` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 10:54:19.061890 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15f1e822-5df2-4dda-b6ba-6418308144d9 admin admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 10:54:19.076396 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64c743aa-d605-4dfb-9e36-618fc4cef574 admin admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 10:54:19.081064 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-64c743aa-d605-4dfb-9e36-618fc4cef574 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 10:54:19.100209 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 856/1690] 10.222.0.25 () {60 vars in 1483 bytes} [Sun Aug 7 10:54:18 2022] PUT /identity/v3/projects/e9bdee4b7e6e4f3fb532ed213bcd5eeb/users/fc3101c81e904918aeef0b7f1fa7644e/roles/b98dfb3b9fee4644af2887a3ac3fd2e6 => generated 0 bytes in 180 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 07 10:54:19.118389 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3feb2db1-dd27-43f1-9af8-16254ca637c4 None None] REQUEST_METHOD: `POST` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:54:19.118907 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3feb2db1-dd27-43f1-9af8-16254ca637c4 None None] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:54:19.119403 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3feb2db1-dd27-43f1-9af8-16254ca637c4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:54:19.135850 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64c743aa-d605-4dfb-9e36-618fc4cef574 admin admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 10:54:19.182094 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.auth.core [None req-3feb2db1-dd27-43f1-9af8-16254ca637c4 None None] MFA Rules not processed for user `fc3101c81e904918aeef0b7f1fa7644e`. Rule list: `[]` (Enabled: `True`). {{(pid=80144) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 10:54:19.183808 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.fernet_utils [None req-3feb2db1-dd27-43f1-9af8-16254ca637c4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80144) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 10:54:19.289256 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41fbdef8-20ec-4343-87c3-3b403fed803e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5ac47e3b09f43d19a23a0f457fe9e23', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 10:54:19.291587 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41fbdef8-20ec-4343-87c3-3b403fed803e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:54:19.293161 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41fbdef8-20ec-4343-87c3-3b403fed803e admin admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:54:19.293636 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41fbdef8-20ec-4343-87c3-3b403fed803e admin admin] PATH_INFO: `/v3/users/76b81660953e460fae209814b208eff7` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:54:19.296474 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41fbdef8-20ec-4343-87c3-3b403fed803e admin admin] RBAC: Authorizing `identity:delete_user(user_id=76b81660953e460fae209814b208eff7)` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 10:54:19.300341 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41fbdef8-20ec-4343-87c3-3b403fed803e admin admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 10:54:19.400830 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 857/1691] 10.222.0.25 () {58 vars in 958 bytes} [Sun Aug 7 10:54:19 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 284 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 07 10:54:19.405602 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3feb2db1-dd27-43f1-9af8-16254ca637c4 None None] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 10:54:19.438179 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3feb2db1-dd27-43f1-9af8-16254ca637c4 None None] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 10:54:19.453110 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c010641d-84a0-489e-918a-9ac781e77728 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5ac47e3b09f43d19a23a0f457fe9e23', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 10:54:19.454030 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c010641d-84a0-489e-918a-9ac781e77728 admin admin] REQUEST_METHOD: `POST` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:54:19.455853 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-41fbdef8-20ec-4343-87c3-3b403fed803e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '76b81660953e460fae209814b208eff7'} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 10:54:19.457510 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c010641d-84a0-489e-918a-9ac781e77728 admin admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:54:19.457862 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c010641d-84a0-489e-918a-9ac781e77728 admin admin] PATH_INFO: `/v3/projects` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:54:19.458943 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c010641d-84a0-489e-918a-9ac781e77728 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 10:54:19.463767 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c010641d-84a0-489e-918a-9ac781e77728 admin admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 10:54:19.474527 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-41fbdef8-20ec-4343-87c3-3b403fed803e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '76b81660953e460fae209814b208eff7'} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 10:54:19.504315 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=88,keystone:INSERT=4,keystone:DELETE=8 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 10:54:19.521542 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-41fbdef8-20ec-4343-87c3-3b403fed803e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '76b81660953e460fae209814b208eff7'} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 10:54:19.526300 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-41fbdef8-20ec-4343-87c3-3b403fed803e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '76b81660953e460fae209814b208eff7'} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 10:54:19.543803 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-41fbdef8-20ec-4343-87c3-3b403fed803e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '76b81660953e460fae209814b208eff7'} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 10:54:19.571293 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 835/1692] 10.222.0.25 () {58 vars in 1224 bytes} [Sun Aug 7 10:54:19 2022] DELETE /identity/v3/users/76b81660953e460fae209814b208eff7 => generated 0 bytes in 495 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 07 10:54:19.577045 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37715355-6b7c-4004-ad56-fa366c5f344d None None] REQUEST_METHOD: `POST` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:54:19.580318 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37715355-6b7c-4004-ad56-fa366c5f344d None None] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:54:19.582447 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37715355-6b7c-4004-ad56-fa366c5f344d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:54:19.589048 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 858/1693] 10.222.0.25 () {60 vars in 1153 bytes} [Sun Aug 7 10:54:19 2022] POST /identity/v3/projects => generated 367 bytes in 184 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 07 10:54:19.591871 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c010641d-84a0-489e-918a-9ac781e77728 admin admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 10:54:19.597018 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.fernet_utils [None req-c010641d-84a0-489e-918a-9ac781e77728 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80144) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 10:54:19.683467 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.auth.core [None req-37715355-6b7c-4004-ad56-fa366c5f344d None None] MFA Rules not processed for user `bac4e6b76fe64217ac8fa44e71cffe6a`. Rule list: `[]` (Enabled: `True`). {{(pid=80143) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 10:54:19.694045 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-37715355-6b7c-4004-ad56-fa366c5f344d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 10:54:19.822009 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c010641d-84a0-489e-918a-9ac781e77728 admin admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 10:54:19.841615 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48f032be-e14a-4b6b-b549-27d1d92fd46f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5ac47e3b09f43d19a23a0f457fe9e23', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 10:54:19.842248 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48f032be-e14a-4b6b-b549-27d1d92fd46f admin admin] REQUEST_METHOD: `POST` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:54:19.842529 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48f032be-e14a-4b6b-b549-27d1d92fd46f admin admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:54:19.843413 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48f032be-e14a-4b6b-b549-27d1d92fd46f admin admin] PATH_INFO: `/v3/users` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:54:19.844104 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48f032be-e14a-4b6b-b549-27d1d92fd46f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 10:54:19.848141 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48f032be-e14a-4b6b-b549-27d1d92fd46f admin admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 10:54:19.896547 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 859/1694] 10.222.0.25 () {60 vars in 1144 bytes} [Sun Aug 7 10:54:19 2022] POST /identity/v3/users => generated 290 bytes in 305 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 07 10:54:19.900632 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48f032be-e14a-4b6b-b549-27d1d92fd46f admin admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 10:54:19.915466 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48f032be-e14a-4b6b-b549-27d1d92fd46f admin admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 10:54:19.923867 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f350e60-b252-4524-acd0-ecd96aa52b39 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5ac47e3b09f43d19a23a0f457fe9e23', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 10:54:19.925069 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f350e60-b252-4524-acd0-ecd96aa52b39 admin admin] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:54:19.925339 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f350e60-b252-4524-acd0-ecd96aa52b39 admin admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:54:19.925723 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f350e60-b252-4524-acd0-ecd96aa52b39 admin admin] PATH_INFO: `/v3/roles` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:54:19.926418 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f350e60-b252-4524-acd0-ecd96aa52b39 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 10:54:19.928317 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f350e60-b252-4524-acd0-ecd96aa52b39 admin admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 10:54:19.931477 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 836/1695] 10.222.0.25 () {58 vars in 958 bytes} [Sun Aug 7 10:54:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 356 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 07 10:54:19.944267 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37715355-6b7c-4004-ad56-fa366c5f344d None None] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 10:54:19.945950 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-37715355-6b7c-4004-ad56-fa366c5f344d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 10:54:19.949772 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 860/1696] 10.222.0.25 () {58 vars in 1122 bytes} [Sun Aug 7 10:54:19 2022] GET /identity/v3/roles => generated 1420 bytes in 49 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 07 10:54:19.957039 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f350e60-b252-4524-acd0-ecd96aa52b39 admin admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 10:54:19.984173 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f350e60-b252-4524-acd0-ecd96aa52b39 admin admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 10:54:19.996954 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a821c604-fd09-4795-8515-a1414a45bf71 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5ac47e3b09f43d19a23a0f457fe9e23', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 10:54:19.997647 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a821c604-fd09-4795-8515-a1414a45bf71 admin admin] REQUEST_METHOD: `PUT` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:54:20.000103 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a821c604-fd09-4795-8515-a1414a45bf71 admin admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:54:20.000910 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a821c604-fd09-4795-8515-a1414a45bf71 admin admin] PATH_INFO: `/v3/projects/2a2a9ab9e45c49ab80630b837cc08639/users/8b3b419b5b824f5087c9c8121adafc0a/roles/81b1f1bcee2a473088fbbd327c1c5b23` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:54:20.076128 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a821c604-fd09-4795-8515-a1414a45bf71 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2a2a9ab9e45c49ab80630b837cc08639, user_id=8b3b419b5b824f5087c9c8121adafc0a, role_id=81b1f1bcee2a473088fbbd327c1c5b23)` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 10:54:20.095588 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a821c604-fd09-4795-8515-a1414a45bf71 admin admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 10:54:20.130668 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 861/1697] 10.222.0.25 () {60 vars in 1483 bytes} [Sun Aug 7 10:54:19 2022] PUT /identity/v3/projects/2a2a9ab9e45c49ab80630b837cc08639/users/8b3b419b5b824f5087c9c8121adafc0a/roles/81b1f1bcee2a473088fbbd327c1c5b23 => generated 0 bytes in 176 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 07 10:54:20.134580 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a821c604-fd09-4795-8515-a1414a45bf71 admin admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 10:54:20.136696 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37715355-6b7c-4004-ad56-fa366c5f344d None None] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 10:54:20.159001 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 10:54:20.275401 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9009766e-0970-46f0-82df-bd4661446158 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be1283c9bf47424c9aa3ad27cab03027', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6de1f3cac02c4bcdb05dd2337176033d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 10:54:20.276165 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9009766e-0970-46f0-82df-bd4661446158 service neutron] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:54:20.276450 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9009766e-0970-46f0-82df-bd4661446158 service neutron] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:54:20.277351 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9009766e-0970-46f0-82df-bd4661446158 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:54:20.292479 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a821c604-fd09-4795-8515-a1414a45bf71 admin admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 10:54:20.300596 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9009766e-0970-46f0-82df-bd4661446158 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 10:54:20.302626 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9009766e-0970-46f0-82df-bd4661446158 service neutron] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 10:54:20.307667 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-958187ee-c9cc-4b54-b109-5b2511eb348c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5ac47e3b09f43d19a23a0f457fe9e23', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 10:54:20.308311 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-958187ee-c9cc-4b54-b109-5b2511eb348c admin admin] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:54:20.308669 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-958187ee-c9cc-4b54-b109-5b2511eb348c admin admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:54:20.308943 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-958187ee-c9cc-4b54-b109-5b2511eb348c admin admin] PATH_INFO: `/v3/roles` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:54:20.309789 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-958187ee-c9cc-4b54-b109-5b2511eb348c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 10:54:20.312068 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-958187ee-c9cc-4b54-b109-5b2511eb348c admin admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 10:54:20.333701 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 837/1698] 10.222.0.25 () {60 vars in 1365 bytes} [Sun Aug 7 10:54:19 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 393 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 07 10:54:20.336180 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 862/1699] 10.222.0.25 () {58 vars in 1122 bytes} [Sun Aug 7 10:54:20 2022] GET /identity/v3/roles => generated 1420 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 07 10:54:20.340926 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9009766e-0970-46f0-82df-bd4661446158 service neutron] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 10:54:20.358864 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9009766e-0970-46f0-82df-bd4661446158 service neutron] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 10:54:20.366212 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8ed1005-620f-40e6-ba74-129f240f3d87 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5ac47e3b09f43d19a23a0f457fe9e23', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 10:54:20.367340 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8ed1005-620f-40e6-ba74-129f240f3d87 admin admin] REQUEST_METHOD: `PUT` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:54:20.368043 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8ed1005-620f-40e6-ba74-129f240f3d87 admin admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:54:20.368605 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8ed1005-620f-40e6-ba74-129f240f3d87 admin admin] PATH_INFO: `/v3/projects/2a2a9ab9e45c49ab80630b837cc08639/users/8b3b419b5b824f5087c9c8121adafc0a/roles/81b1f1bcee2a473088fbbd327c1c5b23` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:54:20.372075 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8ed1005-620f-40e6-ba74-129f240f3d87 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2a2a9ab9e45c49ab80630b837cc08639, user_id=8b3b419b5b824f5087c9c8121adafc0a, role_id=81b1f1bcee2a473088fbbd327c1c5b23)` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 10:54:20.374974 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8ed1005-620f-40e6-ba74-129f240f3d87 admin admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 10:54:20.390732 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 838/1700] 10.222.0.25 () {60 vars in 1483 bytes} [Sun Aug 7 10:54:20 2022] PUT /identity/v3/projects/2a2a9ab9e45c49ab80630b837cc08639/users/8b3b419b5b824f5087c9c8121adafc0a/roles/81b1f1bcee2a473088fbbd327c1c5b23 => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 07 10:54:20.401727 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dca9cba-37cb-445f-85d8-4321bc259d37 None None] REQUEST_METHOD: `POST` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:54:20.403014 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dca9cba-37cb-445f-85d8-4321bc259d37 None None] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:54:20.403599 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dca9cba-37cb-445f-85d8-4321bc259d37 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:54:20.468002 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.auth.core [None req-7dca9cba-37cb-445f-85d8-4321bc259d37 None None] MFA Rules not processed for user `8b3b419b5b824f5087c9c8121adafc0a`. Rule list: `[]` (Enabled: `True`). {{(pid=80144) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 10:54:20.469338 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.fernet_utils [None req-7dca9cba-37cb-445f-85d8-4321bc259d37 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80144) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 10:54:20.716866 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 863/1701] 10.222.0.25 () {58 vars in 958 bytes} [Sun Aug 7 10:54:20 2022] POST /identity/v3/auth/tokens => generated 3311 bytes in 316 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 07 10:54:20.741225 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8ed1005-620f-40e6-ba74-129f240f3d87 admin admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 10:54:20.743089 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-d8ed1005-620f-40e6-ba74-129f240f3d87 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 10:54:20.895269 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=96,keystone:DELETE=2,keystone:INSERT=2 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 10:54:20.963238 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8ed1005-620f-40e6-ba74-129f240f3d87 admin admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 10:54:20.972053 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d975981-29e8-46b7-9576-0a753050b19c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88dbd6dfd1164994a3e16da553561910', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6de1f3cac02c4bcdb05dd2337176033d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 10:54:20.972705 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d975981-29e8-46b7-9576-0a753050b19c service cinder] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:54:20.972992 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d975981-29e8-46b7-9576-0a753050b19c service cinder] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:54:20.973272 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d975981-29e8-46b7-9576-0a753050b19c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:54:20.996659 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d975981-29e8-46b7-9576-0a753050b19c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 10:54:21.006848 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d975981-29e8-46b7-9576-0a753050b19c service cinder] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 10:54:21.034417 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 839/1702] 10.222.0.25 () {60 vars in 1365 bytes} [Sun Aug 7 10:54:20 2022] GET /identity/v3/auth/tokens => generated 3311 bytes in 294 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 07 10:54:21.062165 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-092b353e-783b-4774-8828-447ba3f92886 None None] REQUEST_METHOD: `POST` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:54:21.062477 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-092b353e-783b-4774-8828-447ba3f92886 None None] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:54:21.062755 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-092b353e-783b-4774-8828-447ba3f92886 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:54:21.116346 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d975981-29e8-46b7-9576-0a753050b19c service cinder] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 10:54:21.116346 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-6d975981-29e8-46b7-9576-0a753050b19c service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 10:54:21.163930 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.auth.core [None req-092b353e-783b-4774-8828-447ba3f92886 None None] MFA Rules not processed for user `2b757e3f47e44b9ba3645d12b20f7502`. Rule list: `[]` (Enabled: `True`). {{(pid=80144) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 10:54:21.166182 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.fernet_utils [None req-092b353e-783b-4774-8828-447ba3f92886 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80144) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 10:54:21.281006 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d975981-29e8-46b7-9576-0a753050b19c service cinder] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 10:54:21.312072 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbe6f83d-9887-4f3e-8ce2-48123da2c511 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5ac47e3b09f43d19a23a0f457fe9e23', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 10:54:21.312762 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbe6f83d-9887-4f3e-8ce2-48123da2c511 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:54:21.313048 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbe6f83d-9887-4f3e-8ce2-48123da2c511 admin admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:54:21.313323 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbe6f83d-9887-4f3e-8ce2-48123da2c511 admin admin] PATH_INFO: `/v3/projects/599c100ce4e7439bb77596d7f4cbdd35` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:54:21.320839 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbe6f83d-9887-4f3e-8ce2-48123da2c511 admin admin] RBAC: Authorizing `identity:delete_project(project_id=599c100ce4e7439bb77596d7f4cbdd35)` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 10:54:21.327489 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbe6f83d-9887-4f3e-8ce2-48123da2c511 admin admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 10:54:21.364162 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 864/1703] 10.222.0.25 () {58 vars in 958 bytes} [Sun Aug 7 10:54:21 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 303 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 07 10:54:21.365524 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-092b353e-783b-4774-8828-447ba3f92886 None None] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 10:54:21.367549 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.fernet_utils [None req-092b353e-783b-4774-8828-447ba3f92886 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80144) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 10:54:21.384105 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-092b353e-783b-4774-8828-447ba3f92886 None None] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 10:54:21.391006 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11d06ce9-6c44-4fc8-b3b7-c9e928e514f0 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88dbd6dfd1164994a3e16da553561910', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6de1f3cac02c4bcdb05dd2337176033d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 10:54:21.391848 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11d06ce9-6c44-4fc8-b3b7-c9e928e514f0 service cinder] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:54:21.392486 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11d06ce9-6c44-4fc8-b3b7-c9e928e514f0 service cinder] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:54:21.392985 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11d06ce9-6c44-4fc8-b3b7-c9e928e514f0 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:54:21.396410 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.fernet_utils [None req-11d06ce9-6c44-4fc8-b3b7-c9e928e514f0 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80144) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 10:54:21.445518 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=100 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 10:54:21.508721 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-dbe6f83d-9887-4f3e-8ce2-48123da2c511 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '599c100ce4e7439bb77596d7f4cbdd35', 'request_id': 'req-dbe6f83d-9887-4f3e-8ce2-48123da2c511'} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 10:54:21.523851 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-dbe6f83d-9887-4f3e-8ce2-48123da2c511 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '599c100ce4e7439bb77596d7f4cbdd35', 'request_id': 'req-dbe6f83d-9887-4f3e-8ce2-48123da2c511'} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 10:54:21.536044 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-dbe6f83d-9887-4f3e-8ce2-48123da2c511 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '599c100ce4e7439bb77596d7f4cbdd35', 'request_id': 'req-dbe6f83d-9887-4f3e-8ce2-48123da2c511'} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 10:54:21.563976 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-dbe6f83d-9887-4f3e-8ce2-48123da2c511 admin admin] The token cache is being invalidate because project 599c100ce4e7439bb77596d7f4cbdd35 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80143) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 07 10:54:21.567910 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-dbe6f83d-9887-4f3e-8ce2-48123da2c511 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 10:54:21.569348 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-dbe6f83d-9887-4f3e-8ce2-48123da2c511 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 10:54:21.578598 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 840/1704] 10.222.0.25 () {58 vars in 1233 bytes} [Sun Aug 7 10:54:21 2022] DELETE /identity/v3/projects/599c100ce4e7439bb77596d7f4cbdd35 => generated 0 bytes in 462 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 07 10:54:21.578598 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbe6f83d-9887-4f3e-8ce2-48123da2c511 admin admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 10:54:21.581353 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-dbe6f83d-9887-4f3e-8ce2-48123da2c511 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 10:54:21.613283 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11d06ce9-6c44-4fc8-b3b7-c9e928e514f0 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 10:54:21.614766 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11d06ce9-6c44-4fc8-b3b7-c9e928e514f0 service cinder] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 10:54:21.635400 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 865/1705] 10.222.0.25 () {60 vars in 1365 bytes} [Sun Aug 7 10:54:21 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 270 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 07 10:54:21.636392 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11d06ce9-6c44-4fc8-b3b7-c9e928e514f0 service cinder] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 10:54:21.681425 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11d06ce9-6c44-4fc8-b3b7-c9e928e514f0 service cinder] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 10:54:21.737532 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 10:54:21.758808 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbe6f83d-9887-4f3e-8ce2-48123da2c511 admin admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 10:54:21.780756 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd9cbbbb-d2dd-4155-b263-dfdb3ce73323 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31d16d1b96104b4eb690398432b9829e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6de1f3cac02c4bcdb05dd2337176033d', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'reader', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 10:54:21.781602 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd9cbbbb-d2dd-4155-b263-dfdb3ce73323 service nova] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:54:21.781893 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd9cbbbb-d2dd-4155-b263-dfdb3ce73323 service nova] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:54:21.782145 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd9cbbbb-d2dd-4155-b263-dfdb3ce73323 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:54:21.787952 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-fd9cbbbb-d2dd-4155-b263-dfdb3ce73323 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 10:54:21.793173 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb146aeb-2d2e-4964-bf6b-425a2ba9706a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5ac47e3b09f43d19a23a0f457fe9e23', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 10:54:21.794743 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb146aeb-2d2e-4964-bf6b-425a2ba9706a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:54:21.795064 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb146aeb-2d2e-4964-bf6b-425a2ba9706a admin admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:54:21.795348 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb146aeb-2d2e-4964-bf6b-425a2ba9706a admin admin] PATH_INFO: `/v3/users/d98015c8dd7f46cdb9f4155f15de5018` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:54:21.801682 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb146aeb-2d2e-4964-bf6b-425a2ba9706a admin admin] RBAC: Authorizing `identity:delete_user(user_id=d98015c8dd7f46cdb9f4155f15de5018)` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 10:54:21.804672 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb146aeb-2d2e-4964-bf6b-425a2ba9706a admin admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 10:54:21.893822 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.notifications [None req-cb146aeb-2d2e-4964-bf6b-425a2ba9706a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd98015c8dd7f46cdb9f4155f15de5018'} {{(pid=80144) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 10:54:21.911910 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.notifications [None req-cb146aeb-2d2e-4964-bf6b-425a2ba9706a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd98015c8dd7f46cdb9f4155f15de5018'} {{(pid=80144) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 10:54:21.924989 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd9cbbbb-d2dd-4155-b263-dfdb3ce73323 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 10:54:21.926006 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd9cbbbb-d2dd-4155-b263-dfdb3ce73323 service nova] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 10:54:21.926157 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.notifications [None req-cb146aeb-2d2e-4964-bf6b-425a2ba9706a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd98015c8dd7f46cdb9f4155f15de5018'} {{(pid=80144) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 10:54:21.927521 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.notifications [None req-cb146aeb-2d2e-4964-bf6b-425a2ba9706a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd98015c8dd7f46cdb9f4155f15de5018'} {{(pid=80144) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 10:54:21.928722 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.notifications [None req-cb146aeb-2d2e-4964-bf6b-425a2ba9706a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd98015c8dd7f46cdb9f4155f15de5018'} {{(pid=80144) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 10:54:21.945241 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 841/1706] 10.222.0.25 () {60 vars in 1365 bytes} [Sun Aug 7 10:54:21 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 369 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 07 10:54:21.971668 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 866/1707] 10.222.0.25 () {58 vars in 1224 bytes} [Sun Aug 7 10:54:21 2022] DELETE /identity/v3/users/d98015c8dd7f46cdb9f4155f15de5018 => generated 0 bytes in 332 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 07 10:54:23.132787 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd9cbbbb-d2dd-4155-b263-dfdb3ce73323 service nova] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 10:54:23.134725 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-fd9cbbbb-d2dd-4155-b263-dfdb3ce73323 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 10:54:23.184232 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd9cbbbb-d2dd-4155-b263-dfdb3ce73323 service nova] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 10:54:23.337629 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e021c72c-1ae6-4ac6-be8d-0b13123620de None None] REQUEST_METHOD: `POST` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:54:23.338197 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e021c72c-1ae6-4ac6-be8d-0b13123620de None None] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:54:23.338197 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e021c72c-1ae6-4ac6-be8d-0b13123620de None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:54:23.381470 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a30b7088-d0fb-4354-ba6f-90aff3fa0a94 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5ac47e3b09f43d19a23a0f457fe9e23', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 10:54:23.383301 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a30b7088-d0fb-4354-ba6f-90aff3fa0a94 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:54:23.383456 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a30b7088-d0fb-4354-ba6f-90aff3fa0a94 admin admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:54:23.385322 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a30b7088-d0fb-4354-ba6f-90aff3fa0a94 admin admin] PATH_INFO: `/v3/projects/f392209c49534024a25a0a839234bba0` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:54:23.390706 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a30b7088-d0fb-4354-ba6f-90aff3fa0a94 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f392209c49534024a25a0a839234bba0)` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 10:54:23.393535 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a30b7088-d0fb-4354-ba6f-90aff3fa0a94 admin admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 10:54:23.456816 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.auth.core [None req-e021c72c-1ae6-4ac6-be8d-0b13123620de None None] MFA Rules not processed for user `de1d6c24b1a14d10a20d82a4c65793d8`. Rule list: `[]` (Enabled: `True`). {{(pid=80144) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 10:54:23.458072 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.fernet_utils [None req-e021c72c-1ae6-4ac6-be8d-0b13123620de None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80144) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 10:54:23.471660 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 10:54:23.562198 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-a30b7088-d0fb-4354-ba6f-90aff3fa0a94 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f392209c49534024a25a0a839234bba0', 'request_id': 'req-a30b7088-d0fb-4354-ba6f-90aff3fa0a94'} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 10:54:23.580186 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-a30b7088-d0fb-4354-ba6f-90aff3fa0a94 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f392209c49534024a25a0a839234bba0', 'request_id': 'req-a30b7088-d0fb-4354-ba6f-90aff3fa0a94'} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 10:54:23.594042 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-a30b7088-d0fb-4354-ba6f-90aff3fa0a94 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f392209c49534024a25a0a839234bba0', 'request_id': 'req-a30b7088-d0fb-4354-ba6f-90aff3fa0a94'} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 10:54:23.623102 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-a30b7088-d0fb-4354-ba6f-90aff3fa0a94 admin admin] The token cache is being invalidate because project f392209c49534024a25a0a839234bba0 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80143) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 07 10:54:23.628748 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-a30b7088-d0fb-4354-ba6f-90aff3fa0a94 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 10:54:23.630180 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 867/1708] 10.222.0.25 () {58 vars in 1009 bytes} [Sun Aug 7 10:54:23 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 293 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 07 10:54:23.632595 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-a30b7088-d0fb-4354-ba6f-90aff3fa0a94 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 10:54:23.635378 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 842/1709] 10.222.0.25 () {58 vars in 1233 bytes} [Sun Aug 7 10:54:23 2022] DELETE /identity/v3/projects/f392209c49534024a25a0a839234bba0 => generated 0 bytes in 503 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 07 10:54:23.640559 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f310cc5-4b30-457e-a1c7-350f1be975b6 None None] REQUEST_METHOD: `POST` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:54:23.640860 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f310cc5-4b30-457e-a1c7-350f1be975b6 None None] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:54:23.641230 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f310cc5-4b30-457e-a1c7-350f1be975b6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:54:23.683048 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.auth.core [None req-8f310cc5-4b30-457e-a1c7-350f1be975b6 None None] MFA Rules not processed for user `bac4e6b76fe64217ac8fa44e71cffe6a`. Rule list: `[]` (Enabled: `True`). {{(pid=80144) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 10:54:23.692572 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.fernet_utils [None req-8f310cc5-4b30-457e-a1c7-350f1be975b6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80144) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 10:54:23.905613 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 868/1710] 10.222.0.25 () {58 vars in 958 bytes} [Sun Aug 7 10:54:23 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 266 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 07 10:54:23.913649 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-119a198b-8eef-4a0a-ab09-7bf54b4b9f72 None None] REQUEST_METHOD: `POST` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:54:23.914043 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-119a198b-8eef-4a0a-ab09-7bf54b4b9f72 None None] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:54:23.914147 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-119a198b-8eef-4a0a-ab09-7bf54b4b9f72 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:54:23.964096 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.auth.core [None req-119a198b-8eef-4a0a-ab09-7bf54b4b9f72 None None] MFA Rules not processed for user `bac4e6b76fe64217ac8fa44e71cffe6a`. Rule list: `[]` (Enabled: `True`). {{(pid=80143) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 10:54:23.965329 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-119a198b-8eef-4a0a-ab09-7bf54b4b9f72 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 10:54:23.990356 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 843/1711] 10.222.0.25 () {58 vars in 958 bytes} [Sun Aug 7 10:54:23 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 78 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 07 10:54:23.994917 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f310cc5-4b30-457e-a1c7-350f1be975b6 None None] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 10:54:24.024180 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-119a198b-8eef-4a0a-ab09-7bf54b4b9f72 None None] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 10:54:24.026475 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-119a198b-8eef-4a0a-ab09-7bf54b4b9f72 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 10:54:24.028507 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f310cc5-4b30-457e-a1c7-350f1be975b6 None None] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 10:54:24.054177 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b1ef330-7c4f-4e08-8c18-e88904475a9d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5ac47e3b09f43d19a23a0f457fe9e23', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 10:54:24.054911 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b1ef330-7c4f-4e08-8c18-e88904475a9d admin admin] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:54:24.060150 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b1ef330-7c4f-4e08-8c18-e88904475a9d admin admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:54:24.060150 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b1ef330-7c4f-4e08-8c18-e88904475a9d admin admin] PATH_INFO: `/v3/domains` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:54:24.060533 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b1ef330-7c4f-4e08-8c18-e88904475a9d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80144) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 10:54:24.062456 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b1ef330-7c4f-4e08-8c18-e88904475a9d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 10:54:24.065176 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b1ef330-7c4f-4e08-8c18-e88904475a9d admin admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 10:54:24.110578 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 869/1712] 10.222.0.25 () {58 vars in 1153 bytes} [Sun Aug 7 10:54:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 113 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 10:54:24.117561 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b1ef330-7c4f-4e08-8c18-e88904475a9d admin admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 10:54:24.125137 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 10:54:24.149282 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b1ef330-7c4f-4e08-8c18-e88904475a9d admin admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 10:54:24.162387 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c8e7966-fefd-454f-a4bb-0b591e234edc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5ac47e3b09f43d19a23a0f457fe9e23', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 10:54:24.165829 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c8e7966-fefd-454f-a4bb-0b591e234edc admin admin] REQUEST_METHOD: `POST` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:54:24.167584 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c8e7966-fefd-454f-a4bb-0b591e234edc admin admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:54:24.170050 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c8e7966-fefd-454f-a4bb-0b591e234edc admin admin] PATH_INFO: `/v3/projects` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:54:24.171178 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c8e7966-fefd-454f-a4bb-0b591e234edc admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 10:54:24.184019 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c8e7966-fefd-454f-a4bb-0b591e234edc admin admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 10:54:24.259271 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 870/1713] 10.222.0.25 () {60 vars in 1153 bytes} [Sun Aug 7 10:54:24 2022] POST /identity/v3/projects => generated 355 bytes in 142 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 07 10:54:24.264036 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c8e7966-fefd-454f-a4bb-0b591e234edc admin admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 10:54:24.268023 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.fernet_utils [None req-3c8e7966-fefd-454f-a4bb-0b591e234edc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80144) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 10:54:24.305962 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-119a198b-8eef-4a0a-ab09-7bf54b4b9f72 None None] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 10:54:24.449399 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c8e7966-fefd-454f-a4bb-0b591e234edc admin admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 10:54:24.467722 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22e2cdd9-435d-45d0-9d8f-01653dfb29bd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5ac47e3b09f43d19a23a0f457fe9e23', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 10:54:24.483957 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22e2cdd9-435d-45d0-9d8f-01653dfb29bd admin admin] REQUEST_METHOD: `DELETE` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:54:24.484410 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22e2cdd9-435d-45d0-9d8f-01653dfb29bd admin admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:54:24.484816 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22e2cdd9-435d-45d0-9d8f-01653dfb29bd admin admin] PATH_INFO: `/v3/users/60bf4ef3bca04a418ee55888f86384b1` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:54:24.489762 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22e2cdd9-435d-45d0-9d8f-01653dfb29bd admin admin] RBAC: Authorizing `identity:delete_user(user_id=60bf4ef3bca04a418ee55888f86384b1)` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 10:54:24.491032 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd271a96-7da7-42f8-97c2-356186a7519e service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69490ea2a9a340bf91aca38bf5655b45', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6de1f3cac02c4bcdb05dd2337176033d', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 10:54:24.491953 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22e2cdd9-435d-45d0-9d8f-01653dfb29bd admin admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 10:54:24.492230 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd271a96-7da7-42f8-97c2-356186a7519e service swift] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:54:24.492765 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd271a96-7da7-42f8-97c2-356186a7519e service swift] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:54:24.493219 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd271a96-7da7-42f8-97c2-356186a7519e service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:54:24.496993 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-fd271a96-7da7-42f8-97c2-356186a7519e service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 10:54:24.515473 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=98,keystone:INSERT=1,keystone:DELETE=1 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 10:54:24.593188 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.notifications [None req-22e2cdd9-435d-45d0-9d8f-01653dfb29bd admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '60bf4ef3bca04a418ee55888f86384b1'} {{(pid=80144) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 10:54:24.617199 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.notifications [None req-22e2cdd9-435d-45d0-9d8f-01653dfb29bd admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '60bf4ef3bca04a418ee55888f86384b1'} {{(pid=80144) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 10:54:24.635883 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.notifications [None req-22e2cdd9-435d-45d0-9d8f-01653dfb29bd admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '60bf4ef3bca04a418ee55888f86384b1'} {{(pid=80144) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 10:54:24.636482 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.notifications [None req-22e2cdd9-435d-45d0-9d8f-01653dfb29bd admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '60bf4ef3bca04a418ee55888f86384b1'} {{(pid=80144) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 10:54:24.654751 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.notifications [None req-22e2cdd9-435d-45d0-9d8f-01653dfb29bd admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '60bf4ef3bca04a418ee55888f86384b1'} {{(pid=80144) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 10:54:24.702691 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd271a96-7da7-42f8-97c2-356186a7519e service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 10:54:24.706444 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd271a96-7da7-42f8-97c2-356186a7519e service swift] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 10:54:24.743209 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 871/1714] 10.222.0.25 () {58 vars in 1224 bytes} [Sun Aug 7 10:54:24 2022] DELETE /identity/v3/users/60bf4ef3bca04a418ee55888f86384b1 => generated 0 bytes in 483 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 07 10:54:24.744079 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22e2cdd9-435d-45d0-9d8f-01653dfb29bd admin admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 10:54:24.754631 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.fernet_utils [None req-22e2cdd9-435d-45d0-9d8f-01653dfb29bd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80144) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 10:54:24.758145 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 844/1715] 10.222.0.25 () {60 vars in 1384 bytes} [Sun Aug 7 10:54:24 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 734 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 07 10:54:24.760233 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0b34aa5-fb12-4dd4-b0c0-cf88b9073b02 None None] REQUEST_METHOD: `POST` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:54:24.760928 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0b34aa5-fb12-4dd4-b0c0-cf88b9073b02 None None] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:54:24.761214 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0b34aa5-fb12-4dd4-b0c0-cf88b9073b02 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:54:24.911698 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.auth.core [None req-f0b34aa5-fb12-4dd4-b0c0-cf88b9073b02 None None] MFA Rules not processed for user `616a497708664cca8bf7cfe93cbfd01f`. Rule list: `[]` (Enabled: `True`). {{(pid=80143) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 10:54:24.912976 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-f0b34aa5-fb12-4dd4-b0c0-cf88b9073b02 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 10:54:25.127737 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22e2cdd9-435d-45d0-9d8f-01653dfb29bd admin admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 10:54:25.153979 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=100 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 10:54:25.167027 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec9b2bf0-60f5-4145-bf00-e47824131c1e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5ac47e3b09f43d19a23a0f457fe9e23', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 10:54:25.170100 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec9b2bf0-60f5-4145-bf00-e47824131c1e admin admin] REQUEST_METHOD: `POST` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:54:25.171218 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec9b2bf0-60f5-4145-bf00-e47824131c1e admin admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:54:25.171653 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec9b2bf0-60f5-4145-bf00-e47824131c1e admin admin] PATH_INFO: `/v3/users` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:54:25.172747 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec9b2bf0-60f5-4145-bf00-e47824131c1e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 10:54:25.181868 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec9b2bf0-60f5-4145-bf00-e47824131c1e admin admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 10:54:25.232711 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 845/1716] 10.222.0.25 () {58 vars in 958 bytes} [Sun Aug 7 10:54:24 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 474 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 07 10:54:25.235583 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1841e3f-c9ab-40db-897b-c1f3d5463474 None None] REQUEST_METHOD: `POST` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:54:25.237247 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1841e3f-c9ab-40db-897b-c1f3d5463474 None None] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:54:25.237805 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1841e3f-c9ab-40db-897b-c1f3d5463474 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:54:25.248674 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 872/1717] 10.222.0.25 () {60 vars in 1144 bytes} [Sun Aug 7 10:54:24 2022] POST /identity/v3/users => generated 278 bytes in 504 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 07 10:54:25.251957 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5270c333-3339-4587-b5b3-d01048bf5908 None None] REQUEST_METHOD: `POST` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:54:25.252254 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5270c333-3339-4587-b5b3-d01048bf5908 None None] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:54:25.252720 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5270c333-3339-4587-b5b3-d01048bf5908 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:54:25.274886 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.auth.core [None req-f1841e3f-c9ab-40db-897b-c1f3d5463474 None None] MFA Rules not processed for user `bac4e6b76fe64217ac8fa44e71cffe6a`. Rule list: `[]` (Enabled: `True`). {{(pid=80143) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 10:54:25.285648 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-f1841e3f-c9ab-40db-897b-c1f3d5463474 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 10:54:25.301235 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.auth.core [None req-5270c333-3339-4587-b5b3-d01048bf5908 None None] MFA Rules not processed for user `616a497708664cca8bf7cfe93cbfd01f`. Rule list: `[]` (Enabled: `True`). {{(pid=80144) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 10:54:25.306339 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.fernet_utils [None req-5270c333-3339-4587-b5b3-d01048bf5908 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80144) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 10:54:25.318071 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 846/1718] 10.222.0.25 () {58 vars in 958 bytes} [Sun Aug 7 10:54:25 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 07 10:54:25.320248 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1841e3f-c9ab-40db-897b-c1f3d5463474 None None] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 10:54:25.326093 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 873/1719] 10.222.0.25 () {58 vars in 958 bytes} [Sun Aug 7 10:54:25 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 77 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 07 10:54:25.339235 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5270c333-3339-4587-b5b3-d01048bf5908 None None] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 10:54:25.340642 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.fernet_utils [None req-5270c333-3339-4587-b5b3-d01048bf5908 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80144) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 10:54:25.343151 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1841e3f-c9ab-40db-897b-c1f3d5463474 None None] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 10:54:25.354130 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e58eb30f-8163-48ca-9bbd-75ac0eef8754 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5ac47e3b09f43d19a23a0f457fe9e23', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 10:54:25.355340 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e58eb30f-8163-48ca-9bbd-75ac0eef8754 admin admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:54:25.359386 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e58eb30f-8163-48ca-9bbd-75ac0eef8754 admin admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:54:25.362135 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e58eb30f-8163-48ca-9bbd-75ac0eef8754 admin admin] PATH_INFO: `/v3/roles` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:54:25.367036 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e58eb30f-8163-48ca-9bbd-75ac0eef8754 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 10:54:25.369085 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e58eb30f-8163-48ca-9bbd-75ac0eef8754 admin admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 10:54:25.397084 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 847/1720] 10.222.0.25 () {58 vars in 1122 bytes} [Sun Aug 7 10:54:25 2022] GET /identity/v3/roles => generated 1420 bytes in 78 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 07 10:54:25.399163 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e58eb30f-8163-48ca-9bbd-75ac0eef8754 admin admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 10:54:25.404309 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-e58eb30f-8163-48ca-9bbd-75ac0eef8754 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 10:54:25.512119 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5270c333-3339-4587-b5b3-d01048bf5908 None None] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 10:54:25.522283 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f20c560a-11dc-499e-9213-99e9531d249b service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31d16d1b96104b4eb690398432b9829e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6de1f3cac02c4bcdb05dd2337176033d', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'reader', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 10:54:25.524002 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f20c560a-11dc-499e-9213-99e9531d249b service nova] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:54:25.524415 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f20c560a-11dc-499e-9213-99e9531d249b service nova] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:54:25.524796 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f20c560a-11dc-499e-9213-99e9531d249b service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:54:25.536742 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f20c560a-11dc-499e-9213-99e9531d249b service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 10:54:25.539781 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f20c560a-11dc-499e-9213-99e9531d249b service nova] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 10:54:25.545697 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=87,keystone:DELETE=9,keystone:INSERT=4 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 10:54:25.560357 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 874/1721] 10.222.0.25 () {60 vars in 1365 bytes} [Sun Aug 7 10:54:25 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 221 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 07 10:54:25.562183 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f20c560a-11dc-499e-9213-99e9531d249b service nova] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 10:54:25.569107 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e58eb30f-8163-48ca-9bbd-75ac0eef8754 admin admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 10:54:25.574698 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f20c560a-11dc-499e-9213-99e9531d249b service nova] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 10:54:25.577222 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b2a5bb2-d01a-439f-9be4-8a3174f6da5c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be1283c9bf47424c9aa3ad27cab03027', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6de1f3cac02c4bcdb05dd2337176033d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 10:54:25.578296 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b2a5bb2-d01a-439f-9be4-8a3174f6da5c service neutron] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:54:25.578881 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b2a5bb2-d01a-439f-9be4-8a3174f6da5c service neutron] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:54:25.579195 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b2a5bb2-d01a-439f-9be4-8a3174f6da5c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:54:25.582404 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-344139f0-56c1-4767-b535-e035a530f5e6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5ac47e3b09f43d19a23a0f457fe9e23', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 10:54:25.583379 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-344139f0-56c1-4767-b535-e035a530f5e6 admin admin] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:54:25.583786 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-344139f0-56c1-4767-b535-e035a530f5e6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:54:25.584165 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-344139f0-56c1-4767-b535-e035a530f5e6 admin admin] PATH_INFO: `/v3/roles` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:54:25.585028 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-344139f0-56c1-4767-b535-e035a530f5e6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 10:54:25.586863 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-344139f0-56c1-4767-b535-e035a530f5e6 admin admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 10:54:25.593421 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b2a5bb2-d01a-439f-9be4-8a3174f6da5c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 10:54:25.602314 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 875/1722] 10.222.0.25 () {58 vars in 1122 bytes} [Sun Aug 7 10:54:25 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 07 10:54:25.602560 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b2a5bb2-d01a-439f-9be4-8a3174f6da5c service neutron] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 10:54:25.608409 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-344139f0-56c1-4767-b535-e035a530f5e6 admin admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 10:54:25.635238 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-344139f0-56c1-4767-b535-e035a530f5e6 admin admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 10:54:25.650548 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3f09902-81ab-4c50-acc8-3c518534b97c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5ac47e3b09f43d19a23a0f457fe9e23', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 10:54:25.651388 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3f09902-81ab-4c50-acc8-3c518534b97c admin admin] REQUEST_METHOD: `PUT` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:54:25.651761 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3f09902-81ab-4c50-acc8-3c518534b97c admin admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:54:25.652215 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3f09902-81ab-4c50-acc8-3c518534b97c admin admin] PATH_INFO: `/v3/projects/105d41e438e6421898223c9f12bebd9a/users/f512dd6985224deb91640fd4f6401165/roles/b98dfb3b9fee4644af2887a3ac3fd2e6` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:54:25.668755 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 848/1723] 10.222.0.25 () {60 vars in 1365 bytes} [Sun Aug 7 10:54:25 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 271 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 07 10:54:25.690587 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3f09902-81ab-4c50-acc8-3c518534b97c admin admin] RBAC: Authorizing `identity:create_grant(project_id=105d41e438e6421898223c9f12bebd9a, user_id=f512dd6985224deb91640fd4f6401165, role_id=b98dfb3b9fee4644af2887a3ac3fd2e6)` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 10:54:25.693777 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3f09902-81ab-4c50-acc8-3c518534b97c admin admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 10:54:25.719554 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 876/1724] 10.222.0.25 () {60 vars in 1483 bytes} [Sun Aug 7 10:54:25 2022] PUT /identity/v3/projects/105d41e438e6421898223c9f12bebd9a/users/f512dd6985224deb91640fd4f6401165/roles/b98dfb3b9fee4644af2887a3ac3fd2e6 => generated 0 bytes in 112 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 07 10:54:25.734043 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8204a29a-6a4d-4f35-beca-a2b58bd4429a None None] REQUEST_METHOD: `POST` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:54:25.734692 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8204a29a-6a4d-4f35-beca-a2b58bd4429a None None] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:54:25.736441 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8204a29a-6a4d-4f35-beca-a2b58bd4429a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:54:25.966398 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.auth.core [None req-8204a29a-6a4d-4f35-beca-a2b58bd4429a None None] MFA Rules not processed for user `f512dd6985224deb91640fd4f6401165`. Rule list: `[]` (Enabled: `True`). {{(pid=80143) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 10:54:25.973965 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-8204a29a-6a4d-4f35-beca-a2b58bd4429a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 10:54:26.277182 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 849/1725] 10.222.0.25 () {58 vars in 958 bytes} [Sun Aug 7 10:54:25 2022] POST /identity/v3/auth/tokens => generated 3232 bytes in 542 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 07 10:54:26.279881 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3f09902-81ab-4c50-acc8-3c518534b97c admin admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 10:54:26.438525 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3f09902-81ab-4c50-acc8-3c518534b97c admin admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 10:54:26.445493 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2a7609c-947a-4245-8719-790eb9948bad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5ac47e3b09f43d19a23a0f457fe9e23', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 10:54:26.446129 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2a7609c-947a-4245-8719-790eb9948bad admin admin] REQUEST_METHOD: `POST` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:54:26.446419 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2a7609c-947a-4245-8719-790eb9948bad admin admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:54:26.447250 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2a7609c-947a-4245-8719-790eb9948bad admin admin] PATH_INFO: `/v3/projects` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:54:26.447904 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2a7609c-947a-4245-8719-790eb9948bad admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 10:54:26.449822 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2a7609c-947a-4245-8719-790eb9948bad admin admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 10:54:26.490488 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 877/1726] 10.222.0.25 () {60 vars in 1153 bytes} [Sun Aug 7 10:54:26 2022] POST /identity/v3/projects => generated 357 bytes in 211 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 07 10:54:26.495201 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8204a29a-6a4d-4f35-beca-a2b58bd4429a None None] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 10:54:26.522908 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8204a29a-6a4d-4f35-beca-a2b58bd4429a None None] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 10:54:26.672934 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=100 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 10:54:26.695272 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa953e06-ea5c-46d3-9319-5de80fac2261 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5ac47e3b09f43d19a23a0f457fe9e23', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 10:54:26.697259 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa953e06-ea5c-46d3-9319-5de80fac2261 admin admin] REQUEST_METHOD: `POST` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:54:26.697613 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa953e06-ea5c-46d3-9319-5de80fac2261 admin admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:54:26.697876 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa953e06-ea5c-46d3-9319-5de80fac2261 admin admin] PATH_INFO: `/v3/users` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:54:26.698542 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa953e06-ea5c-46d3-9319-5de80fac2261 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 10:54:26.700525 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa953e06-ea5c-46d3-9319-5de80fac2261 admin admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 10:54:26.749737 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 850/1727] 10.222.0.25 () {60 vars in 1144 bytes} [Sun Aug 7 10:54:26 2022] POST /identity/v3/users => generated 285 bytes in 255 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 07 10:54:26.753829 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2a7609c-947a-4245-8719-790eb9948bad admin admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 10:54:26.775288 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2a7609c-947a-4245-8719-790eb9948bad admin admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 10:54:26.788429 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-453ed647-7e1e-4919-af82-438bf17ecc43 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5ac47e3b09f43d19a23a0f457fe9e23', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 10:54:26.789184 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-453ed647-7e1e-4919-af82-438bf17ecc43 admin admin] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:54:26.789587 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-453ed647-7e1e-4919-af82-438bf17ecc43 admin admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:54:26.790081 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-453ed647-7e1e-4919-af82-438bf17ecc43 admin admin] PATH_INFO: `/v3/roles` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:54:26.791013 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-453ed647-7e1e-4919-af82-438bf17ecc43 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 10:54:26.797473 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-453ed647-7e1e-4919-af82-438bf17ecc43 admin admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 10:54:26.867691 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 878/1728] 10.222.0.25 () {58 vars in 1122 bytes} [Sun Aug 7 10:54:26 2022] GET /identity/v3/roles => generated 1420 bytes in 114 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 07 10:54:26.872012 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa953e06-ea5c-46d3-9319-5de80fac2261 admin admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 10:54:26.899728 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa953e06-ea5c-46d3-9319-5de80fac2261 admin admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 10:54:26.924138 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61474fb9-f96f-40c9-ab2a-32fbbbd2091b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5ac47e3b09f43d19a23a0f457fe9e23', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 10:54:26.925046 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61474fb9-f96f-40c9-ab2a-32fbbbd2091b admin admin] REQUEST_METHOD: `PUT` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:54:26.925433 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61474fb9-f96f-40c9-ab2a-32fbbbd2091b admin admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:54:26.925891 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61474fb9-f96f-40c9-ab2a-32fbbbd2091b admin admin] PATH_INFO: `/v3/projects/f9e32fe227d947a889edc33cde5525f1/users/38eee0c6e1984fe7968706a30667e779/roles/81b1f1bcee2a473088fbbd327c1c5b23` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:54:26.990339 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61474fb9-f96f-40c9-ab2a-32fbbbd2091b admin admin] RBAC: Authorizing `identity:create_grant(project_id=f9e32fe227d947a889edc33cde5525f1, user_id=38eee0c6e1984fe7968706a30667e779, role_id=81b1f1bcee2a473088fbbd327c1c5b23)` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 10:54:26.994252 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61474fb9-f96f-40c9-ab2a-32fbbbd2091b admin admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 10:54:27.024309 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 851/1729] 10.222.0.25 () {60 vars in 1483 bytes} [Sun Aug 7 10:54:26 2022] PUT /identity/v3/projects/f9e32fe227d947a889edc33cde5525f1/users/38eee0c6e1984fe7968706a30667e779/roles/81b1f1bcee2a473088fbbd327c1c5b23 => generated 0 bytes in 153 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 07 10:54:27.030102 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-453ed647-7e1e-4919-af82-438bf17ecc43 admin admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 10:54:27.045336 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-453ed647-7e1e-4919-af82-438bf17ecc43 admin admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 10:54:27.217259 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61474fb9-f96f-40c9-ab2a-32fbbbd2091b admin admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 10:54:27.221470 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-61474fb9-f96f-40c9-ab2a-32fbbbd2091b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 10:54:27.410136 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09d17bef-a8d4-4ff1-a4b9-984cb64252e7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5ac47e3b09f43d19a23a0f457fe9e23', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 10:54:27.415374 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09d17bef-a8d4-4ff1-a4b9-984cb64252e7 admin admin] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:54:27.420086 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09d17bef-a8d4-4ff1-a4b9-984cb64252e7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:54:27.420388 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09d17bef-a8d4-4ff1-a4b9-984cb64252e7 admin admin] PATH_INFO: `/v3/roles` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:54:27.426135 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09d17bef-a8d4-4ff1-a4b9-984cb64252e7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 10:54:27.437527 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09d17bef-a8d4-4ff1-a4b9-984cb64252e7 admin admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 10:54:27.495277 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 879/1730] 10.222.0.25 () {58 vars in 1122 bytes} [Sun Aug 7 10:54:27 2022] GET /identity/v3/roles => generated 1420 bytes in 466 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 07 10:54:27.496826 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09d17bef-a8d4-4ff1-a4b9-984cb64252e7 admin admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 10:54:27.543413 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09d17bef-a8d4-4ff1-a4b9-984cb64252e7 admin admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 10:54:27.556328 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61474fb9-f96f-40c9-ab2a-32fbbbd2091b admin admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 10:54:27.585146 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50a83ffd-f7a8-4604-a9ee-0f65d399f6fb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5ac47e3b09f43d19a23a0f457fe9e23', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 10:54:27.586081 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50a83ffd-f7a8-4604-a9ee-0f65d399f6fb admin admin] REQUEST_METHOD: `PUT` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:54:27.586901 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50a83ffd-f7a8-4604-a9ee-0f65d399f6fb admin admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:54:27.587454 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50a83ffd-f7a8-4604-a9ee-0f65d399f6fb admin admin] PATH_INFO: `/v3/projects/f9e32fe227d947a889edc33cde5525f1/users/38eee0c6e1984fe7968706a30667e779/roles/81b1f1bcee2a473088fbbd327c1c5b23` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:54:27.591799 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50a83ffd-f7a8-4604-a9ee-0f65d399f6fb admin admin] RBAC: Authorizing `identity:create_grant(project_id=f9e32fe227d947a889edc33cde5525f1, user_id=38eee0c6e1984fe7968706a30667e779, role_id=81b1f1bcee2a473088fbbd327c1c5b23)` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 10:54:27.594566 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5bbaf70-d9d4-458d-b69b-35d08d218bfd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5ac47e3b09f43d19a23a0f457fe9e23', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 10:54:27.597819 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5bbaf70-d9d4-458d-b69b-35d08d218bfd admin admin] REQUEST_METHOD: `DELETE` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:54:27.599443 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5bbaf70-d9d4-458d-b69b-35d08d218bfd admin admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:54:27.600283 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50a83ffd-f7a8-4604-a9ee-0f65d399f6fb admin admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 10:54:27.601024 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5bbaf70-d9d4-458d-b69b-35d08d218bfd admin admin] PATH_INFO: `/v3/users/fc3101c81e904918aeef0b7f1fa7644e` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:54:27.607137 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5bbaf70-d9d4-458d-b69b-35d08d218bfd admin admin] RBAC: Authorizing `identity:delete_user(user_id=fc3101c81e904918aeef0b7f1fa7644e)` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 10:54:27.633617 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5bbaf70-d9d4-458d-b69b-35d08d218bfd admin admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 10:54:27.645752 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 880/1731] 10.222.0.25 () {60 vars in 1483 bytes} [Sun Aug 7 10:54:27 2022] PUT /identity/v3/projects/f9e32fe227d947a889edc33cde5525f1/users/38eee0c6e1984fe7968706a30667e779/roles/81b1f1bcee2a473088fbbd327c1c5b23 => generated 0 bytes in 150 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 07 10:54:27.663822 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-102a332a-042a-41a1-934c-9f2bf134a211 None None] REQUEST_METHOD: `POST` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:54:27.664105 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-102a332a-042a-41a1-934c-9f2bf134a211 None None] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:54:27.664390 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-102a332a-042a-41a1-934c-9f2bf134a211 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:54:27.790859 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-a5bbaf70-d9d4-458d-b69b-35d08d218bfd admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'fc3101c81e904918aeef0b7f1fa7644e'} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 10:54:27.805176 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-a5bbaf70-d9d4-458d-b69b-35d08d218bfd admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'fc3101c81e904918aeef0b7f1fa7644e'} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 10:54:27.809411 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.auth.core [None req-102a332a-042a-41a1-934c-9f2bf134a211 None None] MFA Rules not processed for user `38eee0c6e1984fe7968706a30667e779`. Rule list: `[]` (Enabled: `True`). {{(pid=80144) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 10:54:27.812664 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.fernet_utils [None req-102a332a-042a-41a1-934c-9f2bf134a211 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80144) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 10:54:27.839419 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-a5bbaf70-d9d4-458d-b69b-35d08d218bfd admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'fc3101c81e904918aeef0b7f1fa7644e'} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 10:54:27.840301 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-a5bbaf70-d9d4-458d-b69b-35d08d218bfd admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'fc3101c81e904918aeef0b7f1fa7644e'} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 10:54:27.864080 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-a5bbaf70-d9d4-458d-b69b-35d08d218bfd admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'fc3101c81e904918aeef0b7f1fa7644e'} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 10:54:27.875050 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 852/1732] 10.222.0.25 () {58 vars in 1224 bytes} [Sun Aug 7 10:54:27 2022] DELETE /identity/v3/users/fc3101c81e904918aeef0b7f1fa7644e => generated 0 bytes in 655 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 07 10:54:27.875050 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5bbaf70-d9d4-458d-b69b-35d08d218bfd admin admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 10:54:27.879567 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-a5bbaf70-d9d4-458d-b69b-35d08d218bfd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 10:54:27.950368 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 10:54:28.017949 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=85,keystone:INSERT=5,keystone:DELETE=10 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 10:54:28.115599 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5bbaf70-d9d4-458d-b69b-35d08d218bfd admin admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 10:54:28.131383 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53b88ac2-9f91-4b42-90ee-cab13abe31da admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5ac47e3b09f43d19a23a0f457fe9e23', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 10:54:28.132773 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53b88ac2-9f91-4b42-90ee-cab13abe31da admin admin] REQUEST_METHOD: `DELETE` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:54:28.133066 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53b88ac2-9f91-4b42-90ee-cab13abe31da admin admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:54:28.135500 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53b88ac2-9f91-4b42-90ee-cab13abe31da admin admin] PATH_INFO: `/v3/users/253a30f00d6f42d0af49f80acfeadcfa` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:54:28.140725 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53b88ac2-9f91-4b42-90ee-cab13abe31da admin admin] RBAC: Authorizing `identity:delete_user(user_id=253a30f00d6f42d0af49f80acfeadcfa)` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 10:54:28.165493 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53b88ac2-9f91-4b42-90ee-cab13abe31da admin admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 10:54:28.178012 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 881/1733] 10.222.0.25 () {58 vars in 958 bytes} [Sun Aug 7 10:54:27 2022] POST /identity/v3/auth/tokens => generated 3301 bytes in 515 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 07 10:54:28.185651 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa25c519-7cb0-4c28-b7db-0f5d1c68ec58 None None] REQUEST_METHOD: `POST` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:54:28.185995 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa25c519-7cb0-4c28-b7db-0f5d1c68ec58 None None] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:54:28.186283 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa25c519-7cb0-4c28-b7db-0f5d1c68ec58 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:54:28.263544 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.auth.core [None req-aa25c519-7cb0-4c28-b7db-0f5d1c68ec58 None None] MFA Rules not processed for user `bac4e6b76fe64217ac8fa44e71cffe6a`. Rule list: `[]` (Enabled: `True`). {{(pid=80144) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 10:54:28.264791 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.fernet_utils [None req-aa25c519-7cb0-4c28-b7db-0f5d1c68ec58 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80144) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 10:54:28.296796 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 882/1734] 10.222.0.25 () {58 vars in 958 bytes} [Sun Aug 7 10:54:28 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 07 10:54:28.296796 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa25c519-7cb0-4c28-b7db-0f5d1c68ec58 None None] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 10:54:28.299509 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.fernet_utils [None req-aa25c519-7cb0-4c28-b7db-0f5d1c68ec58 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80144) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 10:54:28.348252 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-53b88ac2-9f91-4b42-90ee-cab13abe31da admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '253a30f00d6f42d0af49f80acfeadcfa'} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 10:54:28.375254 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-53b88ac2-9f91-4b42-90ee-cab13abe31da admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '253a30f00d6f42d0af49f80acfeadcfa'} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 10:54:28.457629 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-53b88ac2-9f91-4b42-90ee-cab13abe31da admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '253a30f00d6f42d0af49f80acfeadcfa'} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 10:54:28.460427 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-53b88ac2-9f91-4b42-90ee-cab13abe31da admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '253a30f00d6f42d0af49f80acfeadcfa'} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 10:54:28.484582 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-53b88ac2-9f91-4b42-90ee-cab13abe31da admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '253a30f00d6f42d0af49f80acfeadcfa'} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 10:54:28.507567 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 853/1735] 10.222.0.25 () {58 vars in 1224 bytes} [Sun Aug 7 10:54:27 2022] DELETE /identity/v3/users/253a30f00d6f42d0af49f80acfeadcfa => generated 0 bytes in 635 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 07 10:54:28.510684 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa25c519-7cb0-4c28-b7db-0f5d1c68ec58 None None] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 10:54:28.512829 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e303cbc-d81e-432e-9786-910040a14d46 None None] REQUEST_METHOD: `POST` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:54:28.514502 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e303cbc-d81e-432e-9786-910040a14d46 None None] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:54:28.516345 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e303cbc-d81e-432e-9786-910040a14d46 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:54:28.577889 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.auth.core [None req-8e303cbc-d81e-432e-9786-910040a14d46 None None] MFA Rules not processed for user `be1283c9bf47424c9aa3ad27cab03027`. Rule list: `[]` (Enabled: `True`). {{(pid=80143) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 10:54:28.578812 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-8e303cbc-d81e-432e-9786-910040a14d46 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 10:54:28.670912 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6b8ddc1-3eaa-4ee5-8db6-06bd1ac4c01c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88dbd6dfd1164994a3e16da553561910', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6de1f3cac02c4bcdb05dd2337176033d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 10:54:28.671561 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6b8ddc1-3eaa-4ee5-8db6-06bd1ac4c01c service cinder] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:54:28.672027 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6b8ddc1-3eaa-4ee5-8db6-06bd1ac4c01c service cinder] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:54:28.672339 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6b8ddc1-3eaa-4ee5-8db6-06bd1ac4c01c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:54:28.675373 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.fernet_utils [None req-d6b8ddc1-3eaa-4ee5-8db6-06bd1ac4c01c service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80144) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 10:54:28.788628 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 854/1736] 10.222.0.36 () {58 vars in 1015 bytes} [Sun Aug 7 10:54:28 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 278 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 07 10:54:28.789525 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e303cbc-d81e-432e-9786-910040a14d46 None None] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 10:54:28.791165 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-8e303cbc-d81e-432e-9786-910040a14d46 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 10:54:28.849012 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6b8ddc1-3eaa-4ee5-8db6-06bd1ac4c01c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 10:54:28.850583 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6b8ddc1-3eaa-4ee5-8db6-06bd1ac4c01c service cinder] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 10:54:28.868599 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e303cbc-d81e-432e-9786-910040a14d46 None None] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 10:54:28.883599 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 883/1737] 10.222.0.25 () {60 vars in 1365 bytes} [Sun Aug 7 10:54:28 2022] GET /identity/v3/auth/tokens => generated 3301 bytes in 589 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 07 10:54:28.884972 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d29ab3db-c6d0-4174-8ccc-40c53457894c None None] REQUEST_METHOD: `POST` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:54:28.885263 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d29ab3db-c6d0-4174-8ccc-40c53457894c None None] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:54:28.895316 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d29ab3db-c6d0-4174-8ccc-40c53457894c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:54:28.895453 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4afaba35-f51c-4cb5-8669-cc352cc8ee0f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be1283c9bf47424c9aa3ad27cab03027', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6de1f3cac02c4bcdb05dd2337176033d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 10:54:28.898041 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4afaba35-f51c-4cb5-8669-cc352cc8ee0f service neutron] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:54:28.898649 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4afaba35-f51c-4cb5-8669-cc352cc8ee0f service neutron] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:54:28.900182 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4afaba35-f51c-4cb5-8669-cc352cc8ee0f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:54:28.907028 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-4afaba35-f51c-4cb5-8669-cc352cc8ee0f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 10:54:28.927991 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=100 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 10:54:28.964057 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.auth.core [None req-d29ab3db-c6d0-4174-8ccc-40c53457894c None None] MFA Rules not processed for user `bac4e6b76fe64217ac8fa44e71cffe6a`. Rule list: `[]` (Enabled: `True`). {{(pid=80144) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 10:54:28.965303 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.fernet_utils [None req-d29ab3db-c6d0-4174-8ccc-40c53457894c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80144) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 10:54:29.202869 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4afaba35-f51c-4cb5-8669-cc352cc8ee0f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 10:54:29.205594 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 884/1738] 10.222.0.25 () {58 vars in 958 bytes} [Sun Aug 7 10:54:28 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 321 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 07 10:54:29.207042 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bda8f9a3-19ca-4104-b174-5f969d6e6e37 None None] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:54:29.207952 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4afaba35-f51c-4cb5-8669-cc352cc8ee0f service neutron] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 10:54:29.209985 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bda8f9a3-19ca-4104-b174-5f969d6e6e37 None None] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:54:29.210291 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bda8f9a3-19ca-4104-b174-5f969d6e6e37 None None] PATH_INFO: `/` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:54:29.215941 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 885/1739] 10.222.0.25 () {54 vars in 912 bytes} [Sun Aug 7 10:54:29 2022] GET /identity => generated 270 bytes in 9 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 07 10:54:29.218211 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bda8f9a3-19ca-4104-b174-5f969d6e6e37 None None] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 10:54:29.228328 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.fernet_utils [None req-bda8f9a3-19ca-4104-b174-5f969d6e6e37 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80144) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 10:54:29.234007 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 10:54:29.274670 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 855/1740] 10.222.0.25 () {60 vars in 1365 bytes} [Sun Aug 7 10:54:28 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 485 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 07 10:54:29.280159 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4afaba35-f51c-4cb5-8669-cc352cc8ee0f service neutron] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 10:54:29.285060 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bda8f9a3-19ca-4104-b174-5f969d6e6e37 None None] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 10:54:29.300821 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd7d9ec7-c23e-48cd-a59b-703d5a3678bc service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88dbd6dfd1164994a3e16da553561910', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6de1f3cac02c4bcdb05dd2337176033d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 10:54:29.301027 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd7d9ec7-c23e-48cd-a59b-703d5a3678bc service cinder] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:54:29.301328 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd7d9ec7-c23e-48cd-a59b-703d5a3678bc service cinder] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:54:29.301607 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd7d9ec7-c23e-48cd-a59b-703d5a3678bc service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:54:29.301726 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4afaba35-f51c-4cb5-8669-cc352cc8ee0f service neutron] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 10:54:29.305598 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.fernet_utils [None req-dd7d9ec7-c23e-48cd-a59b-703d5a3678bc service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80144) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 10:54:29.318263 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00d2532f-b976-4933-ba64-1905d993872d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be1283c9bf47424c9aa3ad27cab03027', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6de1f3cac02c4bcdb05dd2337176033d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 10:54:29.318901 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00d2532f-b976-4933-ba64-1905d993872d service neutron] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:54:29.319202 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00d2532f-b976-4933-ba64-1905d993872d service neutron] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:54:29.319477 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00d2532f-b976-4933-ba64-1905d993872d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:54:29.360406 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00d2532f-b976-4933-ba64-1905d993872d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 10:54:29.368157 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00d2532f-b976-4933-ba64-1905d993872d service neutron] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 10:54:29.408734 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 856/1741] 10.222.0.25 () {60 vars in 1365 bytes} [Sun Aug 7 10:54:29 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 133 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 07 10:54:29.410242 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3ecd356-56af-46df-abe5-5ddfcf7584ba None None] REQUEST_METHOD: `POST` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:54:29.410749 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3ecd356-56af-46df-abe5-5ddfcf7584ba None None] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:54:29.411195 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3ecd356-56af-46df-abe5-5ddfcf7584ba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:54:29.463172 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd7d9ec7-c23e-48cd-a59b-703d5a3678bc service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 10:54:29.464967 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd7d9ec7-c23e-48cd-a59b-703d5a3678bc service cinder] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 10:54:29.477797 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 886/1742] 10.222.0.25 () {60 vars in 1365 bytes} [Sun Aug 7 10:54:29 2022] GET /identity/v3/auth/tokens => generated 3232 bytes in 261 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 07 10:54:29.498307 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd7d9ec7-c23e-48cd-a59b-703d5a3678bc service cinder] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 10:54:29.498784 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.auth.core [None req-c3ecd356-56af-46df-abe5-5ddfcf7584ba None None] MFA Rules not processed for user `be1283c9bf47424c9aa3ad27cab03027`. Rule list: `[]` (Enabled: `True`). {{(pid=80143) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 10:54:29.500376 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-c3ecd356-56af-46df-abe5-5ddfcf7584ba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 10:54:29.543572 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 857/1743] 10.222.0.25 () {58 vars in 1009 bytes} [Sun Aug 7 10:54:29 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 134 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 07 10:54:29.544760 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd7d9ec7-c23e-48cd-a59b-703d5a3678bc service cinder] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 10:54:29.556304 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e41547c9-991a-4fc5-8532-2e08c52a0f40 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be1283c9bf47424c9aa3ad27cab03027', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6de1f3cac02c4bcdb05dd2337176033d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 10:54:29.557471 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e41547c9-991a-4fc5-8532-2e08c52a0f40 service neutron] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:54:29.558086 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e41547c9-991a-4fc5-8532-2e08c52a0f40 service neutron] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:54:29.558781 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e41547c9-991a-4fc5-8532-2e08c52a0f40 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:54:29.573531 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e41547c9-991a-4fc5-8532-2e08c52a0f40 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 10:54:29.578282 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e41547c9-991a-4fc5-8532-2e08c52a0f40 service neutron] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 10:54:29.616453 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 887/1744] 10.222.0.25 () {60 vars in 1365 bytes} [Sun Aug 7 10:54:29 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 118 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 07 10:54:30.060821 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81ad56ce-4a0b-4d60-93e9-7612dbaf0926 None None] REQUEST_METHOD: `POST` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:54:30.061419 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81ad56ce-4a0b-4d60-93e9-7612dbaf0926 None None] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:54:30.061925 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81ad56ce-4a0b-4d60-93e9-7612dbaf0926 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:54:30.109128 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e41547c9-991a-4fc5-8532-2e08c52a0f40 service neutron] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 10:54:30.142912 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e41547c9-991a-4fc5-8532-2e08c52a0f40 service neutron] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 10:54:30.173514 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bfcd4e7-cdca-47ec-8074-747351f15200 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be1283c9bf47424c9aa3ad27cab03027', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6de1f3cac02c4bcdb05dd2337176033d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 10:54:30.175272 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bfcd4e7-cdca-47ec-8074-747351f15200 service neutron] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:54:30.186906 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bfcd4e7-cdca-47ec-8074-747351f15200 service neutron] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:54:30.190872 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bfcd4e7-cdca-47ec-8074-747351f15200 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:54:30.197326 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.auth.core [None req-81ad56ce-4a0b-4d60-93e9-7612dbaf0926 None None] MFA Rules not processed for user `2b64b05230b940eba992a063ea7cd778`. Rule list: `[]` (Enabled: `True`). {{(pid=80143) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 10:54:30.206819 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-81ad56ce-4a0b-4d60-93e9-7612dbaf0926 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 10:54:30.217944 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bfcd4e7-cdca-47ec-8074-747351f15200 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 10:54:30.220274 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bfcd4e7-cdca-47ec-8074-747351f15200 service neutron] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 10:54:30.291653 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 888/1745] 10.222.0.25 () {60 vars in 1365 bytes} [Sun Aug 7 10:54:30 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 183 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 07 10:54:30.518507 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 858/1746] 10.222.0.25 () {58 vars in 958 bytes} [Sun Aug 7 10:54:30 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 459 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 07 10:54:31.077704 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bfcd4e7-cdca-47ec-8074-747351f15200 service neutron] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 10:54:31.080517 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.fernet_utils [None req-3bfcd4e7-cdca-47ec-8074-747351f15200 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80144) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 10:54:31.448905 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bfcd4e7-cdca-47ec-8074-747351f15200 service neutron] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 10:54:31.463991 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d5445a5-e174-44b1-9b97-a0f6d5e0c937 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31d16d1b96104b4eb690398432b9829e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6de1f3cac02c4bcdb05dd2337176033d', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'reader', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 10:54:31.464672 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d5445a5-e174-44b1-9b97-a0f6d5e0c937 service nova] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:54:31.464922 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d5445a5-e174-44b1-9b97-a0f6d5e0c937 service nova] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:54:31.465200 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d5445a5-e174-44b1-9b97-a0f6d5e0c937 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:54:31.488075 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d5445a5-e174-44b1-9b97-a0f6d5e0c937 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 10:54:31.489806 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d5445a5-e174-44b1-9b97-a0f6d5e0c937 service nova] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 10:54:31.510429 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 889/1747] 10.222.0.25 () {60 vars in 1365 bytes} [Sun Aug 7 10:54:31 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 433 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 07 10:54:31.853213 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-603b45fc-0a38-4ea0-bec2-f39135a8ba9a None None] REQUEST_METHOD: `POST` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:54:31.853728 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-603b45fc-0a38-4ea0-bec2-f39135a8ba9a None None] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:54:31.853900 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-603b45fc-0a38-4ea0-bec2-f39135a8ba9a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:54:31.901044 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.auth.core [None req-603b45fc-0a38-4ea0-bec2-f39135a8ba9a None None] MFA Rules not processed for user `2b757e3f47e44b9ba3645d12b20f7502`. Rule list: `[]` (Enabled: `True`). {{(pid=80143) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 10:54:31.902590 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-603b45fc-0a38-4ea0-bec2-f39135a8ba9a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 10:54:32.074384 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 859/1748] 10.222.0.25 () {58 vars in 958 bytes} [Sun Aug 7 10:54:31 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 216 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 07 10:54:32.507239 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d5445a5-e174-44b1-9b97-a0f6d5e0c937 service nova] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 10:54:32.509663 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.fernet_utils [None req-4d5445a5-e174-44b1-9b97-a0f6d5e0c937 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80144) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 10:54:32.527410 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=100 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 10:54:32.549892 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d5445a5-e174-44b1-9b97-a0f6d5e0c937 service nova] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 10:54:32.588632 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8c22340-7bcc-4ca6-ad8d-4847532bd392 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5ac47e3b09f43d19a23a0f457fe9e23', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 10:54:32.590057 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8c22340-7bcc-4ca6-ad8d-4847532bd392 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:54:32.590479 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8c22340-7bcc-4ca6-ad8d-4847532bd392 admin admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:54:32.590818 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8c22340-7bcc-4ca6-ad8d-4847532bd392 admin admin] PATH_INFO: `/v3/projects/e827ce1c53e1400c9a5af4924657100a` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:54:32.595513 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8c22340-7bcc-4ca6-ad8d-4847532bd392 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e827ce1c53e1400c9a5af4924657100a)` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 10:54:32.600339 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: WARNING py.warnings [None req-d8c22340-7bcc-4ca6-ad8d-4847532bd392 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 07 10:54:32.600339 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: warnings.warn(msg) Aug 07 10:54:32.600339 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: Aug 07 10:54:32.605565 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8c22340-7bcc-4ca6-ad8d-4847532bd392 admin admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 10:54:32.609807 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-603b45fc-0a38-4ea0-bec2-f39135a8ba9a None None] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 10:54:32.612458 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-603b45fc-0a38-4ea0-bec2-f39135a8ba9a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 10:54:32.664589 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-603b45fc-0a38-4ea0-bec2-f39135a8ba9a None None] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 10:54:32.691352 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aa230e5-b9fe-434f-a76b-a6bdd859bde6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5ac47e3b09f43d19a23a0f457fe9e23', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 10:54:32.694053 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aa230e5-b9fe-434f-a76b-a6bdd859bde6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:54:32.695006 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aa230e5-b9fe-434f-a76b-a6bdd859bde6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:54:32.698451 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aa230e5-b9fe-434f-a76b-a6bdd859bde6 admin admin] PATH_INFO: `/v3/projects/e9bdee4b7e6e4f3fb532ed213bcd5eeb` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:54:32.706006 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2aa230e5-b9fe-434f-a76b-a6bdd859bde6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e9bdee4b7e6e4f3fb532ed213bcd5eeb)` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 10:54:32.708958 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2aa230e5-b9fe-434f-a76b-a6bdd859bde6 admin admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 10:54:32.886625 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.notifications [None req-d8c22340-7bcc-4ca6-ad8d-4847532bd392 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e827ce1c53e1400c9a5af4924657100a', 'request_id': 'req-d8c22340-7bcc-4ca6-ad8d-4847532bd392'} {{(pid=80144) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 10:54:32.913748 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-2aa230e5-b9fe-434f-a76b-a6bdd859bde6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e9bdee4b7e6e4f3fb532ed213bcd5eeb', 'request_id': 'req-2aa230e5-b9fe-434f-a76b-a6bdd859bde6'} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 10:54:32.917152 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.notifications [None req-d8c22340-7bcc-4ca6-ad8d-4847532bd392 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e827ce1c53e1400c9a5af4924657100a', 'request_id': 'req-d8c22340-7bcc-4ca6-ad8d-4847532bd392'} {{(pid=80144) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 10:54:32.953661 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-2aa230e5-b9fe-434f-a76b-a6bdd859bde6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e9bdee4b7e6e4f3fb532ed213bcd5eeb', 'request_id': 'req-2aa230e5-b9fe-434f-a76b-a6bdd859bde6'} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 10:54:32.974860 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.notifications [None req-d8c22340-7bcc-4ca6-ad8d-4847532bd392 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e827ce1c53e1400c9a5af4924657100a', 'request_id': 'req-d8c22340-7bcc-4ca6-ad8d-4847532bd392'} {{(pid=80144) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 10:54:32.989403 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-2aa230e5-b9fe-434f-a76b-a6bdd859bde6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e9bdee4b7e6e4f3fb532ed213bcd5eeb', 'request_id': 'req-2aa230e5-b9fe-434f-a76b-a6bdd859bde6'} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 10:54:33.016975 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 10:54:33.033658 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-2aa230e5-b9fe-434f-a76b-a6bdd859bde6 admin admin] The token cache is being invalidate because project e9bdee4b7e6e4f3fb532ed213bcd5eeb was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80143) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 07 10:54:33.037261 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-2aa230e5-b9fe-434f-a76b-a6bdd859bde6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 10:54:33.040325 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-2aa230e5-b9fe-434f-a76b-a6bdd859bde6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 10:54:33.044847 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 860/1749] 10.222.0.25 () {58 vars in 1233 bytes} [Sun Aug 7 10:54:32 2022] DELETE /identity/v3/projects/e9bdee4b7e6e4f3fb532ed213bcd5eeb => generated 0 bytes in 436 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 07 10:54:33.049027 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.notifications [None req-d8c22340-7bcc-4ca6-ad8d-4847532bd392 admin admin] The token cache is being invalidate because project e827ce1c53e1400c9a5af4924657100a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80144) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 07 10:54:33.049339 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.notifications [None req-d8c22340-7bcc-4ca6-ad8d-4847532bd392 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80144) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 10:54:33.050202 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aa230e5-b9fe-434f-a76b-a6bdd859bde6 admin admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 10:54:33.053735 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.notifications [None req-d8c22340-7bcc-4ca6-ad8d-4847532bd392 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80144) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 10:54:33.053901 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-2aa230e5-b9fe-434f-a76b-a6bdd859bde6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 10:54:33.060833 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 890/1750] 10.222.0.25 () {58 vars in 1233 bytes} [Sun Aug 7 10:54:32 2022] DELETE /identity/v3/projects/e827ce1c53e1400c9a5af4924657100a => generated 0 bytes in 555 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 07 10:54:33.065877 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8c22340-7bcc-4ca6-ad8d-4847532bd392 admin admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 10:54:33.068657 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.fernet_utils [None req-d8c22340-7bcc-4ca6-ad8d-4847532bd392 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80144) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 10:54:33.323665 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aa230e5-b9fe-434f-a76b-a6bdd859bde6 admin admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 10:54:33.339832 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99ee361f-ff60-425e-a61c-2c1e2725e3cf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5ac47e3b09f43d19a23a0f457fe9e23', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 10:54:33.341090 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99ee361f-ff60-425e-a61c-2c1e2725e3cf admin admin] REQUEST_METHOD: `DELETE` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:54:33.341449 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99ee361f-ff60-425e-a61c-2c1e2725e3cf admin admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:54:33.341741 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99ee361f-ff60-425e-a61c-2c1e2725e3cf admin admin] PATH_INFO: `/v3/projects/639f69c020004f59b20d086d5f4149e5` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:54:33.343174 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99ee361f-ff60-425e-a61c-2c1e2725e3cf admin admin] RBAC: Authorizing `identity:delete_project(project_id=639f69c020004f59b20d086d5f4149e5)` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 10:54:33.345436 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99ee361f-ff60-425e-a61c-2c1e2725e3cf admin admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 10:54:33.468508 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8c22340-7bcc-4ca6-ad8d-4847532bd392 admin admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 10:54:33.476800 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-99ee361f-ff60-425e-a61c-2c1e2725e3cf admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '639f69c020004f59b20d086d5f4149e5', 'request_id': 'req-99ee361f-ff60-425e-a61c-2c1e2725e3cf'} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 10:54:33.494397 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-99ee361f-ff60-425e-a61c-2c1e2725e3cf admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '639f69c020004f59b20d086d5f4149e5', 'request_id': 'req-99ee361f-ff60-425e-a61c-2c1e2725e3cf'} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 10:54:33.529808 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-99ee361f-ff60-425e-a61c-2c1e2725e3cf admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '639f69c020004f59b20d086d5f4149e5', 'request_id': 'req-99ee361f-ff60-425e-a61c-2c1e2725e3cf'} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 10:54:33.607736 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-99ee361f-ff60-425e-a61c-2c1e2725e3cf admin admin] The token cache is being invalidate because project 639f69c020004f59b20d086d5f4149e5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80143) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 07 10:54:33.608281 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-99ee361f-ff60-425e-a61c-2c1e2725e3cf admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 10:54:33.623357 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-99ee361f-ff60-425e-a61c-2c1e2725e3cf admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 10:54:33.625891 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 861/1751] 10.222.0.25 () {58 vars in 1233 bytes} [Sun Aug 7 10:54:33 2022] DELETE /identity/v3/projects/639f69c020004f59b20d086d5f4149e5 => generated 0 bytes in 578 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 07 10:54:33.627006 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99ee361f-ff60-425e-a61c-2c1e2725e3cf admin admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 10:54:33.632361 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-99ee361f-ff60-425e-a61c-2c1e2725e3cf admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 10:54:33.814760 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63591590-9473-4f1f-b8ce-8d99ebcdb90b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5ac47e3b09f43d19a23a0f457fe9e23', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 10:54:33.819512 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63591590-9473-4f1f-b8ce-8d99ebcdb90b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:54:33.821256 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63591590-9473-4f1f-b8ce-8d99ebcdb90b admin admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:54:33.822182 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63591590-9473-4f1f-b8ce-8d99ebcdb90b admin admin] PATH_INFO: `/v3/users/8b3b419b5b824f5087c9c8121adafc0a` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:54:33.840080 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63591590-9473-4f1f-b8ce-8d99ebcdb90b admin admin] RBAC: Authorizing `identity:delete_user(user_id=8b3b419b5b824f5087c9c8121adafc0a)` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 10:54:33.857997 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63591590-9473-4f1f-b8ce-8d99ebcdb90b admin admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 10:54:33.875496 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99ee361f-ff60-425e-a61c-2c1e2725e3cf admin admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 10:54:33.890176 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18234710-f07f-44b0-a7bc-1f7803d734d3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5ac47e3b09f43d19a23a0f457fe9e23', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 10:54:33.891672 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18234710-f07f-44b0-a7bc-1f7803d734d3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:54:33.892576 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18234710-f07f-44b0-a7bc-1f7803d734d3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:54:33.893079 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18234710-f07f-44b0-a7bc-1f7803d734d3 admin admin] PATH_INFO: `/v3/users/fe6827bf0a3e4dad816e42f9bb9e0657` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:54:33.898454 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18234710-f07f-44b0-a7bc-1f7803d734d3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=fe6827bf0a3e4dad816e42f9bb9e0657)` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 10:54:33.901096 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18234710-f07f-44b0-a7bc-1f7803d734d3 admin admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 10:54:33.987533 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 10:54:34.002848 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 10:54:34.041122 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.notifications [None req-63591590-9473-4f1f-b8ce-8d99ebcdb90b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8b3b419b5b824f5087c9c8121adafc0a'} {{(pid=80144) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 10:54:34.048535 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-18234710-f07f-44b0-a7bc-1f7803d734d3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'fe6827bf0a3e4dad816e42f9bb9e0657'} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 10:54:34.054417 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.notifications [None req-63591590-9473-4f1f-b8ce-8d99ebcdb90b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8b3b419b5b824f5087c9c8121adafc0a'} {{(pid=80144) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 10:54:34.074815 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-18234710-f07f-44b0-a7bc-1f7803d734d3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'fe6827bf0a3e4dad816e42f9bb9e0657'} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 10:54:34.075782 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.notifications [None req-63591590-9473-4f1f-b8ce-8d99ebcdb90b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8b3b419b5b824f5087c9c8121adafc0a'} {{(pid=80144) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 10:54:34.077949 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.notifications [None req-63591590-9473-4f1f-b8ce-8d99ebcdb90b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8b3b419b5b824f5087c9c8121adafc0a'} {{(pid=80144) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 10:54:34.078883 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.notifications [None req-63591590-9473-4f1f-b8ce-8d99ebcdb90b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8b3b419b5b824f5087c9c8121adafc0a'} {{(pid=80144) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 10:54:34.113027 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-18234710-f07f-44b0-a7bc-1f7803d734d3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'fe6827bf0a3e4dad816e42f9bb9e0657'} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 10:54:34.114890 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-18234710-f07f-44b0-a7bc-1f7803d734d3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'fe6827bf0a3e4dad816e42f9bb9e0657'} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 10:54:34.128708 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 891/1752] 10.222.0.25 () {58 vars in 1224 bytes} [Sun Aug 7 10:54:33 2022] DELETE /identity/v3/users/8b3b419b5b824f5087c9c8121adafc0a => generated 0 bytes in 1066 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 07 10:54:34.130553 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00a5f3a6-b6fd-4b0c-b4bb-0164c7e4cc2c None None] REQUEST_METHOD: `POST` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:54:34.131581 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00a5f3a6-b6fd-4b0c-b4bb-0164c7e4cc2c None None] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:54:34.131581 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00a5f3a6-b6fd-4b0c-b4bb-0164c7e4cc2c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:54:34.135357 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-18234710-f07f-44b0-a7bc-1f7803d734d3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'fe6827bf0a3e4dad816e42f9bb9e0657'} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 10:54:34.160394 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 862/1753] 10.222.0.25 () {58 vars in 1224 bytes} [Sun Aug 7 10:54:33 2022] DELETE /identity/v3/users/fe6827bf0a3e4dad816e42f9bb9e0657 => generated 0 bytes in 530 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 07 10:54:34.164431 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba83f91f-071b-4b03-b7c0-91cff153549d None None] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:54:34.164859 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba83f91f-071b-4b03-b7c0-91cff153549d None None] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:54:34.165736 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba83f91f-071b-4b03-b7c0-91cff153549d None None] PATH_INFO: `/` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:54:34.169769 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 863/1754] 10.222.0.25 () {54 vars in 921 bytes} [Sun Aug 7 10:54:34 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 07 10:54:34.191750 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.auth.core [None req-00a5f3a6-b6fd-4b0c-b4bb-0164c7e4cc2c None None] MFA Rules not processed for user `bac4e6b76fe64217ac8fa44e71cffe6a`. Rule list: `[]` (Enabled: `True`). {{(pid=80144) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 10:54:34.195769 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.fernet_utils [None req-00a5f3a6-b6fd-4b0c-b4bb-0164c7e4cc2c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80144) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 10:54:34.279870 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edb8b8c8-e6bc-4303-8b75-1f0ebea73cbe None None] REQUEST_METHOD: `POST` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:54:34.280152 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edb8b8c8-e6bc-4303-8b75-1f0ebea73cbe None None] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:54:34.280607 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edb8b8c8-e6bc-4303-8b75-1f0ebea73cbe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:54:34.340246 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.auth.core [None req-edb8b8c8-e6bc-4303-8b75-1f0ebea73cbe None None] MFA Rules not processed for user `31d16d1b96104b4eb690398432b9829e`. Rule list: `[]` (Enabled: `True`). {{(pid=80143) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 10:54:34.346660 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-edb8b8c8-e6bc-4303-8b75-1f0ebea73cbe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 10:54:34.442148 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 892/1755] 10.222.0.25 () {58 vars in 958 bytes} [Sun Aug 7 10:54:34 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 313 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 07 10:54:34.450772 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9f5b3f0-2353-41b9-80da-c79a4cb99c74 None None] REQUEST_METHOD: `POST` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:54:34.451298 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9f5b3f0-2353-41b9-80da-c79a4cb99c74 None None] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:54:34.452094 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9f5b3f0-2353-41b9-80da-c79a4cb99c74 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:54:34.512543 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.auth.core [None req-f9f5b3f0-2353-41b9-80da-c79a4cb99c74 None None] MFA Rules not processed for user `bac4e6b76fe64217ac8fa44e71cffe6a`. Rule list: `[]` (Enabled: `True`). {{(pid=80144) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 10:54:34.514091 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.fernet_utils [None req-f9f5b3f0-2353-41b9-80da-c79a4cb99c74 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80144) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 10:54:34.532888 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 893/1756] 10.222.0.25 () {58 vars in 958 bytes} [Sun Aug 7 10:54:34 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 07 10:54:34.537357 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9f5b3f0-2353-41b9-80da-c79a4cb99c74 None None] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 10:54:34.573463 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9f5b3f0-2353-41b9-80da-c79a4cb99c74 None None] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 10:54:34.587754 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-407f7c19-7ac6-43dd-955c-3df7d862e78a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5ac47e3b09f43d19a23a0f457fe9e23', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 10:54:34.589937 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-407f7c19-7ac6-43dd-955c-3df7d862e78a admin admin] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:54:34.594215 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-407f7c19-7ac6-43dd-955c-3df7d862e78a admin admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:54:34.595300 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-407f7c19-7ac6-43dd-955c-3df7d862e78a admin admin] PATH_INFO: `/v3/domains` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:54:34.599012 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-407f7c19-7ac6-43dd-955c-3df7d862e78a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80144) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 10:54:34.600024 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-407f7c19-7ac6-43dd-955c-3df7d862e78a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 10:54:34.604738 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-407f7c19-7ac6-43dd-955c-3df7d862e78a admin admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 10:54:34.622748 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 864/1757] 10.222.0.25 () {58 vars in 1018 bytes} [Sun Aug 7 10:54:34 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 344 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 07 10:54:34.653262 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 894/1758] 10.222.0.25 () {58 vars in 1153 bytes} [Sun Aug 7 10:54:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 117 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 07 10:54:34.658546 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edb8b8c8-e6bc-4303-8b75-1f0ebea73cbe None None] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 10:54:34.670660 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edb8b8c8-e6bc-4303-8b75-1f0ebea73cbe None None] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 10:54:34.681224 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-accc45c0-ce84-4309-82d9-b549f91fdc10 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5ac47e3b09f43d19a23a0f457fe9e23', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 10:54:34.682081 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-accc45c0-ce84-4309-82d9-b549f91fdc10 admin admin] REQUEST_METHOD: `POST` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:54:34.682437 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-accc45c0-ce84-4309-82d9-b549f91fdc10 admin admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:54:34.682861 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-accc45c0-ce84-4309-82d9-b549f91fdc10 admin admin] PATH_INFO: `/v3/projects` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:54:34.684028 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-accc45c0-ce84-4309-82d9-b549f91fdc10 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 10:54:34.685649 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-accc45c0-ce84-4309-82d9-b549f91fdc10 admin admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 10:54:34.748336 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 865/1759] 10.222.0.25 () {60 vars in 1153 bytes} [Sun Aug 7 10:54:34 2022] POST /identity/v3/projects => generated 403 bytes in 91 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 07 10:54:34.751980 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-407f7c19-7ac6-43dd-955c-3df7d862e78a admin admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 10:54:34.768663 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-407f7c19-7ac6-43dd-955c-3df7d862e78a admin admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 10:54:34.916867 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-739ad8f8-b9db-4613-8f7d-5e8b866c9971 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5ac47e3b09f43d19a23a0f457fe9e23', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 10:54:34.918106 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-739ad8f8-b9db-4613-8f7d-5e8b866c9971 admin admin] REQUEST_METHOD: `POST` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:54:34.918393 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-739ad8f8-b9db-4613-8f7d-5e8b866c9971 admin admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:54:34.918665 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-739ad8f8-b9db-4613-8f7d-5e8b866c9971 admin admin] PATH_INFO: `/v3/users` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:54:34.919342 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-739ad8f8-b9db-4613-8f7d-5e8b866c9971 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 10:54:34.921260 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-739ad8f8-b9db-4613-8f7d-5e8b866c9971 admin admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 10:54:34.968941 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-accc45c0-ce84-4309-82d9-b549f91fdc10 admin admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 10:54:34.975422 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 895/1760] 10.222.0.25 () {60 vars in 1144 bytes} [Sun Aug 7 10:54:34 2022] POST /identity/v3/users => generated 302 bytes in 224 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 07 10:54:34.981173 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-accc45c0-ce84-4309-82d9-b549f91fdc10 admin admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 10:54:34.996619 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f2825f8-de3e-4633-87b7-5052f11014dc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5ac47e3b09f43d19a23a0f457fe9e23', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 10:54:34.997948 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f2825f8-de3e-4633-87b7-5052f11014dc admin admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:54:34.998402 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f2825f8-de3e-4633-87b7-5052f11014dc admin admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:54:34.998858 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f2825f8-de3e-4633-87b7-5052f11014dc admin admin] PATH_INFO: `/v3/roles` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:54:34.999721 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f2825f8-de3e-4633-87b7-5052f11014dc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 10:54:35.002126 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f2825f8-de3e-4633-87b7-5052f11014dc admin admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 10:54:35.032567 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 866/1761] 10.222.0.25 () {58 vars in 1122 bytes} [Sun Aug 7 10:54:34 2022] GET /identity/v3/roles => generated 1420 bytes in 64 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 07 10:54:35.035387 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-739ad8f8-b9db-4613-8f7d-5e8b866c9971 admin admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 10:54:35.078534 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-739ad8f8-b9db-4613-8f7d-5e8b866c9971 admin admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 10:54:35.106849 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ee15960-8d85-43ee-b8f8-759b4089ef99 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5ac47e3b09f43d19a23a0f457fe9e23', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 10:54:35.113388 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ee15960-8d85-43ee-b8f8-759b4089ef99 admin admin] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:54:35.114773 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ee15960-8d85-43ee-b8f8-759b4089ef99 admin admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:54:35.115755 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ee15960-8d85-43ee-b8f8-759b4089ef99 admin admin] PATH_INFO: `/v3/roles` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:54:35.121423 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ee15960-8d85-43ee-b8f8-759b4089ef99 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 10:54:35.128459 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ee15960-8d85-43ee-b8f8-759b4089ef99 admin admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 10:54:35.167342 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 896/1762] 10.222.0.25 () {58 vars in 1122 bytes} [Sun Aug 7 10:54:35 2022] GET /identity/v3/roles => generated 1420 bytes in 134 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 07 10:54:35.170588 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f2825f8-de3e-4633-87b7-5052f11014dc admin admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 10:54:35.184363 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f2825f8-de3e-4633-87b7-5052f11014dc admin admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 10:54:35.232724 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d672f880-3b44-4c66-836d-af1c912305a0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5ac47e3b09f43d19a23a0f457fe9e23', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 10:54:35.233687 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d672f880-3b44-4c66-836d-af1c912305a0 admin admin] REQUEST_METHOD: `PUT` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:54:35.233967 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d672f880-3b44-4c66-836d-af1c912305a0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:54:35.234253 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d672f880-3b44-4c66-836d-af1c912305a0 admin admin] PATH_INFO: `/v3/projects/40437563bdb1443589268c67cc9e6c11/users/8fb9d5b3e8524ad09786d0eeedfbd643/roles/b98dfb3b9fee4644af2887a3ac3fd2e6` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:54:35.325755 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d672f880-3b44-4c66-836d-af1c912305a0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=40437563bdb1443589268c67cc9e6c11, user_id=8fb9d5b3e8524ad09786d0eeedfbd643, role_id=b98dfb3b9fee4644af2887a3ac3fd2e6)` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 10:54:35.338610 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d672f880-3b44-4c66-836d-af1c912305a0 admin admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 10:54:35.439385 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 867/1763] 10.222.0.25 () {60 vars in 1483 bytes} [Sun Aug 7 10:54:35 2022] PUT /identity/v3/projects/40437563bdb1443589268c67cc9e6c11/users/8fb9d5b3e8524ad09786d0eeedfbd643/roles/b98dfb3b9fee4644af2887a3ac3fd2e6 => generated 0 bytes in 269 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 07 10:54:35.444634 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fedf338-c4a0-4ef4-abbd-68ab6a831bcd None None] REQUEST_METHOD: `POST` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:54:35.445309 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fedf338-c4a0-4ef4-abbd-68ab6a831bcd None None] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:54:35.446375 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fedf338-c4a0-4ef4-abbd-68ab6a831bcd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:54:35.505707 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.auth.core [None req-9fedf338-c4a0-4ef4-abbd-68ab6a831bcd None None] MFA Rules not processed for user `bac4e6b76fe64217ac8fa44e71cffe6a`. Rule list: `[]` (Enabled: `True`). {{(pid=80144) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 10:54:35.507365 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.fernet_utils [None req-9fedf338-c4a0-4ef4-abbd-68ab6a831bcd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80144) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 10:54:35.530951 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=91,keystone:DELETE=5,keystone:INSERT=4 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 10:54:35.667171 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 897/1764] 10.222.0.25 () {58 vars in 958 bytes} [Sun Aug 7 10:54:35 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 224 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 07 10:54:35.905791 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d672f880-3b44-4c66-836d-af1c912305a0 admin admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 10:54:35.908854 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-d672f880-3b44-4c66-836d-af1c912305a0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 10:54:36.019394 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fedf338-c4a0-4ef4-abbd-68ab6a831bcd None None] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 10:54:36.020916 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.fernet_utils [None req-9fedf338-c4a0-4ef4-abbd-68ab6a831bcd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80144) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 10:54:36.052993 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fedf338-c4a0-4ef4-abbd-68ab6a831bcd None None] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 10:54:36.061621 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=92,keystone:DELETE=5,keystone:INSERT=3 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 10:54:36.062160 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7baa591-00fb-47bb-a91e-f69d2bdf0d56 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5ac47e3b09f43d19a23a0f457fe9e23', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 10:54:36.066411 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7baa591-00fb-47bb-a91e-f69d2bdf0d56 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:54:36.069444 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7baa591-00fb-47bb-a91e-f69d2bdf0d56 admin admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:54:36.070083 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7baa591-00fb-47bb-a91e-f69d2bdf0d56 admin admin] PATH_INFO: `/v3/projects/2a2a9ab9e45c49ab80630b837cc08639` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:54:36.079518 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7baa591-00fb-47bb-a91e-f69d2bdf0d56 admin admin] RBAC: Authorizing `identity:delete_project(project_id=2a2a9ab9e45c49ab80630b837cc08639)` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 10:54:36.089390 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7baa591-00fb-47bb-a91e-f69d2bdf0d56 admin admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 10:54:36.119087 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d672f880-3b44-4c66-836d-af1c912305a0 admin admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 10:54:36.140185 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-483ca0eb-e4e2-478e-8ba6-7ee45ebdfa9c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be1283c9bf47424c9aa3ad27cab03027', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6de1f3cac02c4bcdb05dd2337176033d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 10:54:36.142343 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-483ca0eb-e4e2-478e-8ba6-7ee45ebdfa9c service neutron] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:54:36.143865 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-483ca0eb-e4e2-478e-8ba6-7ee45ebdfa9c service neutron] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:54:36.144222 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-483ca0eb-e4e2-478e-8ba6-7ee45ebdfa9c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:54:36.204732 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-483ca0eb-e4e2-478e-8ba6-7ee45ebdfa9c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 10:54:36.211400 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-483ca0eb-e4e2-478e-8ba6-7ee45ebdfa9c service neutron] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 10:54:36.282047 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 868/1765] 10.222.0.25 () {60 vars in 1365 bytes} [Sun Aug 7 10:54:35 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 377 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 07 10:54:36.306701 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-483ca0eb-e4e2-478e-8ba6-7ee45ebdfa9c service neutron] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 10:54:36.312098 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-483ca0eb-e4e2-478e-8ba6-7ee45ebdfa9c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 10:54:36.371613 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.notifications [None req-d7baa591-00fb-47bb-a91e-f69d2bdf0d56 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2a2a9ab9e45c49ab80630b837cc08639', 'request_id': 'req-d7baa591-00fb-47bb-a91e-f69d2bdf0d56'} {{(pid=80144) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 10:54:36.378511 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-483ca0eb-e4e2-478e-8ba6-7ee45ebdfa9c service neutron] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 10:54:36.397260 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.notifications [None req-d7baa591-00fb-47bb-a91e-f69d2bdf0d56 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2a2a9ab9e45c49ab80630b837cc08639', 'request_id': 'req-d7baa591-00fb-47bb-a91e-f69d2bdf0d56'} {{(pid=80144) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 10:54:36.432929 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.notifications [None req-d7baa591-00fb-47bb-a91e-f69d2bdf0d56 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2a2a9ab9e45c49ab80630b837cc08639', 'request_id': 'req-d7baa591-00fb-47bb-a91e-f69d2bdf0d56'} {{(pid=80144) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 10:54:36.469447 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.notifications [None req-d7baa591-00fb-47bb-a91e-f69d2bdf0d56 admin admin] The token cache is being invalidate because project 2a2a9ab9e45c49ab80630b837cc08639 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80144) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 07 10:54:36.469860 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.notifications [None req-d7baa591-00fb-47bb-a91e-f69d2bdf0d56 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80144) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 10:54:36.471926 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.notifications [None req-d7baa591-00fb-47bb-a91e-f69d2bdf0d56 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80144) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 10:54:36.474418 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 898/1766] 10.222.0.25 () {58 vars in 1233 bytes} [Sun Aug 7 10:54:36 2022] DELETE /identity/v3/projects/2a2a9ab9e45c49ab80630b837cc08639 => generated 0 bytes in 456 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 07 10:54:36.475372 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7baa591-00fb-47bb-a91e-f69d2bdf0d56 admin admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 10:54:36.501082 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.fernet_utils [None req-d7baa591-00fb-47bb-a91e-f69d2bdf0d56 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80144) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 10:54:36.639241 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09b3eb38-8e98-4c78-995f-edf51aa1a8eb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5ac47e3b09f43d19a23a0f457fe9e23', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 10:54:36.642496 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09b3eb38-8e98-4c78-995f-edf51aa1a8eb admin admin] REQUEST_METHOD: `DELETE` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:54:36.644844 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09b3eb38-8e98-4c78-995f-edf51aa1a8eb admin admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:54:36.645310 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09b3eb38-8e98-4c78-995f-edf51aa1a8eb admin admin] PATH_INFO: `/v3/projects/5a16513a1528460cb138b3d11c58e815` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:54:36.653556 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09b3eb38-8e98-4c78-995f-edf51aa1a8eb admin admin] RBAC: Authorizing `identity:delete_project(project_id=5a16513a1528460cb138b3d11c58e815)` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 10:54:36.657023 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09b3eb38-8e98-4c78-995f-edf51aa1a8eb admin admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 10:54:36.760805 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7baa591-00fb-47bb-a91e-f69d2bdf0d56 admin admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 10:54:36.810654 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-09b3eb38-8e98-4c78-995f-edf51aa1a8eb admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5a16513a1528460cb138b3d11c58e815', 'request_id': 'req-09b3eb38-8e98-4c78-995f-edf51aa1a8eb'} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 10:54:36.827819 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-09b3eb38-8e98-4c78-995f-edf51aa1a8eb admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5a16513a1528460cb138b3d11c58e815', 'request_id': 'req-09b3eb38-8e98-4c78-995f-edf51aa1a8eb'} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 10:54:36.837752 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-09b3eb38-8e98-4c78-995f-edf51aa1a8eb admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5a16513a1528460cb138b3d11c58e815', 'request_id': 'req-09b3eb38-8e98-4c78-995f-edf51aa1a8eb'} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 10:54:36.866850 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-09b3eb38-8e98-4c78-995f-edf51aa1a8eb admin admin] The token cache is being invalidate because project 5a16513a1528460cb138b3d11c58e815 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80143) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 07 10:54:36.868624 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 10:54:36.868747 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-09b3eb38-8e98-4c78-995f-edf51aa1a8eb admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 10:54:36.869468 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-09b3eb38-8e98-4c78-995f-edf51aa1a8eb admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 10:54:36.881264 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 869/1767] 10.222.0.25 () {58 vars in 1233 bytes} [Sun Aug 7 10:54:36 2022] DELETE /identity/v3/projects/5a16513a1528460cb138b3d11c58e815 => generated 0 bytes in 593 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 07 10:54:36.881264 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e529dc68-5e29-49ea-b943-b6a509377809 None None] REQUEST_METHOD: `POST` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:54:36.881264 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e529dc68-5e29-49ea-b943-b6a509377809 None None] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:54:36.881264 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e529dc68-5e29-49ea-b943-b6a509377809 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:54:36.936255 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.auth.core [None req-e529dc68-5e29-49ea-b943-b6a509377809 None None] MFA Rules not processed for user `bac4e6b76fe64217ac8fa44e71cffe6a`. Rule list: `[]` (Enabled: `True`). {{(pid=80143) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 10:54:36.937930 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-e529dc68-5e29-49ea-b943-b6a509377809 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 10:54:36.976645 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaa8fd28-e675-45a3-9c08-4f85d1eef827 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31d16d1b96104b4eb690398432b9829e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6de1f3cac02c4bcdb05dd2337176033d', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'reader', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 10:54:36.978225 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaa8fd28-e675-45a3-9c08-4f85d1eef827 service nova] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:54:36.978698 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaa8fd28-e675-45a3-9c08-4f85d1eef827 service nova] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:54:36.979463 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaa8fd28-e675-45a3-9c08-4f85d1eef827 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:54:36.982046 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.fernet_utils [None req-eaa8fd28-e675-45a3-9c08-4f85d1eef827 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80144) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 10:54:37.002538 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaa8fd28-e675-45a3-9c08-4f85d1eef827 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 10:54:37.005071 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaa8fd28-e675-45a3-9c08-4f85d1eef827 service nova] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 10:54:37.022662 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 899/1768] 10.222.0.25 () {60 vars in 1365 bytes} [Sun Aug 7 10:54:36 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 547 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 07 10:54:37.026465 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2a8595c-a852-43e6-951b-84106eb27e8f None None] REQUEST_METHOD: `POST` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:54:37.028170 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2a8595c-a852-43e6-951b-84106eb27e8f None None] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:54:37.029071 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2a8595c-a852-43e6-951b-84106eb27e8f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:54:37.062553 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.auth.core [None req-e2a8595c-a852-43e6-951b-84106eb27e8f None None] MFA Rules not processed for user `bac4e6b76fe64217ac8fa44e71cffe6a`. Rule list: `[]` (Enabled: `True`). {{(pid=80144) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 10:54:37.083747 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.fernet_utils [None req-e2a8595c-a852-43e6-951b-84106eb27e8f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80144) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 10:54:37.130550 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 870/1769] 10.222.0.25 () {58 vars in 958 bytes} [Sun Aug 7 10:54:36 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 253 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 07 10:54:37.140436 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-223cd7a8-b378-4ec3-87ca-77f056664909 None None] REQUEST_METHOD: `POST` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:54:37.140745 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-223cd7a8-b378-4ec3-87ca-77f056664909 None None] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:54:37.141028 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-223cd7a8-b378-4ec3-87ca-77f056664909 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:54:37.200760 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.auth.core [None req-223cd7a8-b378-4ec3-87ca-77f056664909 None None] MFA Rules not processed for user `bac4e6b76fe64217ac8fa44e71cffe6a`. Rule list: `[]` (Enabled: `True`). {{(pid=80143) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 10:54:37.202014 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-223cd7a8-b378-4ec3-87ca-77f056664909 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 10:54:37.273127 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 871/1770] 10.222.0.25 () {58 vars in 958 bytes} [Sun Aug 7 10:54:37 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 134 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 07 10:54:37.279354 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-223cd7a8-b378-4ec3-87ca-77f056664909 None None] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 10:54:37.296848 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 900/1771] 10.222.0.25 () {58 vars in 958 bytes} [Sun Aug 7 10:54:37 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 273 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 07 10:54:37.308730 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-869a3391-356b-4b48-960c-e080b08a0e4e None None] REQUEST_METHOD: `POST` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:54:37.309075 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-869a3391-356b-4b48-960c-e080b08a0e4e None None] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:54:37.309283 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-869a3391-356b-4b48-960c-e080b08a0e4e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:54:37.318884 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-223cd7a8-b378-4ec3-87ca-77f056664909 None None] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 10:54:37.325216 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66f60a2f-6ecc-41f7-8d05-9ec3847a898e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5ac47e3b09f43d19a23a0f457fe9e23', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 10:54:37.326196 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66f60a2f-6ecc-41f7-8d05-9ec3847a898e admin admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:54:37.326704 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66f60a2f-6ecc-41f7-8d05-9ec3847a898e admin admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:54:37.327233 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66f60a2f-6ecc-41f7-8d05-9ec3847a898e admin admin] PATH_INFO: `/v3/domains` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:54:37.328135 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66f60a2f-6ecc-41f7-8d05-9ec3847a898e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80143) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 10:54:37.328660 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66f60a2f-6ecc-41f7-8d05-9ec3847a898e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 10:54:37.331081 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66f60a2f-6ecc-41f7-8d05-9ec3847a898e admin admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 10:54:37.334734 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 10:54:37.346649 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 872/1772] 10.222.0.25 () {58 vars in 1153 bytes} [Sun Aug 7 10:54:37 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 68 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 10:54:37.351900 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66f60a2f-6ecc-41f7-8d05-9ec3847a898e admin admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 10:54:37.368908 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66f60a2f-6ecc-41f7-8d05-9ec3847a898e admin admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 10:54:37.375234 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07279a0d-c200-4b67-8a70-77c8d93c50bf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5ac47e3b09f43d19a23a0f457fe9e23', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 10:54:37.375477 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.auth.core [None req-869a3391-356b-4b48-960c-e080b08a0e4e None None] MFA Rules not processed for user `bac4e6b76fe64217ac8fa44e71cffe6a`. Rule list: `[]` (Enabled: `True`). {{(pid=80144) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 10:54:37.376312 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07279a0d-c200-4b67-8a70-77c8d93c50bf admin admin] REQUEST_METHOD: `POST` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:54:37.377049 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07279a0d-c200-4b67-8a70-77c8d93c50bf admin admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:54:37.377852 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07279a0d-c200-4b67-8a70-77c8d93c50bf admin admin] PATH_INFO: `/v3/projects` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:54:37.378631 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07279a0d-c200-4b67-8a70-77c8d93c50bf admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 10:54:37.379903 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.fernet_utils [None req-869a3391-356b-4b48-960c-e080b08a0e4e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80144) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 10:54:37.382322 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07279a0d-c200-4b67-8a70-77c8d93c50bf admin admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 10:54:37.406553 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 901/1773] 10.222.0.25 () {58 vars in 958 bytes} [Sun Aug 7 10:54:37 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 07 10:54:37.410678 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-869a3391-356b-4b48-960c-e080b08a0e4e None None] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 10:54:37.432430 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-869a3391-356b-4b48-960c-e080b08a0e4e None None] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 10:54:37.433054 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 873/1774] 10.222.0.25 () {60 vars in 1153 bytes} [Sun Aug 7 10:54:37 2022] POST /identity/v3/projects => generated 393 bytes in 83 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 07 10:54:37.435857 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07279a0d-c200-4b67-8a70-77c8d93c50bf admin admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 10:54:37.556520 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cd24f96-ed78-4b4d-850d-849310a3c7f6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5ac47e3b09f43d19a23a0f457fe9e23', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 10:54:37.562636 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cd24f96-ed78-4b4d-850d-849310a3c7f6 admin admin] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:54:37.562636 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cd24f96-ed78-4b4d-850d-849310a3c7f6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:54:37.562636 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cd24f96-ed78-4b4d-850d-849310a3c7f6 admin admin] PATH_INFO: `/v3/domains` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:54:37.562636 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cd24f96-ed78-4b4d-850d-849310a3c7f6 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80144) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 10:54:37.562636 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cd24f96-ed78-4b4d-850d-849310a3c7f6 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 10:54:37.570916 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cd24f96-ed78-4b4d-850d-849310a3c7f6 admin admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 10:54:37.618742 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 902/1775] 10.222.0.25 () {58 vars in 1153 bytes} [Sun Aug 7 10:54:37 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 10:54:37.619809 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cd24f96-ed78-4b4d-850d-849310a3c7f6 admin admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 10:54:37.632929 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cd24f96-ed78-4b4d-850d-849310a3c7f6 admin admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 10:54:37.639737 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18cecb10-dd3e-4002-88a3-a628d87945d7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5ac47e3b09f43d19a23a0f457fe9e23', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 10:54:37.640439 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18cecb10-dd3e-4002-88a3-a628d87945d7 admin admin] REQUEST_METHOD: `POST` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:54:37.640769 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18cecb10-dd3e-4002-88a3-a628d87945d7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:54:37.641048 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18cecb10-dd3e-4002-88a3-a628d87945d7 admin admin] PATH_INFO: `/v3/projects` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:54:37.642667 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18cecb10-dd3e-4002-88a3-a628d87945d7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 10:54:37.645119 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18cecb10-dd3e-4002-88a3-a628d87945d7 admin admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 10:54:37.699240 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 903/1776] 10.222.0.25 () {60 vars in 1153 bytes} [Sun Aug 7 10:54:37 2022] POST /identity/v3/projects => generated 367 bytes in 79 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 07 10:54:37.702250 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18cecb10-dd3e-4002-88a3-a628d87945d7 admin admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 10:54:37.704431 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07279a0d-c200-4b67-8a70-77c8d93c50bf admin admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 10:54:37.771755 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 10:54:37.889758 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6eaf251d-64f1-495a-b4ef-0955d0a67245 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5ac47e3b09f43d19a23a0f457fe9e23', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 10:54:37.890442 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eaf251d-64f1-495a-b4ef-0955d0a67245 admin admin] REQUEST_METHOD: `POST` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:54:37.890729 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eaf251d-64f1-495a-b4ef-0955d0a67245 admin admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:54:37.891098 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eaf251d-64f1-495a-b4ef-0955d0a67245 admin admin] PATH_INFO: `/v3/users` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:54:37.892928 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6eaf251d-64f1-495a-b4ef-0955d0a67245 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 10:54:37.896092 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6eaf251d-64f1-495a-b4ef-0955d0a67245 admin admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 10:54:37.911674 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18cecb10-dd3e-4002-88a3-a628d87945d7 admin admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 10:54:37.954674 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03af73e8-2d04-4251-a695-c956fba39bba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5ac47e3b09f43d19a23a0f457fe9e23', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 10:54:37.954674 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03af73e8-2d04-4251-a695-c956fba39bba admin admin] REQUEST_METHOD: `POST` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:54:37.954674 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03af73e8-2d04-4251-a695-c956fba39bba admin admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:54:37.954674 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03af73e8-2d04-4251-a695-c956fba39bba admin admin] PATH_INFO: `/v3/users` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:54:37.955202 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03af73e8-2d04-4251-a695-c956fba39bba admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 10:54:37.955278 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 874/1777] 10.222.0.25 () {60 vars in 1144 bytes} [Sun Aug 7 10:54:37 2022] POST /identity/v3/users => generated 297 bytes in 517 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 07 10:54:37.955373 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03af73e8-2d04-4251-a695-c956fba39bba admin admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 10:54:37.956481 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6eaf251d-64f1-495a-b4ef-0955d0a67245 admin admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 10:54:37.992192 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6eaf251d-64f1-495a-b4ef-0955d0a67245 admin admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 10:54:37.999126 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 904/1778] 10.222.0.25 () {60 vars in 1144 bytes} [Sun Aug 7 10:54:37 2022] POST /identity/v3/users => generated 284 bytes in 299 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 07 10:54:38.005823 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03af73e8-2d04-4251-a695-c956fba39bba admin admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 10:54:38.008957 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-876183a7-fa14-4a03-945d-50cb36963e0e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5ac47e3b09f43d19a23a0f457fe9e23', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 10:54:38.011194 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-876183a7-fa14-4a03-945d-50cb36963e0e admin admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:54:38.011501 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-876183a7-fa14-4a03-945d-50cb36963e0e admin admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:54:38.012942 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-876183a7-fa14-4a03-945d-50cb36963e0e admin admin] PATH_INFO: `/v3/roles` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:54:38.013636 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-876183a7-fa14-4a03-945d-50cb36963e0e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 10:54:38.018458 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-876183a7-fa14-4a03-945d-50cb36963e0e admin admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 10:54:38.038630 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03af73e8-2d04-4251-a695-c956fba39bba admin admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 10:54:38.052048 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 875/1779] 10.222.0.25 () {58 vars in 1122 bytes} [Sun Aug 7 10:54:37 2022] GET /identity/v3/roles => generated 1420 bytes in 95 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 07 10:54:38.052255 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba60df3b-61f6-4985-b87f-ff3bd0795b95 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5ac47e3b09f43d19a23a0f457fe9e23', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 10:54:38.052255 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba60df3b-61f6-4985-b87f-ff3bd0795b95 admin admin] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:54:38.052255 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba60df3b-61f6-4985-b87f-ff3bd0795b95 admin admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:54:38.053416 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-876183a7-fa14-4a03-945d-50cb36963e0e admin admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 10:54:38.053896 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba60df3b-61f6-4985-b87f-ff3bd0795b95 admin admin] PATH_INFO: `/v3/roles` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:54:38.055078 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba60df3b-61f6-4985-b87f-ff3bd0795b95 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 10:54:38.061130 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba60df3b-61f6-4985-b87f-ff3bd0795b95 admin admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 10:54:38.078783 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-876183a7-fa14-4a03-945d-50cb36963e0e admin admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 10:54:38.092879 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 905/1780] 10.222.0.25 () {58 vars in 1122 bytes} [Sun Aug 7 10:54:38 2022] GET /identity/v3/roles => generated 1420 bytes in 89 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 07 10:54:38.095587 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba60df3b-61f6-4985-b87f-ff3bd0795b95 admin admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 10:54:38.103459 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b80a5dfb-99a9-4953-8fe4-c5a06bbc6a0e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5ac47e3b09f43d19a23a0f457fe9e23', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 10:54:38.104202 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b80a5dfb-99a9-4953-8fe4-c5a06bbc6a0e admin admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:54:38.104495 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b80a5dfb-99a9-4953-8fe4-c5a06bbc6a0e admin admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:54:38.104774 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b80a5dfb-99a9-4953-8fe4-c5a06bbc6a0e admin admin] PATH_INFO: `/v3/roles` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:54:38.105434 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b80a5dfb-99a9-4953-8fe4-c5a06bbc6a0e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 10:54:38.110509 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b80a5dfb-99a9-4953-8fe4-c5a06bbc6a0e admin admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 10:54:38.125592 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba60df3b-61f6-4985-b87f-ff3bd0795b95 admin admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 10:54:38.134654 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b9313dc-357e-44a2-945e-683a60290534 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5ac47e3b09f43d19a23a0f457fe9e23', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 10:54:38.135619 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b9313dc-357e-44a2-945e-683a60290534 admin admin] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:54:38.136077 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b9313dc-357e-44a2-945e-683a60290534 admin admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:54:38.136537 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b9313dc-357e-44a2-945e-683a60290534 admin admin] PATH_INFO: `/v3/roles` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:54:38.137362 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b9313dc-357e-44a2-945e-683a60290534 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 10:54:38.151387 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 876/1781] 10.222.0.25 () {58 vars in 1122 bytes} [Sun Aug 7 10:54:38 2022] GET /identity/v3/roles => generated 1420 bytes in 96 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 07 10:54:38.152518 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b80a5dfb-99a9-4953-8fe4-c5a06bbc6a0e admin admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 10:54:38.157158 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b9313dc-357e-44a2-945e-683a60290534 admin admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 10:54:38.182853 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b80a5dfb-99a9-4953-8fe4-c5a06bbc6a0e admin admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 10:54:38.183816 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 906/1782] 10.222.0.25 () {58 vars in 1122 bytes} [Sun Aug 7 10:54:38 2022] GET /identity/v3/roles => generated 1420 bytes in 89 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 07 10:54:38.188682 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b9313dc-357e-44a2-945e-683a60290534 admin admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 10:54:38.208930 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b6eb127-4758-47dc-9e67-f3203f7bd1d3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5ac47e3b09f43d19a23a0f457fe9e23', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 10:54:38.212980 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b6eb127-4758-47dc-9e67-f3203f7bd1d3 admin admin] REQUEST_METHOD: `PUT` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:54:38.213323 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b6eb127-4758-47dc-9e67-f3203f7bd1d3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:54:38.213606 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b6eb127-4758-47dc-9e67-f3203f7bd1d3 admin admin] PATH_INFO: `/v3/projects/811730b82a534181babbef168ca91049/users/83b543260da44b868c0634e0fa0c8234/roles/b98dfb3b9fee4644af2887a3ac3fd2e6` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:54:38.221860 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b9313dc-357e-44a2-945e-683a60290534 admin admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 10:54:38.234321 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a088d31e-b844-4e23-885d-8df7d35cd6cf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5ac47e3b09f43d19a23a0f457fe9e23', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 10:54:38.236511 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a088d31e-b844-4e23-885d-8df7d35cd6cf admin admin] REQUEST_METHOD: `PUT` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:54:38.236862 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a088d31e-b844-4e23-885d-8df7d35cd6cf admin admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:54:38.237205 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a088d31e-b844-4e23-885d-8df7d35cd6cf admin admin] PATH_INFO: `/v3/projects/929e031fd1ad4c4fb5f4662ff2364ad6/users/867775064a12466aa27f981b2c90e285/roles/b98dfb3b9fee4644af2887a3ac3fd2e6` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:54:38.262794 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b6eb127-4758-47dc-9e67-f3203f7bd1d3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=811730b82a534181babbef168ca91049, user_id=83b543260da44b868c0634e0fa0c8234, role_id=b98dfb3b9fee4644af2887a3ac3fd2e6)` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 10:54:38.266404 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b6eb127-4758-47dc-9e67-f3203f7bd1d3 admin admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 10:54:38.288314 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 877/1783] 10.222.0.25 () {60 vars in 1483 bytes} [Sun Aug 7 10:54:38 2022] PUT /identity/v3/projects/811730b82a534181babbef168ca91049/users/83b543260da44b868c0634e0fa0c8234/roles/b98dfb3b9fee4644af2887a3ac3fd2e6 => generated 0 bytes in 136 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 07 10:54:38.301321 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a088d31e-b844-4e23-885d-8df7d35cd6cf admin admin] RBAC: Authorizing `identity:create_grant(project_id=929e031fd1ad4c4fb5f4662ff2364ad6, user_id=867775064a12466aa27f981b2c90e285, role_id=b98dfb3b9fee4644af2887a3ac3fd2e6)` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 10:54:38.313239 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a088d31e-b844-4e23-885d-8df7d35cd6cf admin admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 10:54:38.314867 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5018520f-1c4f-4cf8-a057-2638817d0bb7 None None] REQUEST_METHOD: `POST` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:54:38.314867 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5018520f-1c4f-4cf8-a057-2638817d0bb7 None None] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:54:38.314867 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5018520f-1c4f-4cf8-a057-2638817d0bb7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:54:38.349523 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 907/1784] 10.222.0.25 () {60 vars in 1483 bytes} [Sun Aug 7 10:54:38 2022] PUT /identity/v3/projects/929e031fd1ad4c4fb5f4662ff2364ad6/users/867775064a12466aa27f981b2c90e285/roles/b98dfb3b9fee4644af2887a3ac3fd2e6 => generated 0 bytes in 161 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 07 10:54:38.351202 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.auth.core [None req-5018520f-1c4f-4cf8-a057-2638817d0bb7 None None] MFA Rules not processed for user `83b543260da44b868c0634e0fa0c8234`. Rule list: `[]` (Enabled: `True`). {{(pid=80143) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 10:54:38.352642 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-5018520f-1c4f-4cf8-a057-2638817d0bb7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 10:54:38.364365 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-547135d3-101f-4506-9326-352f70d0d2b2 None None] REQUEST_METHOD: `POST` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:54:38.364726 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-547135d3-101f-4506-9326-352f70d0d2b2 None None] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:54:38.365067 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-547135d3-101f-4506-9326-352f70d0d2b2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:54:38.378279 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 10:54:38.444201 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.auth.core [None req-547135d3-101f-4506-9326-352f70d0d2b2 None None] MFA Rules not processed for user `867775064a12466aa27f981b2c90e285`. Rule list: `[]` (Enabled: `True`). {{(pid=80144) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 10:54:38.445721 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.fernet_utils [None req-547135d3-101f-4506-9326-352f70d0d2b2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80144) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 10:54:38.488859 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 878/1785] 10.222.0.25 () {58 vars in 958 bytes} [Sun Aug 7 10:54:38 2022] POST /identity/v3/auth/tokens => generated 3270 bytes in 188 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 07 10:54:38.494045 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5018520f-1c4f-4cf8-a057-2638817d0bb7 None None] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 10:54:38.626899 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 908/1786] 10.222.0.25 () {58 vars in 958 bytes} [Sun Aug 7 10:54:38 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 263 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 07 10:54:38.635921 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-547135d3-101f-4506-9326-352f70d0d2b2 None None] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 10:54:38.683571 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5018520f-1c4f-4cf8-a057-2638817d0bb7 None None] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 10:54:38.701013 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f15f94cc-f774-4956-9e3c-fd329ec9cb1a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5ac47e3b09f43d19a23a0f457fe9e23', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 10:54:38.701949 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f15f94cc-f774-4956-9e3c-fd329ec9cb1a admin admin] REQUEST_METHOD: `POST` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:54:38.702389 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f15f94cc-f774-4956-9e3c-fd329ec9cb1a admin admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:54:38.702830 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f15f94cc-f774-4956-9e3c-fd329ec9cb1a admin admin] PATH_INFO: `/v3/projects` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:54:38.703648 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f15f94cc-f774-4956-9e3c-fd329ec9cb1a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 10:54:38.710087 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f15f94cc-f774-4956-9e3c-fd329ec9cb1a admin admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 10:54:38.805104 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 879/1787] 10.222.0.25 () {60 vars in 1153 bytes} [Sun Aug 7 10:54:38 2022] POST /identity/v3/projects => generated 393 bytes in 310 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 07 10:54:38.805423 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f15f94cc-f774-4956-9e3c-fd329ec9cb1a admin admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 10:54:38.809769 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-547135d3-101f-4506-9326-352f70d0d2b2 None None] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 10:54:38.821199 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 10:54:39.024601 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e061f43-a8cf-4362-bc0a-eecaed3de537 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5ac47e3b09f43d19a23a0f457fe9e23', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 10:54:39.025356 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e061f43-a8cf-4362-bc0a-eecaed3de537 admin admin] REQUEST_METHOD: `POST` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:54:39.025744 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e061f43-a8cf-4362-bc0a-eecaed3de537 admin admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:54:39.026342 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e061f43-a8cf-4362-bc0a-eecaed3de537 admin admin] PATH_INFO: `/v3/projects` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:54:39.027067 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e061f43-a8cf-4362-bc0a-eecaed3de537 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 10:54:39.029201 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e061f43-a8cf-4362-bc0a-eecaed3de537 admin admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 10:54:39.065041 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f15f94cc-f774-4956-9e3c-fd329ec9cb1a admin admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 10:54:39.117511 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 909/1788] 10.222.0.25 () {60 vars in 1153 bytes} [Sun Aug 7 10:54:38 2022] POST /identity/v3/projects => generated 365 bytes in 489 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 07 10:54:39.118061 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f6a0fb3-12d2-4f9e-a0e7-e6c4a34740f7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5ac47e3b09f43d19a23a0f457fe9e23', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 10:54:39.118710 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f6a0fb3-12d2-4f9e-a0e7-e6c4a34740f7 admin admin] REQUEST_METHOD: `POST` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:54:39.119022 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f6a0fb3-12d2-4f9e-a0e7-e6c4a34740f7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:54:39.119304 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f6a0fb3-12d2-4f9e-a0e7-e6c4a34740f7 admin admin] PATH_INFO: `/v3/users` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:54:39.119952 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f6a0fb3-12d2-4f9e-a0e7-e6c4a34740f7 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 10:54:39.124383 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-122b0635-ebc2-4375-bfbb-944e47e4a58e None None] REQUEST_METHOD: `POST` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:54:39.126761 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-122b0635-ebc2-4375-bfbb-944e47e4a58e None None] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:54:39.127235 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-122b0635-ebc2-4375-bfbb-944e47e4a58e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:54:39.140579 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f6a0fb3-12d2-4f9e-a0e7-e6c4a34740f7 admin admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 10:54:39.225896 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 880/1789] 10.222.0.25 () {60 vars in 1144 bytes} [Sun Aug 7 10:54:38 2022] POST /identity/v3/users => generated 303 bytes in 418 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 07 10:54:39.226528 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.auth.core [None req-122b0635-ebc2-4375-bfbb-944e47e4a58e None None] MFA Rules not processed for user `2b64b05230b940eba992a063ea7cd778`. Rule list: `[]` (Enabled: `True`). {{(pid=80144) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 10:54:39.226632 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f6a0fb3-12d2-4f9e-a0e7-e6c4a34740f7 admin admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 10:54:39.229275 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.fernet_utils [None req-122b0635-ebc2-4375-bfbb-944e47e4a58e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80144) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 10:54:39.267044 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f6a0fb3-12d2-4f9e-a0e7-e6c4a34740f7 admin admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 10:54:39.368799 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 10:54:39.469130 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 910/1790] 10.222.0.25 () {58 vars in 958 bytes} [Sun Aug 7 10:54:39 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 351 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 07 10:54:39.476176 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-122b0635-ebc2-4375-bfbb-944e47e4a58e None None] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 10:54:39.565296 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf7bf24f-bd46-4010-bbf4-f8174c34cb56 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5ac47e3b09f43d19a23a0f457fe9e23', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 10:54:39.567807 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf7bf24f-bd46-4010-bbf4-f8174c34cb56 admin admin] REQUEST_METHOD: `POST` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:54:39.571898 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf7bf24f-bd46-4010-bbf4-f8174c34cb56 admin admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:54:39.572229 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf7bf24f-bd46-4010-bbf4-f8174c34cb56 admin admin] PATH_INFO: `/v3/users` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:54:39.576376 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf7bf24f-bd46-4010-bbf4-f8174c34cb56 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 10:54:39.587070 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf7bf24f-bd46-4010-bbf4-f8174c34cb56 admin admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 10:54:39.679711 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 881/1791] 10.222.0.25 () {60 vars in 1144 bytes} [Sun Aug 7 10:54:39 2022] POST /identity/v3/users => generated 289 bytes in 455 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 07 10:54:39.680195 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf7bf24f-bd46-4010-bbf4-f8174c34cb56 admin admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 10:54:39.700653 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-122b0635-ebc2-4375-bfbb-944e47e4a58e None None] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 10:54:39.704809 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf7bf24f-bd46-4010-bbf4-f8174c34cb56 admin admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 10:54:39.713480 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d9a15b8-8d01-4df0-92b9-30d84ba664bb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5ac47e3b09f43d19a23a0f457fe9e23', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 10:54:39.714181 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d9a15b8-8d01-4df0-92b9-30d84ba664bb admin admin] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:54:39.714526 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d9a15b8-8d01-4df0-92b9-30d84ba664bb admin admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:54:39.714889 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d9a15b8-8d01-4df0-92b9-30d84ba664bb admin admin] PATH_INFO: `/v3/roles` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:54:39.717336 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d9a15b8-8d01-4df0-92b9-30d84ba664bb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 10:54:39.720435 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6639a17f-5da1-4d31-937a-fbb1f7f8e611 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5ac47e3b09f43d19a23a0f457fe9e23', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 10:54:39.721078 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6639a17f-5da1-4d31-937a-fbb1f7f8e611 admin admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:54:39.721441 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6639a17f-5da1-4d31-937a-fbb1f7f8e611 admin admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:54:39.721622 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6639a17f-5da1-4d31-937a-fbb1f7f8e611 admin admin] PATH_INFO: `/v3/roles` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:54:39.722288 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6639a17f-5da1-4d31-937a-fbb1f7f8e611 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 10:54:39.723065 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d9a15b8-8d01-4df0-92b9-30d84ba664bb admin admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 10:54:39.724610 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6639a17f-5da1-4d31-937a-fbb1f7f8e611 admin admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 10:54:39.750574 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 911/1792] 10.222.0.25 () {58 vars in 1122 bytes} [Sun Aug 7 10:54:39 2022] GET /identity/v3/roles => generated 1420 bytes in 275 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 07 10:54:39.751960 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d9a15b8-8d01-4df0-92b9-30d84ba664bb admin admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 10:54:39.764507 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 882/1793] 10.222.0.25 () {58 vars in 1122 bytes} [Sun Aug 7 10:54:39 2022] GET /identity/v3/roles => generated 1420 bytes in 85 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 07 10:54:39.769090 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6639a17f-5da1-4d31-937a-fbb1f7f8e611 admin admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 10:54:39.789637 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6639a17f-5da1-4d31-937a-fbb1f7f8e611 admin admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 10:54:39.793359 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d9a15b8-8d01-4df0-92b9-30d84ba664bb admin admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 10:54:39.803801 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22b8e8a0-8bad-469d-8092-1e0a0b8b1b84 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5ac47e3b09f43d19a23a0f457fe9e23', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 10:54:39.805570 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22b8e8a0-8bad-469d-8092-1e0a0b8b1b84 admin admin] REQUEST_METHOD: `PUT` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:54:39.806027 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22b8e8a0-8bad-469d-8092-1e0a0b8b1b84 admin admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:54:39.806313 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22b8e8a0-8bad-469d-8092-1e0a0b8b1b84 admin admin] PATH_INFO: `/v3/projects/8cb6be2e01be442382e1682df57b454c/users/0d65e97423d34648a73bd43c687c21a5/roles/81b1f1bcee2a473088fbbd327c1c5b23` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:54:39.807970 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7852b747-cb99-4672-bb88-a703c929fb0b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5ac47e3b09f43d19a23a0f457fe9e23', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 10:54:39.808725 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7852b747-cb99-4672-bb88-a703c929fb0b admin admin] REQUEST_METHOD: `PUT` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:54:39.809174 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7852b747-cb99-4672-bb88-a703c929fb0b admin admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:54:39.809463 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7852b747-cb99-4672-bb88-a703c929fb0b admin admin] PATH_INFO: `/v3/projects/86d8cd42509443aca5ed02397336e0be/users/3119b07d13104006bbf63ec995eabbc9/roles/81b1f1bcee2a473088fbbd327c1c5b23` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:54:39.837361 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22b8e8a0-8bad-469d-8092-1e0a0b8b1b84 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8cb6be2e01be442382e1682df57b454c, user_id=0d65e97423d34648a73bd43c687c21a5, role_id=81b1f1bcee2a473088fbbd327c1c5b23)` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 10:54:39.840074 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22b8e8a0-8bad-469d-8092-1e0a0b8b1b84 admin admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 10:54:39.855963 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7852b747-cb99-4672-bb88-a703c929fb0b admin admin] RBAC: Authorizing `identity:create_grant(project_id=86d8cd42509443aca5ed02397336e0be, user_id=3119b07d13104006bbf63ec995eabbc9, role_id=81b1f1bcee2a473088fbbd327c1c5b23)` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 10:54:39.864855 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7852b747-cb99-4672-bb88-a703c929fb0b admin admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 10:54:39.873478 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 883/1794] 10.222.0.25 () {60 vars in 1483 bytes} [Sun Aug 7 10:54:39 2022] PUT /identity/v3/projects/8cb6be2e01be442382e1682df57b454c/users/0d65e97423d34648a73bd43c687c21a5/roles/81b1f1bcee2a473088fbbd327c1c5b23 => generated 0 bytes in 101 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 07 10:54:39.874753 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22b8e8a0-8bad-469d-8092-1e0a0b8b1b84 admin admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 10:54:39.906746 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 912/1795] 10.222.0.25 () {60 vars in 1483 bytes} [Sun Aug 7 10:54:39 2022] PUT /identity/v3/projects/86d8cd42509443aca5ed02397336e0be/users/3119b07d13104006bbf63ec995eabbc9/roles/81b1f1bcee2a473088fbbd327c1c5b23 => generated 0 bytes in 153 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 07 10:54:39.909449 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7852b747-cb99-4672-bb88-a703c929fb0b admin admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 10:54:39.937773 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 10:54:39.969280 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22b8e8a0-8bad-469d-8092-1e0a0b8b1b84 admin admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 10:54:40.072810 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7852b747-cb99-4672-bb88-a703c929fb0b admin admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 10:54:40.082839 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b949d1dc-995d-4c01-99dc-5870b39bb51c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5ac47e3b09f43d19a23a0f457fe9e23', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 10:54:40.085045 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b949d1dc-995d-4c01-99dc-5870b39bb51c admin admin] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:54:40.085405 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b949d1dc-995d-4c01-99dc-5870b39bb51c admin admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:54:40.085814 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b949d1dc-995d-4c01-99dc-5870b39bb51c admin admin] PATH_INFO: `/v3/roles` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:54:40.086575 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b949d1dc-995d-4c01-99dc-5870b39bb51c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 10:54:40.090204 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b949d1dc-995d-4c01-99dc-5870b39bb51c admin admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 10:54:40.135178 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 913/1796] 10.222.0.25 () {58 vars in 1122 bytes} [Sun Aug 7 10:54:39 2022] GET /identity/v3/roles => generated 1420 bytes in 225 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 07 10:54:40.136047 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b949d1dc-995d-4c01-99dc-5870b39bb51c admin admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 10:54:40.143379 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c3ff10c-355e-4620-883d-ada15da402c8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5ac47e3b09f43d19a23a0f457fe9e23', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 10:54:40.144286 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c3ff10c-355e-4620-883d-ada15da402c8 admin admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:54:40.144661 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c3ff10c-355e-4620-883d-ada15da402c8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:54:40.145036 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c3ff10c-355e-4620-883d-ada15da402c8 admin admin] PATH_INFO: `/v3/roles` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:54:40.145830 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c3ff10c-355e-4620-883d-ada15da402c8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 10:54:40.152657 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c3ff10c-355e-4620-883d-ada15da402c8 admin admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 10:54:40.176916 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b949d1dc-995d-4c01-99dc-5870b39bb51c admin admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 10:54:40.180015 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 884/1797] 10.222.0.25 () {58 vars in 1122 bytes} [Sun Aug 7 10:54:39 2022] GET /identity/v3/roles => generated 1420 bytes in 306 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 07 10:54:40.181752 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c3ff10c-355e-4620-883d-ada15da402c8 admin admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 10:54:40.198029 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c3ff10c-355e-4620-883d-ada15da402c8 admin admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 10:54:40.198177 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9edfa9d-06e8-43eb-9b65-4ec24469c955 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5ac47e3b09f43d19a23a0f457fe9e23', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 10:54:40.198754 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9edfa9d-06e8-43eb-9b65-4ec24469c955 admin admin] REQUEST_METHOD: `PUT` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:54:40.201818 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9edfa9d-06e8-43eb-9b65-4ec24469c955 admin admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:54:40.202382 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9edfa9d-06e8-43eb-9b65-4ec24469c955 admin admin] PATH_INFO: `/v3/projects/86d8cd42509443aca5ed02397336e0be/users/3119b07d13104006bbf63ec995eabbc9/roles/81b1f1bcee2a473088fbbd327c1c5b23` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:54:40.206608 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9edfa9d-06e8-43eb-9b65-4ec24469c955 admin admin] RBAC: Authorizing `identity:create_grant(project_id=86d8cd42509443aca5ed02397336e0be, user_id=3119b07d13104006bbf63ec995eabbc9, role_id=81b1f1bcee2a473088fbbd327c1c5b23)` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 10:54:40.207108 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c1ceebe-8037-4e09-aa1d-8d7a993ad317 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5ac47e3b09f43d19a23a0f457fe9e23', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 10:54:40.210714 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c1ceebe-8037-4e09-aa1d-8d7a993ad317 admin admin] REQUEST_METHOD: `PUT` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:54:40.211554 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c1ceebe-8037-4e09-aa1d-8d7a993ad317 admin admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:54:40.212080 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9edfa9d-06e8-43eb-9b65-4ec24469c955 admin admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 10:54:40.212431 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c1ceebe-8037-4e09-aa1d-8d7a993ad317 admin admin] PATH_INFO: `/v3/projects/8cb6be2e01be442382e1682df57b454c/users/0d65e97423d34648a73bd43c687c21a5/roles/81b1f1bcee2a473088fbbd327c1c5b23` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:54:40.215933 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c1ceebe-8037-4e09-aa1d-8d7a993ad317 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8cb6be2e01be442382e1682df57b454c, user_id=0d65e97423d34648a73bd43c687c21a5, role_id=81b1f1bcee2a473088fbbd327c1c5b23)` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 10:54:40.220678 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c1ceebe-8037-4e09-aa1d-8d7a993ad317 admin admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 10:54:40.237300 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 885/1798] 10.222.0.25 () {60 vars in 1483 bytes} [Sun Aug 7 10:54:40 2022] PUT /identity/v3/projects/8cb6be2e01be442382e1682df57b454c/users/0d65e97423d34648a73bd43c687c21a5/roles/81b1f1bcee2a473088fbbd327c1c5b23 => generated 0 bytes in 57 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 07 10:54:40.247823 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 914/1799] 10.222.0.25 () {60 vars in 1483 bytes} [Sun Aug 7 10:54:40 2022] PUT /identity/v3/projects/86d8cd42509443aca5ed02397336e0be/users/3119b07d13104006bbf63ec995eabbc9/roles/81b1f1bcee2a473088fbbd327c1c5b23 => generated 0 bytes in 105 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 07 10:54:40.252708 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f4d1dc3-24d7-44c4-81bc-82487fa5340f None None] REQUEST_METHOD: `POST` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:54:40.252956 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f4d1dc3-24d7-44c4-81bc-82487fa5340f None None] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:54:40.253222 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f4d1dc3-24d7-44c4-81bc-82487fa5340f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:54:40.267238 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc9a1f21-5fcf-425a-b155-16f22d2d31c5 None None] REQUEST_METHOD: `POST` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:54:40.267422 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc9a1f21-5fcf-425a-b155-16f22d2d31c5 None None] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:54:40.267678 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc9a1f21-5fcf-425a-b155-16f22d2d31c5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:54:40.325387 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.auth.core [None req-1f4d1dc3-24d7-44c4-81bc-82487fa5340f None None] MFA Rules not processed for user `0d65e97423d34648a73bd43c687c21a5`. Rule list: `[]` (Enabled: `True`). {{(pid=80144) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 10:54:40.327464 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.fernet_utils [None req-1f4d1dc3-24d7-44c4-81bc-82487fa5340f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80144) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 10:54:40.344988 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 10:54:40.354345 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.auth.core [None req-dc9a1f21-5fcf-425a-b155-16f22d2d31c5 None None] MFA Rules not processed for user `3119b07d13104006bbf63ec995eabbc9`. Rule list: `[]` (Enabled: `True`). {{(pid=80143) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 10:54:40.355614 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-dc9a1f21-5fcf-425a-b155-16f22d2d31c5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 10:54:40.449703 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 915/1800] 10.222.0.25 () {58 vars in 958 bytes} [Sun Aug 7 10:54:40 2022] POST /identity/v3/auth/tokens => generated 3309 bytes in 198 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 07 10:54:40.462590 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f4d1dc3-24d7-44c4-81bc-82487fa5340f None None] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 10:54:40.467685 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.fernet_utils [None req-1f4d1dc3-24d7-44c4-81bc-82487fa5340f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80144) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 10:54:40.514284 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 886/1801] 10.222.0.25 () {58 vars in 958 bytes} [Sun Aug 7 10:54:40 2022] POST /identity/v3/auth/tokens => generated 3337 bytes in 250 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 07 10:54:40.527895 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc9a1f21-5fcf-425a-b155-16f22d2d31c5 None None] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 10:54:40.529679 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-dc9a1f21-5fcf-425a-b155-16f22d2d31c5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 10:54:40.538946 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f4d1dc3-24d7-44c4-81bc-82487fa5340f None None] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 10:54:40.546000 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbc98a19-2a17-4f5c-a62b-a558865f03b8 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88dbd6dfd1164994a3e16da553561910', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6de1f3cac02c4bcdb05dd2337176033d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 10:54:40.547340 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbc98a19-2a17-4f5c-a62b-a558865f03b8 service cinder] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:54:40.548190 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbc98a19-2a17-4f5c-a62b-a558865f03b8 service cinder] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:54:40.548481 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbc98a19-2a17-4f5c-a62b-a558865f03b8 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:54:40.551226 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc9a1f21-5fcf-425a-b155-16f22d2d31c5 None None] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 10:54:40.559876 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6527e077-124d-4361-9db0-0670f259b571 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88dbd6dfd1164994a3e16da553561910', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6de1f3cac02c4bcdb05dd2337176033d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 10:54:40.560523 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6527e077-124d-4361-9db0-0670f259b571 service cinder] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:54:40.560887 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6527e077-124d-4361-9db0-0670f259b571 service cinder] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:54:40.561251 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6527e077-124d-4361-9db0-0670f259b571 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:54:40.573404 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbc98a19-2a17-4f5c-a62b-a558865f03b8 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 10:54:40.575151 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbc98a19-2a17-4f5c-a62b-a558865f03b8 service cinder] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 10:54:40.583780 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6527e077-124d-4361-9db0-0670f259b571 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 10:54:40.590112 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6527e077-124d-4361-9db0-0670f259b571 service cinder] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 10:54:40.601975 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 916/1802] 10.222.0.25 () {60 vars in 1365 bytes} [Sun Aug 7 10:54:40 2022] GET /identity/v3/auth/tokens => generated 3309 bytes in 139 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 07 10:54:40.606900 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 887/1803] 10.222.0.25 () {60 vars in 1365 bytes} [Sun Aug 7 10:54:40 2022] GET /identity/v3/auth/tokens => generated 3337 bytes in 79 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 07 10:54:40.737068 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbc98a19-2a17-4f5c-a62b-a558865f03b8 service cinder] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 10:54:40.749387 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbc98a19-2a17-4f5c-a62b-a558865f03b8 service cinder] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 10:54:40.758481 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-775699c4-4d92-4f94-b30c-c15a0c235c94 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88dbd6dfd1164994a3e16da553561910', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6de1f3cac02c4bcdb05dd2337176033d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 10:54:40.760219 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-775699c4-4d92-4f94-b30c-c15a0c235c94 service cinder] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:54:40.760504 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-775699c4-4d92-4f94-b30c-c15a0c235c94 service cinder] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:54:40.764125 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-775699c4-4d92-4f94-b30c-c15a0c235c94 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:54:40.768688 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6527e077-124d-4361-9db0-0670f259b571 service cinder] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 10:54:40.778496 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6527e077-124d-4361-9db0-0670f259b571 service cinder] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 10:54:40.788165 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a932121-89bf-4fff-a10f-bfa386ef4c54 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88dbd6dfd1164994a3e16da553561910', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6de1f3cac02c4bcdb05dd2337176033d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 10:54:40.788839 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a932121-89bf-4fff-a10f-bfa386ef4c54 service cinder] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:54:40.789076 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a932121-89bf-4fff-a10f-bfa386ef4c54 service cinder] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:54:40.789797 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a932121-89bf-4fff-a10f-bfa386ef4c54 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:54:40.811521 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 10:54:40.912284 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-775699c4-4d92-4f94-b30c-c15a0c235c94 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 10:54:40.913736 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-775699c4-4d92-4f94-b30c-c15a0c235c94 service cinder] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 10:54:40.928905 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a932121-89bf-4fff-a10f-bfa386ef4c54 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 10:54:40.930400 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a932121-89bf-4fff-a10f-bfa386ef4c54 service cinder] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 10:54:40.950114 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 917/1804] 10.222.0.25 () {60 vars in 1365 bytes} [Sun Aug 7 10:54:40 2022] GET /identity/v3/auth/tokens => generated 3270 bytes in 214 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 07 10:54:40.988606 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 888/1805] 10.222.0.25 () {60 vars in 1365 bytes} [Sun Aug 7 10:54:40 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 221 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 07 10:54:41.096269 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2654d4a-6fe4-4397-8175-2e189872ed67 None None] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:54:41.096561 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2654d4a-6fe4-4397-8175-2e189872ed67 None None] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:54:41.096907 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2654d4a-6fe4-4397-8175-2e189872ed67 None None] PATH_INFO: `/` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:54:41.098723 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 918/1806] 10.222.0.25 () {54 vars in 921 bytes} [Sun Aug 7 10:54:41 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 07 10:54:41.122218 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69e42392-c2c5-4b14-8bf5-2579ed4a871d None None] REQUEST_METHOD: `POST` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:54:41.122512 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69e42392-c2c5-4b14-8bf5-2579ed4a871d None None] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:54:41.122863 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69e42392-c2c5-4b14-8bf5-2579ed4a871d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:54:41.165132 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.auth.core [None req-69e42392-c2c5-4b14-8bf5-2579ed4a871d None None] MFA Rules not processed for user `31d16d1b96104b4eb690398432b9829e`. Rule list: `[]` (Enabled: `True`). {{(pid=80143) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 10:54:41.166530 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-69e42392-c2c5-4b14-8bf5-2579ed4a871d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 10:54:41.347645 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 889/1807] 10.222.0.25 () {58 vars in 1018 bytes} [Sun Aug 7 10:54:41 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 226 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 07 10:54:41.924321 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2654d4a-6fe4-4397-8175-2e189872ed67 None None] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 10:54:41.926438 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.fernet_utils [None req-f2654d4a-6fe4-4397-8175-2e189872ed67 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80144) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 10:54:41.948936 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2654d4a-6fe4-4397-8175-2e189872ed67 None None] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 10:54:41.956592 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13de34c4-dd90-4dc4-a283-56df66ffac0c service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31d16d1b96104b4eb690398432b9829e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6de1f3cac02c4bcdb05dd2337176033d', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'reader', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 10:54:41.957478 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13de34c4-dd90-4dc4-a283-56df66ffac0c service nova] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:54:41.958102 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13de34c4-dd90-4dc4-a283-56df66ffac0c service nova] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:54:41.958102 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13de34c4-dd90-4dc4-a283-56df66ffac0c service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:54:41.975040 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13de34c4-dd90-4dc4-a283-56df66ffac0c service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 10:54:41.976716 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13de34c4-dd90-4dc4-a283-56df66ffac0c service nova] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 10:54:41.989056 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 919/1808] 10.222.0.25 () {60 vars in 1365 bytes} [Sun Aug 7 10:54:41 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 68 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 07 10:54:42.582792 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c580722e-f407-428a-848b-1febd0db8b40 None None] REQUEST_METHOD: `POST` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:54:42.583363 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c580722e-f407-428a-848b-1febd0db8b40 None None] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:54:42.583734 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c580722e-f407-428a-848b-1febd0db8b40 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:54:42.625926 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.auth.core [None req-c580722e-f407-428a-848b-1febd0db8b40 None None] MFA Rules not processed for user `8fb9d5b3e8524ad09786d0eeedfbd643`. Rule list: `[]` (Enabled: `True`). {{(pid=80143) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 10:54:42.629099 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-c580722e-f407-428a-848b-1febd0db8b40 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 10:54:42.634411 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=100 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 10:54:42.740255 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 890/1809] 10.222.0.25 () {58 vars in 958 bytes} [Sun Aug 7 10:54:42 2022] POST /identity/v3/auth/tokens => generated 3280 bytes in 159 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 07 10:54:42.745301 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13de34c4-dd90-4dc4-a283-56df66ffac0c service nova] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 10:54:42.747109 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.fernet_utils [None req-13de34c4-dd90-4dc4-a283-56df66ffac0c service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80144) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 10:54:42.839524 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13de34c4-dd90-4dc4-a283-56df66ffac0c service nova] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 10:54:42.845646 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1c6d4ee-97ac-4b74-a2c1-1b9d84f0c93e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5ac47e3b09f43d19a23a0f457fe9e23', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 10:54:42.846783 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1c6d4ee-97ac-4b74-a2c1-1b9d84f0c93e admin admin] REQUEST_METHOD: `POST` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:54:42.847163 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1c6d4ee-97ac-4b74-a2c1-1b9d84f0c93e admin admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:54:42.847513 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1c6d4ee-97ac-4b74-a2c1-1b9d84f0c93e admin admin] PATH_INFO: `/v3/projects` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:54:42.848388 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1c6d4ee-97ac-4b74-a2c1-1b9d84f0c93e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 10:54:42.850708 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1c6d4ee-97ac-4b74-a2c1-1b9d84f0c93e admin admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 10:54:42.890959 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 920/1810] 10.222.0.25 () {60 vars in 1153 bytes} [Sun Aug 7 10:54:42 2022] POST /identity/v3/projects => generated 405 bytes in 146 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 07 10:54:42.895478 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c580722e-f407-428a-848b-1febd0db8b40 None None] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 10:54:42.907316 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c580722e-f407-428a-848b-1febd0db8b40 None None] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 10:54:42.986958 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3c7899d-de55-4e68-b1c5-385c4f040772 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5ac47e3b09f43d19a23a0f457fe9e23', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 10:54:42.987595 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3c7899d-de55-4e68-b1c5-385c4f040772 admin admin] REQUEST_METHOD: `POST` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:54:42.987907 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3c7899d-de55-4e68-b1c5-385c4f040772 admin admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:54:42.988188 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3c7899d-de55-4e68-b1c5-385c4f040772 admin admin] PATH_INFO: `/v3/users` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:54:42.988817 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3c7899d-de55-4e68-b1c5-385c4f040772 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 10:54:42.990770 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3c7899d-de55-4e68-b1c5-385c4f040772 admin admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 10:54:43.017355 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 891/1811] 10.222.0.25 () {60 vars in 1144 bytes} [Sun Aug 7 10:54:42 2022] POST /identity/v3/users => generated 309 bytes in 119 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 07 10:54:43.019214 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1c6d4ee-97ac-4b74-a2c1-1b9d84f0c93e admin admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 10:54:43.031303 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1c6d4ee-97ac-4b74-a2c1-1b9d84f0c93e admin admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 10:54:43.037039 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5740fdd7-824a-4194-90b5-2417c68fbf0b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5ac47e3b09f43d19a23a0f457fe9e23', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 10:54:43.037664 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5740fdd7-824a-4194-90b5-2417c68fbf0b admin admin] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:54:43.037947 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5740fdd7-824a-4194-90b5-2417c68fbf0b admin admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:54:43.038224 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5740fdd7-824a-4194-90b5-2417c68fbf0b admin admin] PATH_INFO: `/v3/roles` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:54:43.038872 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5740fdd7-824a-4194-90b5-2417c68fbf0b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 10:54:43.040549 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5740fdd7-824a-4194-90b5-2417c68fbf0b admin admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 10:54:43.051030 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 921/1812] 10.222.0.25 () {58 vars in 1122 bytes} [Sun Aug 7 10:54:43 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 07 10:54:43.055179 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3c7899d-de55-4e68-b1c5-385c4f040772 admin admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 10:54:43.066576 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3c7899d-de55-4e68-b1c5-385c4f040772 admin admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 10:54:43.071551 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7a288d4-4dd4-402e-99e9-a4734bf23d3c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5ac47e3b09f43d19a23a0f457fe9e23', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 10:54:43.072223 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7a288d4-4dd4-402e-99e9-a4734bf23d3c admin admin] REQUEST_METHOD: `PUT` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:54:43.072642 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7a288d4-4dd4-402e-99e9-a4734bf23d3c admin admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:54:43.072926 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7a288d4-4dd4-402e-99e9-a4734bf23d3c admin admin] PATH_INFO: `/v3/projects/a1cfdb59966d40c08edb158c6dacae5a/users/0483154330904c508e417398adf42130/roles/81b1f1bcee2a473088fbbd327c1c5b23` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:54:43.090944 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7a288d4-4dd4-402e-99e9-a4734bf23d3c admin admin] RBAC: Authorizing `identity:create_grant(project_id=a1cfdb59966d40c08edb158c6dacae5a, user_id=0483154330904c508e417398adf42130, role_id=81b1f1bcee2a473088fbbd327c1c5b23)` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 10:54:43.093642 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7a288d4-4dd4-402e-99e9-a4734bf23d3c admin admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 10:54:43.111961 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 892/1813] 10.222.0.25 () {60 vars in 1483 bytes} [Sun Aug 7 10:54:43 2022] PUT /identity/v3/projects/a1cfdb59966d40c08edb158c6dacae5a/users/0483154330904c508e417398adf42130/roles/81b1f1bcee2a473088fbbd327c1c5b23 => generated 0 bytes in 57 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 07 10:54:43.122766 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5740fdd7-824a-4194-90b5-2417c68fbf0b admin admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 10:54:43.133671 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5740fdd7-824a-4194-90b5-2417c68fbf0b admin admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 10:54:43.192417 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15c77746-3190-4c43-9e39-87b460969ffc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5ac47e3b09f43d19a23a0f457fe9e23', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 10:54:43.193035 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15c77746-3190-4c43-9e39-87b460969ffc admin admin] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:54:43.193339 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15c77746-3190-4c43-9e39-87b460969ffc admin admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:54:43.193619 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15c77746-3190-4c43-9e39-87b460969ffc admin admin] PATH_INFO: `/v3/roles` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:54:43.194296 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15c77746-3190-4c43-9e39-87b460969ffc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 10:54:43.195910 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15c77746-3190-4c43-9e39-87b460969ffc admin admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 10:54:43.207545 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 922/1814] 10.222.0.25 () {58 vars in 1122 bytes} [Sun Aug 7 10:54:43 2022] GET /identity/v3/roles => generated 1420 bytes in 85 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 07 10:54:43.211553 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7a288d4-4dd4-402e-99e9-a4734bf23d3c admin admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 10:54:43.222757 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7a288d4-4dd4-402e-99e9-a4734bf23d3c admin admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 10:54:43.228115 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c86d12f-b26c-4da4-b380-b1c5df1651b3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5ac47e3b09f43d19a23a0f457fe9e23', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 10:54:43.228776 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c86d12f-b26c-4da4-b380-b1c5df1651b3 admin admin] REQUEST_METHOD: `PUT` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:54:43.229059 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c86d12f-b26c-4da4-b380-b1c5df1651b3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:54:43.229330 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c86d12f-b26c-4da4-b380-b1c5df1651b3 admin admin] PATH_INFO: `/v3/projects/a1cfdb59966d40c08edb158c6dacae5a/users/0483154330904c508e417398adf42130/roles/81b1f1bcee2a473088fbbd327c1c5b23` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:54:43.231488 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c86d12f-b26c-4da4-b380-b1c5df1651b3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a1cfdb59966d40c08edb158c6dacae5a, user_id=0483154330904c508e417398adf42130, role_id=81b1f1bcee2a473088fbbd327c1c5b23)` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 10:54:43.234533 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c86d12f-b26c-4da4-b380-b1c5df1651b3 admin admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 10:54:43.247546 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 893/1815] 10.222.0.25 () {60 vars in 1483 bytes} [Sun Aug 7 10:54:43 2022] PUT /identity/v3/projects/a1cfdb59966d40c08edb158c6dacae5a/users/0483154330904c508e417398adf42130/roles/81b1f1bcee2a473088fbbd327c1c5b23 => generated 0 bytes in 37 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 07 10:54:44.073111 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2bb6a7b-f8bc-4b00-a375-9613ec148c30 None None] REQUEST_METHOD: `POST` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:54:44.074141 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2bb6a7b-f8bc-4b00-a375-9613ec148c30 None None] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:54:44.074644 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2bb6a7b-f8bc-4b00-a375-9613ec148c30 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:54:44.106793 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.auth.core [None req-d2bb6a7b-f8bc-4b00-a375-9613ec148c30 None None] MFA Rules not processed for user `0483154330904c508e417398adf42130`. Rule list: `[]` (Enabled: `True`). {{(pid=80144) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 10:54:44.107896 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.fernet_utils [None req-d2bb6a7b-f8bc-4b00-a375-9613ec148c30 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80144) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 10:54:44.113175 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 10:54:44.209849 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 923/1816] 10.222.0.25 () {58 vars in 958 bytes} [Sun Aug 7 10:54:44 2022] POST /identity/v3/auth/tokens => generated 3349 bytes in 138 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 07 10:54:44.725817 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c86d12f-b26c-4da4-b380-b1c5df1651b3 admin admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 10:54:44.793970 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 10:54:44.798722 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c86d12f-b26c-4da4-b380-b1c5df1651b3 admin admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 10:54:44.805650 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3ac3667-88d9-49fd-9c0e-f3327a7098b7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5ac47e3b09f43d19a23a0f457fe9e23', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 10:54:44.806143 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3ac3667-88d9-49fd-9c0e-f3327a7098b7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:54:44.806617 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3ac3667-88d9-49fd-9c0e-f3327a7098b7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:54:44.807006 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3ac3667-88d9-49fd-9c0e-f3327a7098b7 admin admin] PATH_INFO: `/v3/users/83b543260da44b868c0634e0fa0c8234` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:54:44.808182 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3ac3667-88d9-49fd-9c0e-f3327a7098b7 admin admin] RBAC: Authorizing `identity:delete_user(user_id=83b543260da44b868c0634e0fa0c8234)` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 10:54:44.810110 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3ac3667-88d9-49fd-9c0e-f3327a7098b7 admin admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 10:54:44.861920 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-b3ac3667-88d9-49fd-9c0e-f3327a7098b7 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '83b543260da44b868c0634e0fa0c8234'} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 10:54:44.868807 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-b3ac3667-88d9-49fd-9c0e-f3327a7098b7 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '83b543260da44b868c0634e0fa0c8234'} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 10:54:44.882301 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-b3ac3667-88d9-49fd-9c0e-f3327a7098b7 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '83b543260da44b868c0634e0fa0c8234'} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 10:54:44.883236 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-b3ac3667-88d9-49fd-9c0e-f3327a7098b7 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '83b543260da44b868c0634e0fa0c8234'} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 10:54:44.892543 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-b3ac3667-88d9-49fd-9c0e-f3327a7098b7 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '83b543260da44b868c0634e0fa0c8234'} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 10:54:44.902119 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 894/1817] 10.222.0.25 () {58 vars in 1224 bytes} [Sun Aug 7 10:54:44 2022] DELETE /identity/v3/users/83b543260da44b868c0634e0fa0c8234 => generated 0 bytes in 174 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 07 10:54:44.904041 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7616a500-b408-41b7-a17e-9a80ebf96676 None None] REQUEST_METHOD: `POST` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:54:44.904332 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7616a500-b408-41b7-a17e-9a80ebf96676 None None] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:54:44.904972 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7616a500-b408-41b7-a17e-9a80ebf96676 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:54:44.945881 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.auth.core [None req-7616a500-b408-41b7-a17e-9a80ebf96676 None None] MFA Rules not processed for user `bac4e6b76fe64217ac8fa44e71cffe6a`. Rule list: `[]` (Enabled: `True`). {{(pid=80144) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 10:54:44.947029 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.fernet_utils [None req-7616a500-b408-41b7-a17e-9a80ebf96676 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80144) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 10:54:45.004284 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14bfc557-90d4-451f-9139-8479af76b88d None None] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:54:45.005484 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14bfc557-90d4-451f-9139-8479af76b88d None None] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:54:45.008296 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14bfc557-90d4-451f-9139-8479af76b88d None None] PATH_INFO: `/` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:54:45.015538 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 895/1818] 10.222.0.25 () {54 vars in 921 bytes} [Sun Aug 7 10:54:44 2022] GET /identity => generated 270 bytes in 18 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 07 10:54:45.041813 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ba595c8-73b0-4597-a01c-0d43b2a62f8d None None] REQUEST_METHOD: `POST` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:54:45.041813 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ba595c8-73b0-4597-a01c-0d43b2a62f8d None None] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:54:45.044680 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ba595c8-73b0-4597-a01c-0d43b2a62f8d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:54:45.061360 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 924/1819] 10.222.0.25 () {58 vars in 958 bytes} [Sun Aug 7 10:54:44 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 156 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 07 10:54:45.072052 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7616a500-b408-41b7-a17e-9a80ebf96676 None None] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 10:54:45.073664 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.fernet_utils [None req-7616a500-b408-41b7-a17e-9a80ebf96676 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80144) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 10:54:45.080950 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.auth.core [None req-5ba595c8-73b0-4597-a01c-0d43b2a62f8d None None] MFA Rules not processed for user `6ee2e4c08af04c219ed395d7ca7085a6`. Rule list: `[]` (Enabled: `True`). {{(pid=80143) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 10:54:45.082615 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-5ba595c8-73b0-4597-a01c-0d43b2a62f8d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 10:54:45.211536 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 896/1820] 10.222.0.25 () {58 vars in 1018 bytes} [Sun Aug 7 10:54:45 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 07 10:54:45.229695 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7616a500-b408-41b7-a17e-9a80ebf96676 None None] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 10:54:45.230948 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ba595c8-73b0-4597-a01c-0d43b2a62f8d None None] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 10:54:45.233810 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-5ba595c8-73b0-4597-a01c-0d43b2a62f8d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 10:54:45.236736 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31432a29-31a2-4f80-bdaf-0e43f8bbc66e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be1283c9bf47424c9aa3ad27cab03027', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6de1f3cac02c4bcdb05dd2337176033d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 10:54:45.237577 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31432a29-31a2-4f80-bdaf-0e43f8bbc66e service neutron] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:54:45.238004 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31432a29-31a2-4f80-bdaf-0e43f8bbc66e service neutron] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:54:45.239307 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31432a29-31a2-4f80-bdaf-0e43f8bbc66e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:54:45.249970 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31432a29-31a2-4f80-bdaf-0e43f8bbc66e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 10:54:45.251742 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31432a29-31a2-4f80-bdaf-0e43f8bbc66e service neutron] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 10:54:45.279431 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 925/1821] 10.222.0.25 () {60 vars in 1365 bytes} [Sun Aug 7 10:54:45 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 208 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 07 10:54:45.280605 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ba595c8-73b0-4597-a01c-0d43b2a62f8d None None] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 10:54:45.288466 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a53f019e-f516-4864-8248-cd72f59c0c56 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6ee2e4c08af04c219ed395d7ca7085a6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6de1f3cac02c4bcdb05dd2337176033d', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 10:54:45.289145 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a53f019e-f516-4864-8248-cd72f59c0c56 service placement] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:54:45.289425 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a53f019e-f516-4864-8248-cd72f59c0c56 service placement] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:54:45.289693 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a53f019e-f516-4864-8248-cd72f59c0c56 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:54:45.302360 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a53f019e-f516-4864-8248-cd72f59c0c56 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 10:54:45.302360 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a53f019e-f516-4864-8248-cd72f59c0c56 service placement] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 10:54:45.312357 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 897/1822] 10.222.0.25 () {60 vars in 1365 bytes} [Sun Aug 7 10:54:45 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 82 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 07 10:54:45.779152 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31432a29-31a2-4f80-bdaf-0e43f8bbc66e service neutron] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 10:54:45.780956 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.fernet_utils [None req-31432a29-31a2-4f80-bdaf-0e43f8bbc66e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80144) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 10:54:45.803967 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31432a29-31a2-4f80-bdaf-0e43f8bbc66e service neutron] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 10:54:45.809638 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be19d7b2-fd98-4bb5-8f72-8271211407e0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5ac47e3b09f43d19a23a0f457fe9e23', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 10:54:45.811590 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be19d7b2-fd98-4bb5-8f72-8271211407e0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:54:45.812065 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be19d7b2-fd98-4bb5-8f72-8271211407e0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:54:45.812908 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be19d7b2-fd98-4bb5-8f72-8271211407e0 admin admin] PATH_INFO: `/v3/projects/811730b82a534181babbef168ca91049` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:54:45.815902 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be19d7b2-fd98-4bb5-8f72-8271211407e0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=811730b82a534181babbef168ca91049)` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 10:54:45.818041 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be19d7b2-fd98-4bb5-8f72-8271211407e0 admin admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 10:54:45.869105 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=97,keystone:DELETE=3 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 10:54:45.877393 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.notifications [None req-be19d7b2-fd98-4bb5-8f72-8271211407e0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '811730b82a534181babbef168ca91049', 'request_id': 'req-be19d7b2-fd98-4bb5-8f72-8271211407e0'} {{(pid=80144) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 10:54:45.884026 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.notifications [None req-be19d7b2-fd98-4bb5-8f72-8271211407e0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '811730b82a534181babbef168ca91049', 'request_id': 'req-be19d7b2-fd98-4bb5-8f72-8271211407e0'} {{(pid=80144) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 10:54:45.894384 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.notifications [None req-be19d7b2-fd98-4bb5-8f72-8271211407e0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '811730b82a534181babbef168ca91049', 'request_id': 'req-be19d7b2-fd98-4bb5-8f72-8271211407e0'} {{(pid=80144) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 10:54:45.908418 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.notifications [None req-be19d7b2-fd98-4bb5-8f72-8271211407e0 admin admin] The token cache is being invalidate because project 811730b82a534181babbef168ca91049 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80144) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 07 10:54:45.908776 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.notifications [None req-be19d7b2-fd98-4bb5-8f72-8271211407e0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80144) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 10:54:45.909512 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.notifications [None req-be19d7b2-fd98-4bb5-8f72-8271211407e0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80144) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 10:54:45.911233 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 926/1823] 10.222.0.25 () {58 vars in 1233 bytes} [Sun Aug 7 10:54:45 2022] DELETE /identity/v3/projects/811730b82a534181babbef168ca91049 => generated 0 bytes in 133 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 07 10:54:45.915478 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a53f019e-f516-4864-8248-cd72f59c0c56 service placement] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 10:54:45.927675 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a53f019e-f516-4864-8248-cd72f59c0c56 service placement] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 10:54:45.948916 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd98172b-aede-4684-b66d-9384c5cb141b None None] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:54:45.949207 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd98172b-aede-4684-b66d-9384c5cb141b None None] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:54:45.949503 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd98172b-aede-4684-b66d-9384c5cb141b None None] PATH_INFO: `/` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:54:45.950821 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 927/1824] 10.222.0.25 () {54 vars in 921 bytes} [Sun Aug 7 10:54:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 07 10:54:45.956017 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a39c3464-e8e5-42fb-a2e8-f1424e6cc35b None None] REQUEST_METHOD: `POST` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:54:45.956244 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a39c3464-e8e5-42fb-a2e8-f1424e6cc35b None None] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:54:45.956594 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a39c3464-e8e5-42fb-a2e8-f1424e6cc35b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:54:45.992040 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.auth.core [None req-a39c3464-e8e5-42fb-a2e8-f1424e6cc35b None None] MFA Rules not processed for user `6ee2e4c08af04c219ed395d7ca7085a6`. Rule list: `[]` (Enabled: `True`). {{(pid=80144) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 10:54:45.993164 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.fernet_utils [None req-a39c3464-e8e5-42fb-a2e8-f1424e6cc35b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80144) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 10:54:46.001681 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f781aacb-13a3-42e4-b3ef-90dc4fe7a9bd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5ac47e3b09f43d19a23a0f457fe9e23', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 10:54:46.002469 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f781aacb-13a3-42e4-b3ef-90dc4fe7a9bd admin admin] REQUEST_METHOD: `DELETE` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:54:46.002849 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f781aacb-13a3-42e4-b3ef-90dc4fe7a9bd admin admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:54:46.003270 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f781aacb-13a3-42e4-b3ef-90dc4fe7a9bd admin admin] PATH_INFO: `/v3/users/3119b07d13104006bbf63ec995eabbc9` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:54:46.008210 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f781aacb-13a3-42e4-b3ef-90dc4fe7a9bd admin admin] RBAC: Authorizing `identity:delete_user(user_id=3119b07d13104006bbf63ec995eabbc9)` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 10:54:46.010779 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f781aacb-13a3-42e4-b3ef-90dc4fe7a9bd admin admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 10:54:46.013106 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 10:54:46.106744 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 928/1825] 10.222.0.25 () {58 vars in 1018 bytes} [Sun Aug 7 10:54:45 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 152 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 07 10:54:46.111104 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-f781aacb-13a3-42e4-b3ef-90dc4fe7a9bd admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3119b07d13104006bbf63ec995eabbc9'} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 10:54:46.128334 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-f781aacb-13a3-42e4-b3ef-90dc4fe7a9bd admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3119b07d13104006bbf63ec995eabbc9'} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 10:54:46.146647 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-f781aacb-13a3-42e4-b3ef-90dc4fe7a9bd admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3119b07d13104006bbf63ec995eabbc9'} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 10:54:46.148046 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-f781aacb-13a3-42e4-b3ef-90dc4fe7a9bd admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3119b07d13104006bbf63ec995eabbc9'} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 10:54:46.157835 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-f781aacb-13a3-42e4-b3ef-90dc4fe7a9bd admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3119b07d13104006bbf63ec995eabbc9'} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 10:54:46.168233 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 898/1826] 10.222.0.25 () {58 vars in 1224 bytes} [Sun Aug 7 10:54:45 2022] DELETE /identity/v3/users/3119b07d13104006bbf63ec995eabbc9 => generated 0 bytes in 253 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 07 10:54:46.227477 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a39c3464-e8e5-42fb-a2e8-f1424e6cc35b None None] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 10:54:46.229333 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.fernet_utils [None req-a39c3464-e8e5-42fb-a2e8-f1424e6cc35b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80144) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 10:54:46.244146 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a39c3464-e8e5-42fb-a2e8-f1424e6cc35b None None] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 10:54:46.291883 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f781aacb-13a3-42e4-b3ef-90dc4fe7a9bd admin admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 10:54:46.293342 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-f781aacb-13a3-42e4-b3ef-90dc4fe7a9bd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 10:54:46.366427 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee6bd11f-8b63-4f3a-b859-ecf4a908b9a7 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6ee2e4c08af04c219ed395d7ca7085a6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6de1f3cac02c4bcdb05dd2337176033d', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 10:54:46.367174 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee6bd11f-8b63-4f3a-b859-ecf4a908b9a7 service placement] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:54:46.367770 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee6bd11f-8b63-4f3a-b859-ecf4a908b9a7 service placement] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:54:46.368014 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee6bd11f-8b63-4f3a-b859-ecf4a908b9a7 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:54:46.370205 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.fernet_utils [None req-ee6bd11f-8b63-4f3a-b859-ecf4a908b9a7 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80144) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 10:54:46.385799 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee6bd11f-8b63-4f3a-b859-ecf4a908b9a7 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 10:54:46.387797 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee6bd11f-8b63-4f3a-b859-ecf4a908b9a7 service placement] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 10:54:46.401925 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 929/1827] 10.222.0.25 () {60 vars in 1365 bytes} [Sun Aug 7 10:54:46 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 174 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 07 10:54:46.410975 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f781aacb-13a3-42e4-b3ef-90dc4fe7a9bd admin admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 10:54:46.442494 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21cd096e-325a-4909-bdfa-6cff64cfe91d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5ac47e3b09f43d19a23a0f457fe9e23', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 10:54:46.443282 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21cd096e-325a-4909-bdfa-6cff64cfe91d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:54:46.443592 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21cd096e-325a-4909-bdfa-6cff64cfe91d admin admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:54:46.443869 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21cd096e-325a-4909-bdfa-6cff64cfe91d admin admin] PATH_INFO: `/v3/users/8fb9d5b3e8524ad09786d0eeedfbd643` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:54:46.445702 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21cd096e-325a-4909-bdfa-6cff64cfe91d admin admin] RBAC: Authorizing `identity:delete_user(user_id=8fb9d5b3e8524ad09786d0eeedfbd643)` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 10:54:46.448375 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21cd096e-325a-4909-bdfa-6cff64cfe91d admin admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 10:54:46.506215 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-21cd096e-325a-4909-bdfa-6cff64cfe91d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8fb9d5b3e8524ad09786d0eeedfbd643'} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 10:54:46.514181 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-21cd096e-325a-4909-bdfa-6cff64cfe91d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8fb9d5b3e8524ad09786d0eeedfbd643'} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 10:54:46.532651 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-21cd096e-325a-4909-bdfa-6cff64cfe91d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8fb9d5b3e8524ad09786d0eeedfbd643'} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 10:54:46.534256 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-21cd096e-325a-4909-bdfa-6cff64cfe91d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8fb9d5b3e8524ad09786d0eeedfbd643'} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 10:54:46.546051 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-21cd096e-325a-4909-bdfa-6cff64cfe91d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8fb9d5b3e8524ad09786d0eeedfbd643'} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 10:54:46.548524 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=78,keystone:DELETE=20,keystone:INSERT=2 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 10:54:46.565545 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 899/1828] 10.222.0.25 () {58 vars in 1224 bytes} [Sun Aug 7 10:54:46 2022] DELETE /identity/v3/users/8fb9d5b3e8524ad09786d0eeedfbd643 => generated 0 bytes in 274 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 07 10:54:46.647028 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee6bd11f-8b63-4f3a-b859-ecf4a908b9a7 service placement] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 10:54:46.648832 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.fernet_utils [None req-ee6bd11f-8b63-4f3a-b859-ecf4a908b9a7 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80144) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 10:54:46.669216 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee6bd11f-8b63-4f3a-b859-ecf4a908b9a7 service placement] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 10:54:46.761843 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=95,keystone:DELETE=4,keystone:INSERT=1 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 10:54:46.777202 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d21ccef2-cd50-4c07-8208-7153a5ba139c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5ac47e3b09f43d19a23a0f457fe9e23', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 10:54:46.777998 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d21ccef2-cd50-4c07-8208-7153a5ba139c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:54:46.778406 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d21ccef2-cd50-4c07-8208-7153a5ba139c admin admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:54:46.779163 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d21ccef2-cd50-4c07-8208-7153a5ba139c admin admin] PATH_INFO: `/v3/projects/86d8cd42509443aca5ed02397336e0be` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:54:46.780216 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d21ccef2-cd50-4c07-8208-7153a5ba139c admin admin] RBAC: Authorizing `identity:delete_project(project_id=86d8cd42509443aca5ed02397336e0be)` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 10:54:46.783050 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d21ccef2-cd50-4c07-8208-7153a5ba139c admin admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 10:54:46.866368 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.notifications [None req-d21ccef2-cd50-4c07-8208-7153a5ba139c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '86d8cd42509443aca5ed02397336e0be', 'request_id': 'req-d21ccef2-cd50-4c07-8208-7153a5ba139c'} {{(pid=80144) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 10:54:46.874316 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.notifications [None req-d21ccef2-cd50-4c07-8208-7153a5ba139c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '86d8cd42509443aca5ed02397336e0be', 'request_id': 'req-d21ccef2-cd50-4c07-8208-7153a5ba139c'} {{(pid=80144) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 10:54:46.881365 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21cd096e-325a-4909-bdfa-6cff64cfe91d admin admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 10:54:46.883002 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-21cd096e-325a-4909-bdfa-6cff64cfe91d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 10:54:46.916188 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21cd096e-325a-4909-bdfa-6cff64cfe91d admin admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 10:54:46.928303 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.notifications [None req-d21ccef2-cd50-4c07-8208-7153a5ba139c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '86d8cd42509443aca5ed02397336e0be', 'request_id': 'req-d21ccef2-cd50-4c07-8208-7153a5ba139c'} {{(pid=80144) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 10:54:46.958649 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.notifications [None req-d21ccef2-cd50-4c07-8208-7153a5ba139c admin admin] The token cache is being invalidate because project 86d8cd42509443aca5ed02397336e0be was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80144) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 07 10:54:46.959351 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.notifications [None req-d21ccef2-cd50-4c07-8208-7153a5ba139c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80144) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 10:54:46.960292 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.notifications [None req-d21ccef2-cd50-4c07-8208-7153a5ba139c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80144) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 10:54:46.966318 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 930/1829] 10.222.0.25 () {58 vars in 1233 bytes} [Sun Aug 7 10:54:46 2022] DELETE /identity/v3/projects/86d8cd42509443aca5ed02397336e0be => generated 0 bytes in 316 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 07 10:54:46.968270 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eefb8397-f1fa-403b-a333-68ae769c6f02 None None] REQUEST_METHOD: `POST` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:54:46.968761 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eefb8397-f1fa-403b-a333-68ae769c6f02 None None] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:54:46.969052 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eefb8397-f1fa-403b-a333-68ae769c6f02 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:54:46.996555 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.auth.core [None req-eefb8397-f1fa-403b-a333-68ae769c6f02 None None] MFA Rules not processed for user `bac4e6b76fe64217ac8fa44e71cffe6a`. Rule list: `[]` (Enabled: `True`). {{(pid=80144) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 10:54:46.997661 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.fernet_utils [None req-eefb8397-f1fa-403b-a333-68ae769c6f02 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80144) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 10:54:47.028289 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf259d25-0978-4ca4-9eb1-b722980286a2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5ac47e3b09f43d19a23a0f457fe9e23', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 10:54:47.029662 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf259d25-0978-4ca4-9eb1-b722980286a2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:54:47.030677 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf259d25-0978-4ca4-9eb1-b722980286a2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:54:47.031617 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf259d25-0978-4ca4-9eb1-b722980286a2 admin admin] PATH_INFO: `/v3/projects/40437563bdb1443589268c67cc9e6c11` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:54:47.033757 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf259d25-0978-4ca4-9eb1-b722980286a2 admin admin] RBAC: Authorizing `identity:delete_project(project_id=40437563bdb1443589268c67cc9e6c11)` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 10:54:47.037546 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf259d25-0978-4ca4-9eb1-b722980286a2 admin admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 10:54:47.074519 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 931/1830] 10.222.0.25 () {58 vars in 958 bytes} [Sun Aug 7 10:54:46 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 07 10:54:47.085122 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ea20355-a1e0-4769-a59c-1c0ec16226e9 None None] REQUEST_METHOD: `POST` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:54:47.085676 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ea20355-a1e0-4769-a59c-1c0ec16226e9 None None] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:54:47.086038 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ea20355-a1e0-4769-a59c-1c0ec16226e9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:54:47.109149 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.auth.core [None req-8ea20355-a1e0-4769-a59c-1c0ec16226e9 None None] MFA Rules not processed for user `bac4e6b76fe64217ac8fa44e71cffe6a`. Rule list: `[]` (Enabled: `True`). {{(pid=80144) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 10:54:47.111006 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.fernet_utils [None req-8ea20355-a1e0-4769-a59c-1c0ec16226e9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80144) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 10:54:47.113690 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-bf259d25-0978-4ca4-9eb1-b722980286a2 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '40437563bdb1443589268c67cc9e6c11', 'request_id': 'req-bf259d25-0978-4ca4-9eb1-b722980286a2'} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 10:54:47.122135 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-bf259d25-0978-4ca4-9eb1-b722980286a2 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '40437563bdb1443589268c67cc9e6c11', 'request_id': 'req-bf259d25-0978-4ca4-9eb1-b722980286a2'} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 10:54:47.130363 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-bf259d25-0978-4ca4-9eb1-b722980286a2 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '40437563bdb1443589268c67cc9e6c11', 'request_id': 'req-bf259d25-0978-4ca4-9eb1-b722980286a2'} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 10:54:47.151687 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-bf259d25-0978-4ca4-9eb1-b722980286a2 admin admin] The token cache is being invalidate because project 40437563bdb1443589268c67cc9e6c11 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80143) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 07 10:54:47.159599 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-bf259d25-0978-4ca4-9eb1-b722980286a2 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 10:54:47.161136 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-bf259d25-0978-4ca4-9eb1-b722980286a2 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 10:54:47.163812 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 900/1831] 10.222.0.25 () {58 vars in 1233 bytes} [Sun Aug 7 10:54:46 2022] DELETE /identity/v3/projects/40437563bdb1443589268c67cc9e6c11 => generated 0 bytes in 283 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 07 10:54:47.168336 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf259d25-0978-4ca4-9eb1-b722980286a2 admin admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 10:54:47.170232 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-bf259d25-0978-4ca4-9eb1-b722980286a2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 10:54:47.208370 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 932/1832] 10.222.0.25 () {58 vars in 958 bytes} [Sun Aug 7 10:54:47 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 07 10:54:47.212326 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ea20355-a1e0-4769-a59c-1c0ec16226e9 None None] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 10:54:47.226417 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ea20355-a1e0-4769-a59c-1c0ec16226e9 None None] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 10:54:47.253311 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e467c88d-9e1f-4166-948f-528956a9e75c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5ac47e3b09f43d19a23a0f457fe9e23', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 10:54:47.254984 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e467c88d-9e1f-4166-948f-528956a9e75c admin admin] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:54:47.255488 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e467c88d-9e1f-4166-948f-528956a9e75c admin admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:54:47.257528 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e467c88d-9e1f-4166-948f-528956a9e75c admin admin] PATH_INFO: `/v3/domains` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:54:47.258416 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e467c88d-9e1f-4166-948f-528956a9e75c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80144) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 10:54:47.259619 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e467c88d-9e1f-4166-948f-528956a9e75c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 10:54:47.263510 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e467c88d-9e1f-4166-948f-528956a9e75c admin admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 10:54:47.299445 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 933/1833] 10.222.0.25 () {58 vars in 1153 bytes} [Sun Aug 7 10:54:47 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 88 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 07 10:54:47.304290 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e467c88d-9e1f-4166-948f-528956a9e75c admin admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 10:54:47.309122 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf259d25-0978-4ca4-9eb1-b722980286a2 admin admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 10:54:47.322593 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e467c88d-9e1f-4166-948f-528956a9e75c admin admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 10:54:47.325214 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0a291ba-8773-446f-8ae9-4d4714d6500d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5ac47e3b09f43d19a23a0f457fe9e23', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 10:54:47.325923 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0a291ba-8773-446f-8ae9-4d4714d6500d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:54:47.326528 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0a291ba-8773-446f-8ae9-4d4714d6500d admin admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:54:47.326528 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0a291ba-8773-446f-8ae9-4d4714d6500d admin admin] PATH_INFO: `/v3/users/0483154330904c508e417398adf42130` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:54:47.330326 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0a291ba-8773-446f-8ae9-4d4714d6500d admin admin] RBAC: Authorizing `identity:delete_user(user_id=0483154330904c508e417398adf42130)` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 10:54:47.332400 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0a291ba-8773-446f-8ae9-4d4714d6500d admin admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 10:54:47.340030 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b244dcb-8c30-47ee-ad73-87520821e7a4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5ac47e3b09f43d19a23a0f457fe9e23', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 10:54:47.340788 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b244dcb-8c30-47ee-ad73-87520821e7a4 admin admin] REQUEST_METHOD: `POST` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:54:47.341123 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b244dcb-8c30-47ee-ad73-87520821e7a4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:54:47.341405 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b244dcb-8c30-47ee-ad73-87520821e7a4 admin admin] PATH_INFO: `/v3/projects` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:54:47.342043 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b244dcb-8c30-47ee-ad73-87520821e7a4 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 10:54:47.345764 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b244dcb-8c30-47ee-ad73-87520821e7a4 admin admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 10:54:47.366407 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 10:54:47.392893 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 934/1834] 10.222.0.25 () {60 vars in 1153 bytes} [Sun Aug 7 10:54:47 2022] POST /identity/v3/projects => generated 377 bytes in 89 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 07 10:54:47.396912 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b244dcb-8c30-47ee-ad73-87520821e7a4 admin admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 10:54:47.413904 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 10:54:47.429118 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-e0a291ba-8773-446f-8ae9-4d4714d6500d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0483154330904c508e417398adf42130'} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 10:54:47.438746 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-e0a291ba-8773-446f-8ae9-4d4714d6500d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0483154330904c508e417398adf42130'} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 10:54:47.456763 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-e0a291ba-8773-446f-8ae9-4d4714d6500d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0483154330904c508e417398adf42130'} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 10:54:47.457541 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-e0a291ba-8773-446f-8ae9-4d4714d6500d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0483154330904c508e417398adf42130'} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 10:54:47.478957 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-e0a291ba-8773-446f-8ae9-4d4714d6500d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0483154330904c508e417398adf42130'} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 10:54:47.511389 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 901/1835] 10.222.0.25 () {58 vars in 1224 bytes} [Sun Aug 7 10:54:47 2022] DELETE /identity/v3/users/0483154330904c508e417398adf42130 => generated 0 bytes in 329 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 07 10:54:47.596886 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b244dcb-8c30-47ee-ad73-87520821e7a4 admin admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 10:54:47.745375 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0a291ba-8773-446f-8ae9-4d4714d6500d admin admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 10:54:47.747958 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-e0a291ba-8773-446f-8ae9-4d4714d6500d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 10:54:47.779806 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-848aa3df-abac-4d64-8e75-aa8d46211a77 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5ac47e3b09f43d19a23a0f457fe9e23', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 10:54:47.785896 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-848aa3df-abac-4d64-8e75-aa8d46211a77 admin admin] REQUEST_METHOD: `POST` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:54:47.786733 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-848aa3df-abac-4d64-8e75-aa8d46211a77 admin admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:54:47.787352 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-848aa3df-abac-4d64-8e75-aa8d46211a77 admin admin] PATH_INFO: `/v3/users` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:54:47.788175 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-848aa3df-abac-4d64-8e75-aa8d46211a77 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 10:54:47.790294 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-848aa3df-abac-4d64-8e75-aa8d46211a77 admin admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 10:54:47.827156 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 935/1836] 10.222.0.25 () {60 vars in 1144 bytes} [Sun Aug 7 10:54:47 2022] POST /identity/v3/users => generated 289 bytes in 427 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 07 10:54:47.827156 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-848aa3df-abac-4d64-8e75-aa8d46211a77 admin admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 10:54:47.830807 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.fernet_utils [None req-848aa3df-abac-4d64-8e75-aa8d46211a77 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80144) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 10:54:47.854537 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-848aa3df-abac-4d64-8e75-aa8d46211a77 admin admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 10:54:47.861637 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0a291ba-8773-446f-8ae9-4d4714d6500d admin admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 10:54:47.863120 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f5db9c9-7ce6-4d27-b03c-187cd51dff4a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5ac47e3b09f43d19a23a0f457fe9e23', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 10:54:47.863799 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f5db9c9-7ce6-4d27-b03c-187cd51dff4a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:54:47.864332 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f5db9c9-7ce6-4d27-b03c-187cd51dff4a admin admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:54:47.866155 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f5db9c9-7ce6-4d27-b03c-187cd51dff4a admin admin] PATH_INFO: `/v3/users/867775064a12466aa27f981b2c90e285` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:54:47.867389 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f5db9c9-7ce6-4d27-b03c-187cd51dff4a admin admin] RBAC: Authorizing `identity:delete_user(user_id=867775064a12466aa27f981b2c90e285)` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 10:54:47.870336 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f5db9c9-7ce6-4d27-b03c-187cd51dff4a admin admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 10:54:47.872930 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee5bdf0f-eb1e-44b2-9b71-e2dd1616f681 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5ac47e3b09f43d19a23a0f457fe9e23', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 10:54:47.874080 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee5bdf0f-eb1e-44b2-9b71-e2dd1616f681 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:54:47.874376 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee5bdf0f-eb1e-44b2-9b71-e2dd1616f681 admin admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:54:47.874656 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee5bdf0f-eb1e-44b2-9b71-e2dd1616f681 admin admin] PATH_INFO: `/v3/projects/a1cfdb59966d40c08edb158c6dacae5a` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:54:47.876844 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee5bdf0f-eb1e-44b2-9b71-e2dd1616f681 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a1cfdb59966d40c08edb158c6dacae5a)` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 10:54:47.879988 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee5bdf0f-eb1e-44b2-9b71-e2dd1616f681 admin admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 10:54:47.948780 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.notifications [None req-2f5db9c9-7ce6-4d27-b03c-187cd51dff4a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '867775064a12466aa27f981b2c90e285'} {{(pid=80144) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 10:54:47.955891 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-ee5bdf0f-eb1e-44b2-9b71-e2dd1616f681 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a1cfdb59966d40c08edb158c6dacae5a', 'request_id': 'req-ee5bdf0f-eb1e-44b2-9b71-e2dd1616f681'} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 10:54:47.958295 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.notifications [None req-2f5db9c9-7ce6-4d27-b03c-187cd51dff4a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '867775064a12466aa27f981b2c90e285'} {{(pid=80144) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 10:54:47.963282 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-ee5bdf0f-eb1e-44b2-9b71-e2dd1616f681 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a1cfdb59966d40c08edb158c6dacae5a', 'request_id': 'req-ee5bdf0f-eb1e-44b2-9b71-e2dd1616f681'} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 10:54:47.970617 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.notifications [None req-2f5db9c9-7ce6-4d27-b03c-187cd51dff4a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '867775064a12466aa27f981b2c90e285'} {{(pid=80144) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 10:54:47.971459 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-ee5bdf0f-eb1e-44b2-9b71-e2dd1616f681 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a1cfdb59966d40c08edb158c6dacae5a', 'request_id': 'req-ee5bdf0f-eb1e-44b2-9b71-e2dd1616f681'} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 10:54:47.971839 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.notifications [None req-2f5db9c9-7ce6-4d27-b03c-187cd51dff4a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '867775064a12466aa27f981b2c90e285'} {{(pid=80144) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 10:54:47.972698 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.notifications [None req-2f5db9c9-7ce6-4d27-b03c-187cd51dff4a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '867775064a12466aa27f981b2c90e285'} {{(pid=80144) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 10:54:47.986005 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-ee5bdf0f-eb1e-44b2-9b71-e2dd1616f681 admin admin] The token cache is being invalidate because project a1cfdb59966d40c08edb158c6dacae5a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80143) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 07 10:54:47.986304 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-ee5bdf0f-eb1e-44b2-9b71-e2dd1616f681 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 10:54:47.987443 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-ee5bdf0f-eb1e-44b2-9b71-e2dd1616f681 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 10:54:47.989176 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 902/1837] 10.222.0.25 () {58 vars in 1233 bytes} [Sun Aug 7 10:54:47 2022] DELETE /identity/v3/projects/a1cfdb59966d40c08edb158c6dacae5a => generated 0 bytes in 245 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 07 10:54:47.992634 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee5bdf0f-eb1e-44b2-9b71-e2dd1616f681 admin admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 10:54:47.992744 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=85,keystone:INSERT=5,keystone:DELETE=10 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 10:54:47.993861 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-ee5bdf0f-eb1e-44b2-9b71-e2dd1616f681 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 10:54:47.997591 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 936/1838] 10.222.0.25 () {58 vars in 1224 bytes} [Sun Aug 7 10:54:47 2022] DELETE /identity/v3/users/867775064a12466aa27f981b2c90e285 => generated 0 bytes in 174 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 07 10:54:47.999516 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bac3b47-afbd-411a-b76a-f1134ef62bf3 None None] REQUEST_METHOD: `POST` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:54:48.000171 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bac3b47-afbd-411a-b76a-f1134ef62bf3 None None] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:54:48.001968 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bac3b47-afbd-411a-b76a-f1134ef62bf3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:54:48.024080 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.auth.core [None req-0bac3b47-afbd-411a-b76a-f1134ef62bf3 None None] MFA Rules not processed for user `bac4e6b76fe64217ac8fa44e71cffe6a`. Rule list: `[]` (Enabled: `True`). {{(pid=80144) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 10:54:48.025300 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.fernet_utils [None req-0bac3b47-afbd-411a-b76a-f1134ef62bf3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80144) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 10:54:48.047229 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=86,keystone:DELETE=12,keystone:INSERT=2 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 10:54:48.080056 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee5bdf0f-eb1e-44b2-9b71-e2dd1616f681 admin admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 10:54:48.103110 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 937/1839] 10.222.0.25 () {58 vars in 958 bytes} [Sun Aug 7 10:54:47 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 07 10:54:48.104758 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a30270c-4062-4191-89f3-571902f4e0d5 None None] REQUEST_METHOD: `POST` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:54:48.105054 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a30270c-4062-4191-89f3-571902f4e0d5 None None] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:54:48.105482 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a30270c-4062-4191-89f3-571902f4e0d5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:54:48.124801 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.auth.core [None req-1a30270c-4062-4191-89f3-571902f4e0d5 None None] MFA Rules not processed for user `bac4e6b76fe64217ac8fa44e71cffe6a`. Rule list: `[]` (Enabled: `True`). {{(pid=80144) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 10:54:48.126628 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.fernet_utils [None req-1a30270c-4062-4191-89f3-571902f4e0d5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80144) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 10:54:48.134339 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 938/1840] 10.222.0.25 () {58 vars in 958 bytes} [Sun Aug 7 10:54:48 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 07 10:54:48.135553 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48c9db11-ec6d-4fd9-b767-a91c9d260803 None None] REQUEST_METHOD: `POST` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:54:48.135838 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48c9db11-ec6d-4fd9-b767-a91c9d260803 None None] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:54:48.136274 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48c9db11-ec6d-4fd9-b767-a91c9d260803 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:54:48.146530 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-105adab1-fab3-4560-9fd7-2e50559b33b0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5ac47e3b09f43d19a23a0f457fe9e23', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 10:54:48.147610 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-105adab1-fab3-4560-9fd7-2e50559b33b0 admin admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:54:48.148004 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-105adab1-fab3-4560-9fd7-2e50559b33b0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:54:48.148459 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-105adab1-fab3-4560-9fd7-2e50559b33b0 admin admin] PATH_INFO: `/v3/roles` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:54:48.149273 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-105adab1-fab3-4560-9fd7-2e50559b33b0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 10:54:48.151519 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-105adab1-fab3-4560-9fd7-2e50559b33b0 admin admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 10:54:48.157121 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.auth.core [None req-48c9db11-ec6d-4fd9-b767-a91c9d260803 None None] MFA Rules not processed for user `bac4e6b76fe64217ac8fa44e71cffe6a`. Rule list: `[]` (Enabled: `True`). {{(pid=80144) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 10:54:48.158655 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.fernet_utils [None req-48c9db11-ec6d-4fd9-b767-a91c9d260803 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80144) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 10:54:48.169720 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 903/1841] 10.222.0.25 () {58 vars in 1122 bytes} [Sun Aug 7 10:54:47 2022] GET /identity/v3/roles => generated 1420 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 07 10:54:48.169720 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-105adab1-fab3-4560-9fd7-2e50559b33b0 admin admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 10:54:48.170284 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 939/1842] 10.222.0.25 () {58 vars in 958 bytes} [Sun Aug 7 10:54:48 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 07 10:54:48.171576 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48c9db11-ec6d-4fd9-b767-a91c9d260803 None None] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 10:54:48.173239 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-105adab1-fab3-4560-9fd7-2e50559b33b0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 10:54:48.182908 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48c9db11-ec6d-4fd9-b767-a91c9d260803 None None] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 10:54:48.187832 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18c2635f-c7f5-4371-b51d-7f5f7ee8343f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5ac47e3b09f43d19a23a0f457fe9e23', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 10:54:48.189460 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18c2635f-c7f5-4371-b51d-7f5f7ee8343f admin admin] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:54:48.189460 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18c2635f-c7f5-4371-b51d-7f5f7ee8343f admin admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:54:48.189683 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18c2635f-c7f5-4371-b51d-7f5f7ee8343f admin admin] PATH_INFO: `/v3/roles` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:54:48.190299 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18c2635f-c7f5-4371-b51d-7f5f7ee8343f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 10:54:48.191918 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18c2635f-c7f5-4371-b51d-7f5f7ee8343f admin admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 10:54:48.202478 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 940/1843] 10.222.0.25 () {58 vars in 1122 bytes} [Sun Aug 7 10:54:48 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 07 10:54:48.203231 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18c2635f-c7f5-4371-b51d-7f5f7ee8343f admin admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 10:54:48.217694 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18c2635f-c7f5-4371-b51d-7f5f7ee8343f admin admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 10:54:48.223308 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a9dd7c8-736f-4e06-b711-1222631c2ce7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5ac47e3b09f43d19a23a0f457fe9e23', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 10:54:48.223945 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a9dd7c8-736f-4e06-b711-1222631c2ce7 admin admin] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:54:48.224231 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a9dd7c8-736f-4e06-b711-1222631c2ce7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:54:48.224503 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a9dd7c8-736f-4e06-b711-1222631c2ce7 admin admin] PATH_INFO: `/v3/domains` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:54:48.225298 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a9dd7c8-736f-4e06-b711-1222631c2ce7 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80144) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 10:54:48.225598 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a9dd7c8-736f-4e06-b711-1222631c2ce7 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 10:54:48.227237 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a9dd7c8-736f-4e06-b711-1222631c2ce7 admin admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 10:54:48.244855 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 941/1844] 10.222.0.25 () {58 vars in 1153 bytes} [Sun Aug 7 10:54:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 10:54:48.244855 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a9dd7c8-736f-4e06-b711-1222631c2ce7 admin admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 10:54:48.254931 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-105adab1-fab3-4560-9fd7-2e50559b33b0 admin admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 10:54:48.258990 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c3e5a8a-3477-409a-aee2-413db0776be3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be1283c9bf47424c9aa3ad27cab03027', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6de1f3cac02c4bcdb05dd2337176033d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 10:54:48.259701 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c3e5a8a-3477-409a-aee2-413db0776be3 service neutron] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:54:48.260048 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c3e5a8a-3477-409a-aee2-413db0776be3 service neutron] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:54:48.260386 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c3e5a8a-3477-409a-aee2-413db0776be3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:54:48.261644 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a9dd7c8-736f-4e06-b711-1222631c2ce7 admin admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 10:54:48.267692 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5463eef-e03a-49f5-aa33-25ba75fdb3cc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5ac47e3b09f43d19a23a0f457fe9e23', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 10:54:48.268366 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5463eef-e03a-49f5-aa33-25ba75fdb3cc admin admin] REQUEST_METHOD: `PUT` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:54:48.268708 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5463eef-e03a-49f5-aa33-25ba75fdb3cc admin admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:54:48.268993 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5463eef-e03a-49f5-aa33-25ba75fdb3cc admin admin] PATH_INFO: `/v3/projects/23e2b09610fd4ee0af16405257b2daf3/users/05a0d408980b49acafd2c9a93bf527f2/roles/b98dfb3b9fee4644af2887a3ac3fd2e6` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:54:48.271864 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c3e5a8a-3477-409a-aee2-413db0776be3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 10:54:48.273307 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c3e5a8a-3477-409a-aee2-413db0776be3 service neutron] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 10:54:48.288110 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5463eef-e03a-49f5-aa33-25ba75fdb3cc admin admin] RBAC: Authorizing `identity:create_grant(project_id=23e2b09610fd4ee0af16405257b2daf3, user_id=05a0d408980b49acafd2c9a93bf527f2, role_id=b98dfb3b9fee4644af2887a3ac3fd2e6)` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 10:54:48.290750 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5463eef-e03a-49f5-aa33-25ba75fdb3cc admin admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 10:54:48.295098 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 904/1845] 10.222.0.25 () {60 vars in 1365 bytes} [Sun Aug 7 10:54:48 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 128 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 07 10:54:48.296054 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c3e5a8a-3477-409a-aee2-413db0776be3 service neutron] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 10:54:48.311182 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 942/1846] 10.222.0.25 () {60 vars in 1483 bytes} [Sun Aug 7 10:54:48 2022] PUT /identity/v3/projects/23e2b09610fd4ee0af16405257b2daf3/users/05a0d408980b49acafd2c9a93bf527f2/roles/b98dfb3b9fee4644af2887a3ac3fd2e6 => generated 0 bytes in 68 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 07 10:54:48.320121 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c3e5a8a-3477-409a-aee2-413db0776be3 service neutron] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 10:54:48.321633 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb11b034-2e84-4d39-a51c-c1d60d2f2c5a None None] REQUEST_METHOD: `POST` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:54:48.321930 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb11b034-2e84-4d39-a51c-c1d60d2f2c5a None None] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:54:48.322376 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb11b034-2e84-4d39-a51c-c1d60d2f2c5a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:54:48.425243 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.auth.core [None req-fb11b034-2e84-4d39-a51c-c1d60d2f2c5a None None] MFA Rules not processed for user `05a0d408980b49acafd2c9a93bf527f2`. Rule list: `[]` (Enabled: `True`). {{(pid=80144) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 10:54:48.425787 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-628f30f3-0723-43fc-826e-4db1969da064 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5ac47e3b09f43d19a23a0f457fe9e23', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 10:54:48.426510 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.fernet_utils [None req-fb11b034-2e84-4d39-a51c-c1d60d2f2c5a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80144) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 10:54:48.427207 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-628f30f3-0723-43fc-826e-4db1969da064 admin admin] REQUEST_METHOD: `POST` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:54:48.427503 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-628f30f3-0723-43fc-826e-4db1969da064 admin admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:54:48.427897 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-628f30f3-0723-43fc-826e-4db1969da064 admin admin] PATH_INFO: `/v3/projects` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:54:48.428919 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-628f30f3-0723-43fc-826e-4db1969da064 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 10:54:48.431811 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-628f30f3-0723-43fc-826e-4db1969da064 admin admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 10:54:48.492500 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 905/1847] 10.222.0.25 () {60 vars in 1153 bytes} [Sun Aug 7 10:54:48 2022] POST /identity/v3/projects => generated 379 bytes in 197 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 07 10:54:48.493508 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-628f30f3-0723-43fc-826e-4db1969da064 admin admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 10:54:48.505210 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 10:54:48.552904 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 943/1848] 10.222.0.25 () {58 vars in 958 bytes} [Sun Aug 7 10:54:48 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 232 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 07 10:54:48.568932 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb11b034-2e84-4d39-a51c-c1d60d2f2c5a None None] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 10:54:48.572189 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.fernet_utils [None req-fb11b034-2e84-4d39-a51c-c1d60d2f2c5a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80144) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 10:54:48.603031 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-628f30f3-0723-43fc-826e-4db1969da064 admin admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 10:54:48.609736 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fe21ab6-53cb-4c50-b35d-cee745a6fd17 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5ac47e3b09f43d19a23a0f457fe9e23', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 10:54:48.611839 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fe21ab6-53cb-4c50-b35d-cee745a6fd17 admin admin] REQUEST_METHOD: `POST` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:54:48.611839 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fe21ab6-53cb-4c50-b35d-cee745a6fd17 admin admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:54:48.612182 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fe21ab6-53cb-4c50-b35d-cee745a6fd17 admin admin] PATH_INFO: `/v3/users` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:54:48.613088 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fe21ab6-53cb-4c50-b35d-cee745a6fd17 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 10:54:48.616281 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fe21ab6-53cb-4c50-b35d-cee745a6fd17 admin admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 10:54:48.640878 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 10:54:48.665506 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 906/1849] 10.222.0.25 () {60 vars in 1144 bytes} [Sun Aug 7 10:54:48 2022] POST /identity/v3/users => generated 290 bytes in 172 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 07 10:54:48.670120 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fe21ab6-53cb-4c50-b35d-cee745a6fd17 admin admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 10:54:48.690916 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fe21ab6-53cb-4c50-b35d-cee745a6fd17 admin admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 10:54:48.698458 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c9c6db7-2aea-497b-ad3b-907e1b8a9ea3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5ac47e3b09f43d19a23a0f457fe9e23', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 10:54:48.698995 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c9c6db7-2aea-497b-ad3b-907e1b8a9ea3 admin admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:54:48.699292 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c9c6db7-2aea-497b-ad3b-907e1b8a9ea3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:54:48.699650 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c9c6db7-2aea-497b-ad3b-907e1b8a9ea3 admin admin] PATH_INFO: `/v3/roles` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:54:48.700316 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c9c6db7-2aea-497b-ad3b-907e1b8a9ea3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 10:54:48.701973 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c9c6db7-2aea-497b-ad3b-907e1b8a9ea3 admin admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 10:54:48.716535 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 907/1850] 10.222.0.25 () {58 vars in 1122 bytes} [Sun Aug 7 10:54:48 2022] GET /identity/v3/roles => generated 1420 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 07 10:54:48.720602 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c9c6db7-2aea-497b-ad3b-907e1b8a9ea3 admin admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 10:54:48.724206 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb11b034-2e84-4d39-a51c-c1d60d2f2c5a None None] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 10:54:48.737196 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c9c6db7-2aea-497b-ad3b-907e1b8a9ea3 admin admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 10:54:48.744203 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05bef2fe-5735-4c77-ae76-564d35278020 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5ac47e3b09f43d19a23a0f457fe9e23', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 10:54:48.745128 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05bef2fe-5735-4c77-ae76-564d35278020 admin admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:54:48.745498 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05bef2fe-5735-4c77-ae76-564d35278020 admin admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:54:48.746399 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05bef2fe-5735-4c77-ae76-564d35278020 admin admin] PATH_INFO: `/v3/roles` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:54:48.747540 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05bef2fe-5735-4c77-ae76-564d35278020 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 10:54:48.750245 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05bef2fe-5735-4c77-ae76-564d35278020 admin admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 10:54:48.768267 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 908/1851] 10.222.0.25 () {58 vars in 1122 bytes} [Sun Aug 7 10:54:48 2022] GET /identity/v3/roles => generated 1420 bytes in 48 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 07 10:54:48.772855 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05bef2fe-5735-4c77-ae76-564d35278020 admin admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 10:54:48.786698 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05bef2fe-5735-4c77-ae76-564d35278020 admin admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 10:54:48.793560 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d771b438-ef44-4840-991c-e432f3775607 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5ac47e3b09f43d19a23a0f457fe9e23', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 10:54:48.794245 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d771b438-ef44-4840-991c-e432f3775607 admin admin] REQUEST_METHOD: `PUT` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:54:48.794526 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d771b438-ef44-4840-991c-e432f3775607 admin admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:54:48.794807 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d771b438-ef44-4840-991c-e432f3775607 admin admin] PATH_INFO: `/v3/projects/db541d7336684717aacdd90e77341550/users/ba31bfb88d074806a7c4f79af716a3b2/roles/b98dfb3b9fee4644af2887a3ac3fd2e6` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:54:48.817225 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d771b438-ef44-4840-991c-e432f3775607 admin admin] RBAC: Authorizing `identity:create_grant(project_id=db541d7336684717aacdd90e77341550, user_id=ba31bfb88d074806a7c4f79af716a3b2, role_id=b98dfb3b9fee4644af2887a3ac3fd2e6)` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 10:54:48.820664 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d771b438-ef44-4840-991c-e432f3775607 admin admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 10:54:48.827977 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b069caae-60fa-4b48-8ec0-f08a02df6de8 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a11186d9d0d6488b953ede2f1628148d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6de1f3cac02c4bcdb05dd2337176033d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 10:54:48.828742 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b069caae-60fa-4b48-8ec0-f08a02df6de8 service glance] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:54:48.829170 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b069caae-60fa-4b48-8ec0-f08a02df6de8 service glance] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:54:48.829475 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b069caae-60fa-4b48-8ec0-f08a02df6de8 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:54:48.838624 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 909/1852] 10.222.0.25 () {60 vars in 1483 bytes} [Sun Aug 7 10:54:48 2022] PUT /identity/v3/projects/db541d7336684717aacdd90e77341550/users/ba31bfb88d074806a7c4f79af716a3b2/roles/b98dfb3b9fee4644af2887a3ac3fd2e6 => generated 0 bytes in 66 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 07 10:54:48.852407 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d3e4d60-1b1b-474f-a8c7-5c40bb6f87bb None None] REQUEST_METHOD: `POST` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:54:48.852687 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d3e4d60-1b1b-474f-a8c7-5c40bb6f87bb None None] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:54:48.852965 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d3e4d60-1b1b-474f-a8c7-5c40bb6f87bb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:54:48.904062 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.auth.core [None req-8d3e4d60-1b1b-474f-a8c7-5c40bb6f87bb None None] MFA Rules not processed for user `ba31bfb88d074806a7c4f79af716a3b2`. Rule list: `[]` (Enabled: `True`). {{(pid=80143) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 10:54:48.904924 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-8d3e4d60-1b1b-474f-a8c7-5c40bb6f87bb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 10:54:48.928354 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b069caae-60fa-4b48-8ec0-f08a02df6de8 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 10:54:48.930198 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b069caae-60fa-4b48-8ec0-f08a02df6de8 service glance] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 10:54:48.976404 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 944/1853] 10.222.0.25 () {60 vars in 1365 bytes} [Sun Aug 7 10:54:48 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 411 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 07 10:54:48.978379 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b069caae-60fa-4b48-8ec0-f08a02df6de8 service glance] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 10:54:48.981753 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.fernet_utils [None req-b069caae-60fa-4b48-8ec0-f08a02df6de8 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80144) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 10:54:49.005533 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b069caae-60fa-4b48-8ec0-f08a02df6de8 service glance] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 10:54:49.048602 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 910/1854] 10.222.0.25 () {58 vars in 958 bytes} [Sun Aug 7 10:54:48 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 194 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 07 10:54:49.049931 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d3e4d60-1b1b-474f-a8c7-5c40bb6f87bb None None] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 10:54:49.114022 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83968c53-8017-4545-be4e-7dc0066e1c62 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5ac47e3b09f43d19a23a0f457fe9e23', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 10:54:49.114903 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83968c53-8017-4545-be4e-7dc0066e1c62 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:54:49.115282 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83968c53-8017-4545-be4e-7dc0066e1c62 admin admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:54:49.115641 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83968c53-8017-4545-be4e-7dc0066e1c62 admin admin] PATH_INFO: `/v3/projects/929e031fd1ad4c4fb5f4662ff2364ad6` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:54:49.117419 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83968c53-8017-4545-be4e-7dc0066e1c62 admin admin] RBAC: Authorizing `identity:delete_project(project_id=929e031fd1ad4c4fb5f4662ff2364ad6)` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 10:54:49.140478 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83968c53-8017-4545-be4e-7dc0066e1c62 admin admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 10:54:49.191635 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 10:54:49.227104 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d3e4d60-1b1b-474f-a8c7-5c40bb6f87bb None None] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 10:54:49.234398 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=100 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 10:54:49.238512 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6e45353-ec43-4c0a-b988-e5b0ea968112 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5ac47e3b09f43d19a23a0f457fe9e23', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 10:54:49.239862 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6e45353-ec43-4c0a-b988-e5b0ea968112 admin admin] REQUEST_METHOD: `POST` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:54:49.240546 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6e45353-ec43-4c0a-b988-e5b0ea968112 admin admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:54:49.241239 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6e45353-ec43-4c0a-b988-e5b0ea968112 admin admin] PATH_INFO: `/v3/projects` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:54:49.242619 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6e45353-ec43-4c0a-b988-e5b0ea968112 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 10:54:49.246123 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6e45353-ec43-4c0a-b988-e5b0ea968112 admin admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 10:54:49.290532 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 911/1855] 10.222.0.25 () {60 vars in 1153 bytes} [Sun Aug 7 10:54:49 2022] POST /identity/v3/projects => generated 379 bytes in 241 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 07 10:54:49.295414 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6e45353-ec43-4c0a-b988-e5b0ea968112 admin admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 10:54:49.305848 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.notifications [None req-83968c53-8017-4545-be4e-7dc0066e1c62 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '929e031fd1ad4c4fb5f4662ff2364ad6', 'request_id': 'req-83968c53-8017-4545-be4e-7dc0066e1c62'} {{(pid=80144) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 10:54:49.316686 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.notifications [None req-83968c53-8017-4545-be4e-7dc0066e1c62 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '929e031fd1ad4c4fb5f4662ff2364ad6', 'request_id': 'req-83968c53-8017-4545-be4e-7dc0066e1c62'} {{(pid=80144) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 10:54:49.329805 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.notifications [None req-83968c53-8017-4545-be4e-7dc0066e1c62 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '929e031fd1ad4c4fb5f4662ff2364ad6', 'request_id': 'req-83968c53-8017-4545-be4e-7dc0066e1c62'} {{(pid=80144) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 10:54:49.350624 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.notifications [None req-83968c53-8017-4545-be4e-7dc0066e1c62 admin admin] The token cache is being invalidate because project 929e031fd1ad4c4fb5f4662ff2364ad6 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80144) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 07 10:54:49.350949 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.notifications [None req-83968c53-8017-4545-be4e-7dc0066e1c62 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80144) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 10:54:49.351751 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.notifications [None req-83968c53-8017-4545-be4e-7dc0066e1c62 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80144) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 10:54:49.358817 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 945/1856] 10.222.0.25 () {58 vars in 1233 bytes} [Sun Aug 7 10:54:48 2022] DELETE /identity/v3/projects/929e031fd1ad4c4fb5f4662ff2364ad6 => generated 0 bytes in 377 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 07 10:54:49.360369 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83968c53-8017-4545-be4e-7dc0066e1c62 admin admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 10:54:49.362235 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.fernet_utils [None req-83968c53-8017-4545-be4e-7dc0066e1c62 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80144) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 10:54:49.432853 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6e45353-ec43-4c0a-b988-e5b0ea968112 admin admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 10:54:49.446023 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ae66294-50c8-4cbc-9043-973dd693e9cd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5ac47e3b09f43d19a23a0f457fe9e23', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 10:54:49.446710 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ae66294-50c8-4cbc-9043-973dd693e9cd admin admin] REQUEST_METHOD: `POST` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:54:49.447046 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ae66294-50c8-4cbc-9043-973dd693e9cd admin admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:54:49.447368 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ae66294-50c8-4cbc-9043-973dd693e9cd admin admin] PATH_INFO: `/v3/users` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:54:49.448026 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ae66294-50c8-4cbc-9043-973dd693e9cd admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 10:54:49.450731 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ae66294-50c8-4cbc-9043-973dd693e9cd admin admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 10:54:49.492582 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 912/1857] 10.222.0.25 () {60 vars in 1144 bytes} [Sun Aug 7 10:54:49 2022] POST /identity/v3/users => generated 290 bytes in 198 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 07 10:54:49.498016 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2bd082a-bbb1-4345-8500-0e07fe179b88 None None] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:54:49.498016 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2bd082a-bbb1-4345-8500-0e07fe179b88 None None] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:54:49.498016 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2bd082a-bbb1-4345-8500-0e07fe179b88 None None] PATH_INFO: `/` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:54:49.499466 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 913/1858] 10.222.0.25 () {54 vars in 921 bytes} [Sun Aug 7 10:54:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 07 10:54:49.501387 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2bd082a-bbb1-4345-8500-0e07fe179b88 None None] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 10:54:49.503160 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-e2bd082a-bbb1-4345-8500-0e07fe179b88 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 10:54:49.521163 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83968c53-8017-4545-be4e-7dc0066e1c62 admin admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 10:54:49.542394 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-007ae97c-8db8-4fd3-99db-0616ef395fac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5ac47e3b09f43d19a23a0f457fe9e23', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 10:54:49.543099 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-007ae97c-8db8-4fd3-99db-0616ef395fac admin admin] REQUEST_METHOD: `DELETE` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:54:49.543389 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-007ae97c-8db8-4fd3-99db-0616ef395fac admin admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:54:49.543669 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-007ae97c-8db8-4fd3-99db-0616ef395fac admin admin] PATH_INFO: `/v3/users/0d65e97423d34648a73bd43c687c21a5` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:54:49.544914 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-007ae97c-8db8-4fd3-99db-0616ef395fac admin admin] RBAC: Authorizing `identity:delete_user(user_id=0d65e97423d34648a73bd43c687c21a5)` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 10:54:49.546979 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-007ae97c-8db8-4fd3-99db-0616ef395fac admin admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 10:54:49.569882 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2bd082a-bbb1-4345-8500-0e07fe179b88 None None] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 10:54:49.585205 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2ef7834-f5ce-4f42-a4dd-46ec8ec186e1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5ac47e3b09f43d19a23a0f457fe9e23', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 10:54:49.586045 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2ef7834-f5ce-4f42-a4dd-46ec8ec186e1 admin admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:54:49.586824 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2ef7834-f5ce-4f42-a4dd-46ec8ec186e1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:54:49.587243 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2ef7834-f5ce-4f42-a4dd-46ec8ec186e1 admin admin] PATH_INFO: `/v3/roles` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:54:49.588228 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2ef7834-f5ce-4f42-a4dd-46ec8ec186e1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 10:54:49.590769 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2ef7834-f5ce-4f42-a4dd-46ec8ec186e1 admin admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 10:54:49.623888 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 914/1859] 10.222.0.25 () {58 vars in 1122 bytes} [Sun Aug 7 10:54:49 2022] GET /identity/v3/roles => generated 1420 bytes in 120 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 07 10:54:49.629292 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edf8729e-11c3-4369-b2a0-f9934ea8b566 None None] REQUEST_METHOD: `POST` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:54:49.629724 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edf8729e-11c3-4369-b2a0-f9934ea8b566 None None] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:54:49.630107 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edf8729e-11c3-4369-b2a0-f9934ea8b566 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:54:49.651464 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.notifications [None req-007ae97c-8db8-4fd3-99db-0616ef395fac admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0d65e97423d34648a73bd43c687c21a5'} {{(pid=80144) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 10:54:49.667596 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.notifications [None req-007ae97c-8db8-4fd3-99db-0616ef395fac admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0d65e97423d34648a73bd43c687c21a5'} {{(pid=80144) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 10:54:49.677699 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.auth.core [None req-edf8729e-11c3-4369-b2a0-f9934ea8b566 None None] MFA Rules not processed for user `31d16d1b96104b4eb690398432b9829e`. Rule list: `[]` (Enabled: `True`). {{(pid=80143) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 10:54:49.680552 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-edf8729e-11c3-4369-b2a0-f9934ea8b566 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 10:54:49.686458 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.notifications [None req-007ae97c-8db8-4fd3-99db-0616ef395fac admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0d65e97423d34648a73bd43c687c21a5'} {{(pid=80144) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 10:54:49.687185 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.notifications [None req-007ae97c-8db8-4fd3-99db-0616ef395fac admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0d65e97423d34648a73bd43c687c21a5'} {{(pid=80144) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 10:54:49.687888 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.notifications [None req-007ae97c-8db8-4fd3-99db-0616ef395fac admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0d65e97423d34648a73bd43c687c21a5'} {{(pid=80144) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 10:54:49.739068 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 946/1860] 10.222.0.25 () {58 vars in 1224 bytes} [Sun Aug 7 10:54:49 2022] DELETE /identity/v3/users/0d65e97423d34648a73bd43c687c21a5 => generated 0 bytes in 380 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 07 10:54:49.754553 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-007ae97c-8db8-4fd3-99db-0616ef395fac admin admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 10:54:49.757339 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.fernet_utils [None req-007ae97c-8db8-4fd3-99db-0616ef395fac admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80144) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 10:54:49.830723 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 10:54:49.895628 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-007ae97c-8db8-4fd3-99db-0616ef395fac admin admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 10:54:49.900943 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bac28b2-1086-4e42-90e7-2a5a7f3c022a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5ac47e3b09f43d19a23a0f457fe9e23', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 10:54:49.901584 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bac28b2-1086-4e42-90e7-2a5a7f3c022a admin admin] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:54:49.901942 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bac28b2-1086-4e42-90e7-2a5a7f3c022a admin admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:54:49.902291 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bac28b2-1086-4e42-90e7-2a5a7f3c022a admin admin] PATH_INFO: `/v3/roles` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:54:49.903069 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bac28b2-1086-4e42-90e7-2a5a7f3c022a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 10:54:49.905505 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bac28b2-1086-4e42-90e7-2a5a7f3c022a admin admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 10:54:49.923343 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 947/1861] 10.222.0.25 () {58 vars in 1122 bytes} [Sun Aug 7 10:54:49 2022] GET /identity/v3/roles => generated 1420 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 07 10:54:49.924625 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bac28b2-1086-4e42-90e7-2a5a7f3c022a admin admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 10:54:49.948014 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 915/1862] 10.222.0.25 () {58 vars in 1018 bytes} [Sun Aug 7 10:54:49 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 323 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 07 10:54:49.963838 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edf8729e-11c3-4369-b2a0-f9934ea8b566 None None] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 10:54:49.965627 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-edf8729e-11c3-4369-b2a0-f9934ea8b566 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 10:54:49.967136 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bac28b2-1086-4e42-90e7-2a5a7f3c022a admin admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 10:54:49.987667 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76160bb5-96be-457b-adf3-3a22aa8590c6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5ac47e3b09f43d19a23a0f457fe9e23', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 10:54:49.990274 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76160bb5-96be-457b-adf3-3a22aa8590c6 admin admin] REQUEST_METHOD: `PUT` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:54:49.991141 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76160bb5-96be-457b-adf3-3a22aa8590c6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:54:49.991945 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76160bb5-96be-457b-adf3-3a22aa8590c6 admin admin] PATH_INFO: `/v3/projects/758d124435054ee0a53b42100561728f/users/1f550b79e60a405cacb953d861bad2b7/roles/b98dfb3b9fee4644af2887a3ac3fd2e6` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:54:50.002632 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edf8729e-11c3-4369-b2a0-f9934ea8b566 None None] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 10:54:50.070528 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76160bb5-96be-457b-adf3-3a22aa8590c6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=758d124435054ee0a53b42100561728f, user_id=1f550b79e60a405cacb953d861bad2b7, role_id=b98dfb3b9fee4644af2887a3ac3fd2e6)` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 10:54:50.074803 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 10:54:50.075293 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76160bb5-96be-457b-adf3-3a22aa8590c6 admin admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 10:54:50.103423 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 948/1863] 10.222.0.25 () {60 vars in 1483 bytes} [Sun Aug 7 10:54:49 2022] PUT /identity/v3/projects/758d124435054ee0a53b42100561728f/users/1f550b79e60a405cacb953d861bad2b7/roles/b98dfb3b9fee4644af2887a3ac3fd2e6 => generated 0 bytes in 179 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 07 10:54:50.124743 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad5e8a1f-da3c-4eaa-aa6e-76115aeaef98 None None] REQUEST_METHOD: `POST` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:54:50.125066 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad5e8a1f-da3c-4eaa-aa6e-76115aeaef98 None None] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:54:50.125425 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad5e8a1f-da3c-4eaa-aa6e-76115aeaef98 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:54:50.167463 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.auth.core [None req-ad5e8a1f-da3c-4eaa-aa6e-76115aeaef98 None None] MFA Rules not processed for user `1f550b79e60a405cacb953d861bad2b7`. Rule list: `[]` (Enabled: `True`). {{(pid=80144) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 10:54:50.169668 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.fernet_utils [None req-ad5e8a1f-da3c-4eaa-aa6e-76115aeaef98 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80144) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 10:54:50.173724 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9d41d4d-b141-4f02-915b-8dbff6e42d1e service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31d16d1b96104b4eb690398432b9829e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6de1f3cac02c4bcdb05dd2337176033d', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'reader', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 10:54:50.175544 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9d41d4d-b141-4f02-915b-8dbff6e42d1e service nova] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:54:50.176167 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9d41d4d-b141-4f02-915b-8dbff6e42d1e service nova] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:54:50.176451 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9d41d4d-b141-4f02-915b-8dbff6e42d1e service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:54:50.196335 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9d41d4d-b141-4f02-915b-8dbff6e42d1e service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 10:54:50.198731 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9d41d4d-b141-4f02-915b-8dbff6e42d1e service nova] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 10:54:50.233915 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 916/1864] 10.222.0.25 () {60 vars in 1365 bytes} [Sun Aug 7 10:54:49 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 271 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 07 10:54:50.327895 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 949/1865] 10.222.0.25 () {58 vars in 958 bytes} [Sun Aug 7 10:54:50 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 205 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 07 10:54:50.334476 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9d41d4d-b141-4f02-915b-8dbff6e42d1e service nova] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 10:54:50.348068 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9d41d4d-b141-4f02-915b-8dbff6e42d1e service nova] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 10:54:50.542277 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2abd1635-e0f8-4fce-9ade-e5795f74ac30 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5ac47e3b09f43d19a23a0f457fe9e23', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 10:54:50.544311 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2abd1635-e0f8-4fce-9ade-e5795f74ac30 admin admin] REQUEST_METHOD: `POST` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:54:50.562051 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2abd1635-e0f8-4fce-9ade-e5795f74ac30 admin admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:54:50.562356 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2abd1635-e0f8-4fce-9ade-e5795f74ac30 admin admin] PATH_INFO: `/v3/projects` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:54:50.566233 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2abd1635-e0f8-4fce-9ade-e5795f74ac30 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 10:54:50.568397 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2abd1635-e0f8-4fce-9ade-e5795f74ac30 admin admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 10:54:50.590910 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad5e8a1f-da3c-4eaa-aa6e-76115aeaef98 None None] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 10:54:50.593373 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.fernet_utils [None req-ad5e8a1f-da3c-4eaa-aa6e-76115aeaef98 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80144) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 10:54:50.639253 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 917/1866] 10.222.0.25 () {60 vars in 1153 bytes} [Sun Aug 7 10:54:50 2022] POST /identity/v3/projects => generated 377 bytes in 302 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 07 10:54:50.639512 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad5e8a1f-da3c-4eaa-aa6e-76115aeaef98 None None] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 10:54:50.641230 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2abd1635-e0f8-4fce-9ade-e5795f74ac30 admin admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 10:54:50.777415 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2abd1635-e0f8-4fce-9ade-e5795f74ac30 admin admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 10:54:50.786010 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01d9a3a3-05e7-4b21-a81b-83707b179860 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5ac47e3b09f43d19a23a0f457fe9e23', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 10:54:50.787752 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01d9a3a3-05e7-4b21-a81b-83707b179860 admin admin] REQUEST_METHOD: `POST` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:54:50.787752 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01d9a3a3-05e7-4b21-a81b-83707b179860 admin admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:54:50.787984 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01d9a3a3-05e7-4b21-a81b-83707b179860 admin admin] PATH_INFO: `/v3/users` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:54:50.788787 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01d9a3a3-05e7-4b21-a81b-83707b179860 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 10:54:50.791282 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01d9a3a3-05e7-4b21-a81b-83707b179860 admin admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 10:54:50.810250 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e446d3f-f5ea-495d-8ce4-0492c654c167 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5ac47e3b09f43d19a23a0f457fe9e23', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 10:54:50.811414 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e446d3f-f5ea-495d-8ce4-0492c654c167 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:54:50.811774 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e446d3f-f5ea-495d-8ce4-0492c654c167 admin admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:54:50.812163 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e446d3f-f5ea-495d-8ce4-0492c654c167 admin admin] PATH_INFO: `/v3/users/f512dd6985224deb91640fd4f6401165` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:54:50.814026 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e446d3f-f5ea-495d-8ce4-0492c654c167 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f512dd6985224deb91640fd4f6401165)` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 10:54:50.816464 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e446d3f-f5ea-495d-8ce4-0492c654c167 admin admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 10:54:50.827681 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 918/1867] 10.222.0.25 () {60 vars in 1144 bytes} [Sun Aug 7 10:54:50 2022] POST /identity/v3/users => generated 295 bytes in 187 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 07 10:54:50.834282 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01d9a3a3-05e7-4b21-a81b-83707b179860 admin admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 10:54:50.865132 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01d9a3a3-05e7-4b21-a81b-83707b179860 admin admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 10:54:50.875014 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccfd5448-95e1-42d4-b7cc-b4c2bf818071 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5ac47e3b09f43d19a23a0f457fe9e23', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 10:54:50.882908 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccfd5448-95e1-42d4-b7cc-b4c2bf818071 admin admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:54:50.884642 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccfd5448-95e1-42d4-b7cc-b4c2bf818071 admin admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:54:50.885081 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccfd5448-95e1-42d4-b7cc-b4c2bf818071 admin admin] PATH_INFO: `/v3/roles` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:54:50.886731 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccfd5448-95e1-42d4-b7cc-b4c2bf818071 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 10:54:50.894637 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccfd5448-95e1-42d4-b7cc-b4c2bf818071 admin admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 10:54:50.906503 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=94,keystone:INSERT=1,keystone:DELETE=5 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 10:54:50.925014 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 919/1868] 10.222.0.25 () {58 vars in 1122 bytes} [Sun Aug 7 10:54:50 2022] GET /identity/v3/roles => generated 1420 bytes in 92 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 07 10:54:50.938288 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccfd5448-95e1-42d4-b7cc-b4c2bf818071 admin admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 10:54:50.958600 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccfd5448-95e1-42d4-b7cc-b4c2bf818071 admin admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 10:54:50.961377 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.notifications [None req-0e446d3f-f5ea-495d-8ce4-0492c654c167 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f512dd6985224deb91640fd4f6401165'} {{(pid=80144) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 10:54:50.964877 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe5f0264-2dcd-451b-8691-e2fbb4bbf570 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5ac47e3b09f43d19a23a0f457fe9e23', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 10:54:50.965635 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe5f0264-2dcd-451b-8691-e2fbb4bbf570 admin admin] REQUEST_METHOD: `PUT` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:54:50.965833 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe5f0264-2dcd-451b-8691-e2fbb4bbf570 admin admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:54:50.966108 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe5f0264-2dcd-451b-8691-e2fbb4bbf570 admin admin] PATH_INFO: `/v3/projects/d20fc70dfb7e4e96a6b5d6f06f7058ed/users/7bbd634e87404804bf411c2385f0666b/roles/81b1f1bcee2a473088fbbd327c1c5b23` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:54:50.980710 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.notifications [None req-0e446d3f-f5ea-495d-8ce4-0492c654c167 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f512dd6985224deb91640fd4f6401165'} {{(pid=80144) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 10:54:50.998871 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.notifications [None req-0e446d3f-f5ea-495d-8ce4-0492c654c167 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f512dd6985224deb91640fd4f6401165'} {{(pid=80144) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 10:54:51.000091 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.notifications [None req-0e446d3f-f5ea-495d-8ce4-0492c654c167 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f512dd6985224deb91640fd4f6401165'} {{(pid=80144) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 10:54:51.000229 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe5f0264-2dcd-451b-8691-e2fbb4bbf570 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d20fc70dfb7e4e96a6b5d6f06f7058ed, user_id=7bbd634e87404804bf411c2385f0666b, role_id=81b1f1bcee2a473088fbbd327c1c5b23)` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 10:54:51.001176 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.notifications [None req-0e446d3f-f5ea-495d-8ce4-0492c654c167 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f512dd6985224deb91640fd4f6401165'} {{(pid=80144) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 10:54:51.003919 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe5f0264-2dcd-451b-8691-e2fbb4bbf570 admin admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 10:54:51.039933 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 920/1869] 10.222.0.25 () {60 vars in 1483 bytes} [Sun Aug 7 10:54:50 2022] PUT /identity/v3/projects/d20fc70dfb7e4e96a6b5d6f06f7058ed/users/7bbd634e87404804bf411c2385f0666b/roles/81b1f1bcee2a473088fbbd327c1c5b23 => generated 0 bytes in 106 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 07 10:54:51.044745 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 950/1870] 10.222.0.25 () {58 vars in 1224 bytes} [Sun Aug 7 10:54:50 2022] DELETE /identity/v3/users/f512dd6985224deb91640fd4f6401165 => generated 0 bytes in 458 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 07 10:54:51.045617 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe5f0264-2dcd-451b-8691-e2fbb4bbf570 admin admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 10:54:51.050340 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-fe5f0264-2dcd-451b-8691-e2fbb4bbf570 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 10:54:51.052762 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60c26f61-1f09-41c2-83d9-86eb231545d0 None None] REQUEST_METHOD: `POST` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:54:51.053348 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60c26f61-1f09-41c2-83d9-86eb231545d0 None None] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:54:51.053975 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60c26f61-1f09-41c2-83d9-86eb231545d0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:54:51.060473 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 10:54:51.133306 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.auth.core [None req-60c26f61-1f09-41c2-83d9-86eb231545d0 None None] MFA Rules not processed for user `bac4e6b76fe64217ac8fa44e71cffe6a`. Rule list: `[]` (Enabled: `True`). {{(pid=80144) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 10:54:51.134603 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.fernet_utils [None req-60c26f61-1f09-41c2-83d9-86eb231545d0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80144) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 10:54:51.208969 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe5f0264-2dcd-451b-8691-e2fbb4bbf570 admin admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 10:54:51.237607 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 951/1871] 10.222.0.25 () {58 vars in 958 bytes} [Sun Aug 7 10:54:51 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 187 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 07 10:54:51.274455 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60c26f61-1f09-41c2-83d9-86eb231545d0 None None] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 10:54:51.276999 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.fernet_utils [None req-60c26f61-1f09-41c2-83d9-86eb231545d0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80144) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 10:54:51.342762 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e641e597-cb7e-4415-9f29-3baf29ee55e3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5ac47e3b09f43d19a23a0f457fe9e23', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 10:54:51.343804 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e641e597-cb7e-4415-9f29-3baf29ee55e3 admin admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:54:51.344289 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e641e597-cb7e-4415-9f29-3baf29ee55e3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:54:51.344732 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e641e597-cb7e-4415-9f29-3baf29ee55e3 admin admin] PATH_INFO: `/v3/roles` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:54:51.346210 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e641e597-cb7e-4415-9f29-3baf29ee55e3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 10:54:51.348220 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e641e597-cb7e-4415-9f29-3baf29ee55e3 admin admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 10:54:51.367839 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 921/1872] 10.222.0.25 () {58 vars in 1122 bytes} [Sun Aug 7 10:54:51 2022] GET /identity/v3/roles => generated 1420 bytes in 323 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 07 10:54:51.372091 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e641e597-cb7e-4415-9f29-3baf29ee55e3 admin admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 10:54:51.386959 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60c26f61-1f09-41c2-83d9-86eb231545d0 None None] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 10:54:51.389650 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e641e597-cb7e-4415-9f29-3baf29ee55e3 admin admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 10:54:51.393476 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90d3f50e-56bf-4b64-a1ca-0ff39ddd2aba service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be1283c9bf47424c9aa3ad27cab03027', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6de1f3cac02c4bcdb05dd2337176033d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 10:54:51.394873 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90d3f50e-56bf-4b64-a1ca-0ff39ddd2aba service neutron] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:54:51.395270 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90d3f50e-56bf-4b64-a1ca-0ff39ddd2aba service neutron] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:54:51.395598 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90d3f50e-56bf-4b64-a1ca-0ff39ddd2aba service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:54:51.397262 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ed343c7-7bd0-4c07-9a65-12239cc95a4e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5ac47e3b09f43d19a23a0f457fe9e23', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 10:54:51.398202 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ed343c7-7bd0-4c07-9a65-12239cc95a4e admin admin] REQUEST_METHOD: `PUT` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:54:51.398706 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ed343c7-7bd0-4c07-9a65-12239cc95a4e admin admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:54:51.399218 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ed343c7-7bd0-4c07-9a65-12239cc95a4e admin admin] PATH_INFO: `/v3/projects/d20fc70dfb7e4e96a6b5d6f06f7058ed/users/7bbd634e87404804bf411c2385f0666b/roles/81b1f1bcee2a473088fbbd327c1c5b23` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:54:51.401619 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ed343c7-7bd0-4c07-9a65-12239cc95a4e admin admin] RBAC: Authorizing `identity:create_grant(project_id=d20fc70dfb7e4e96a6b5d6f06f7058ed, user_id=7bbd634e87404804bf411c2385f0666b, role_id=81b1f1bcee2a473088fbbd327c1c5b23)` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 10:54:51.404880 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ed343c7-7bd0-4c07-9a65-12239cc95a4e admin admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 10:54:51.413889 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90d3f50e-56bf-4b64-a1ca-0ff39ddd2aba service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 10:54:51.431435 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90d3f50e-56bf-4b64-a1ca-0ff39ddd2aba service neutron] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 10:54:51.461122 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 922/1873] 10.222.0.25 () {60 vars in 1483 bytes} [Sun Aug 7 10:54:51 2022] PUT /identity/v3/projects/d20fc70dfb7e4e96a6b5d6f06f7058ed/users/7bbd634e87404804bf411c2385f0666b/roles/81b1f1bcee2a473088fbbd327c1c5b23 => generated 0 bytes in 77 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 07 10:54:51.461122 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ed343c7-7bd0-4c07-9a65-12239cc95a4e admin admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 10:54:51.466084 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-8ed343c7-7bd0-4c07-9a65-12239cc95a4e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 10:54:51.466379 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 952/1874] 10.222.0.25 () {60 vars in 1365 bytes} [Sun Aug 7 10:54:51 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 192 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 07 10:54:51.472397 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cff4f3ef-784e-493f-bea0-9309e8ee9d21 None None] REQUEST_METHOD: `POST` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:54:51.473176 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cff4f3ef-784e-493f-bea0-9309e8ee9d21 None None] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:54:51.473562 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cff4f3ef-784e-493f-bea0-9309e8ee9d21 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:54:51.554022 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.auth.core [None req-cff4f3ef-784e-493f-bea0-9309e8ee9d21 None None] MFA Rules not processed for user `7bbd634e87404804bf411c2385f0666b`. Rule list: `[]` (Enabled: `True`). {{(pid=80144) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 10:54:51.557496 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.fernet_utils [None req-cff4f3ef-784e-493f-bea0-9309e8ee9d21 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80144) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 10:54:51.612638 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ed343c7-7bd0-4c07-9a65-12239cc95a4e admin admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 10:54:51.619029 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f868309c-a746-474b-a699-ad7c0114f577 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5ac47e3b09f43d19a23a0f457fe9e23', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 10:54:51.630549 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f868309c-a746-474b-a699-ad7c0114f577 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:54:51.631320 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f868309c-a746-474b-a699-ad7c0114f577 admin admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:54:51.631320 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f868309c-a746-474b-a699-ad7c0114f577 admin admin] PATH_INFO: `/v3/projects/8cb6be2e01be442382e1682df57b454c` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:54:51.632987 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f868309c-a746-474b-a699-ad7c0114f577 admin admin] RBAC: Authorizing `identity:delete_project(project_id=8cb6be2e01be442382e1682df57b454c)` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 10:54:51.641615 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f868309c-a746-474b-a699-ad7c0114f577 admin admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 10:54:51.662047 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=94,keystone:DELETE=5,keystone:INSERT=1 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 10:54:51.716123 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 953/1875] 10.222.0.25 () {58 vars in 958 bytes} [Sun Aug 7 10:54:51 2022] POST /identity/v3/auth/tokens => generated 3321 bytes in 249 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 07 10:54:51.728703 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cff4f3ef-784e-493f-bea0-9309e8ee9d21 None None] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 10:54:51.730155 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.fernet_utils [None req-cff4f3ef-784e-493f-bea0-9309e8ee9d21 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80144) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 10:54:51.746842 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=97,keystone:INSERT=1,keystone:DELETE=2 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 10:54:51.771845 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-f868309c-a746-474b-a699-ad7c0114f577 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8cb6be2e01be442382e1682df57b454c', 'request_id': 'req-f868309c-a746-474b-a699-ad7c0114f577'} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 10:54:51.790043 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-f868309c-a746-474b-a699-ad7c0114f577 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8cb6be2e01be442382e1682df57b454c', 'request_id': 'req-f868309c-a746-474b-a699-ad7c0114f577'} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 10:54:51.827419 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-f868309c-a746-474b-a699-ad7c0114f577 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8cb6be2e01be442382e1682df57b454c', 'request_id': 'req-f868309c-a746-474b-a699-ad7c0114f577'} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 10:54:51.871523 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-f868309c-a746-474b-a699-ad7c0114f577 admin admin] The token cache is being invalidate because project 8cb6be2e01be442382e1682df57b454c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80143) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 07 10:54:51.872799 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-f868309c-a746-474b-a699-ad7c0114f577 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 10:54:51.874292 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-f868309c-a746-474b-a699-ad7c0114f577 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 10:54:51.878434 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 923/1876] 10.222.0.25 () {58 vars in 1233 bytes} [Sun Aug 7 10:54:51 2022] DELETE /identity/v3/projects/8cb6be2e01be442382e1682df57b454c => generated 0 bytes in 429 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 07 10:54:51.884521 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6853d692-6ad7-4d22-b835-9d8c2008f581 None None] REQUEST_METHOD: `POST` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:54:51.885129 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6853d692-6ad7-4d22-b835-9d8c2008f581 None None] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:54:51.885629 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6853d692-6ad7-4d22-b835-9d8c2008f581 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:54:51.887364 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cff4f3ef-784e-493f-bea0-9309e8ee9d21 None None] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 10:54:51.927881 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.auth.core [None req-6853d692-6ad7-4d22-b835-9d8c2008f581 None None] MFA Rules not processed for user `bac4e6b76fe64217ac8fa44e71cffe6a`. Rule list: `[]` (Enabled: `True`). {{(pid=80143) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 10:54:51.929753 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-6853d692-6ad7-4d22-b835-9d8c2008f581 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 10:54:51.990602 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9489b526-4149-4a88-9aa7-3c63f98dc0a4 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88dbd6dfd1164994a3e16da553561910', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6de1f3cac02c4bcdb05dd2337176033d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 10:54:51.994291 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9489b526-4149-4a88-9aa7-3c63f98dc0a4 service cinder] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:54:51.994562 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9489b526-4149-4a88-9aa7-3c63f98dc0a4 service cinder] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:54:51.994835 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9489b526-4149-4a88-9aa7-3c63f98dc0a4 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:54:52.003326 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.fernet_utils [None req-9489b526-4149-4a88-9aa7-3c63f98dc0a4 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80144) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 10:54:52.060169 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 924/1877] 10.222.0.25 () {58 vars in 958 bytes} [Sun Aug 7 10:54:51 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 176 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 07 10:54:52.068053 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b45e96f9-a4c0-42f4-af5d-c860c6457249 None None] REQUEST_METHOD: `POST` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:54:52.068923 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b45e96f9-a4c0-42f4-af5d-c860c6457249 None None] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:54:52.069628 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b45e96f9-a4c0-42f4-af5d-c860c6457249 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:54:52.098642 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.auth.core [None req-b45e96f9-a4c0-42f4-af5d-c860c6457249 None None] MFA Rules not processed for user `bac4e6b76fe64217ac8fa44e71cffe6a`. Rule list: `[]` (Enabled: `True`). {{(pid=80143) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 10:54:52.101037 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-b45e96f9-a4c0-42f4-af5d-c860c6457249 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 10:54:52.115725 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 925/1878] 10.222.0.25 () {58 vars in 958 bytes} [Sun Aug 7 10:54:52 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 49 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 07 10:54:52.119953 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9489b526-4149-4a88-9aa7-3c63f98dc0a4 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 10:54:52.120694 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b45e96f9-a4c0-42f4-af5d-c860c6457249 None None] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 10:54:52.123593 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9489b526-4149-4a88-9aa7-3c63f98dc0a4 service cinder] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 10:54:52.144892 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b45e96f9-a4c0-42f4-af5d-c860c6457249 None None] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 10:54:52.158898 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40d01e71-11ad-4b3f-8098-bfe9beb92a2c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5ac47e3b09f43d19a23a0f457fe9e23', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 10:54:52.158898 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40d01e71-11ad-4b3f-8098-bfe9beb92a2c admin admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:54:52.159350 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 954/1879] 10.222.0.25 () {60 vars in 1365 bytes} [Sun Aug 7 10:54:51 2022] GET /identity/v3/auth/tokens => generated 3321 bytes in 430 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 07 10:54:52.159435 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40d01e71-11ad-4b3f-8098-bfe9beb92a2c admin admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:54:52.159435 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40d01e71-11ad-4b3f-8098-bfe9beb92a2c admin admin] PATH_INFO: `/v3/domains` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:54:52.159435 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40d01e71-11ad-4b3f-8098-bfe9beb92a2c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80143) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 10:54:52.159435 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40d01e71-11ad-4b3f-8098-bfe9beb92a2c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 10:54:52.162697 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40d01e71-11ad-4b3f-8098-bfe9beb92a2c admin admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 10:54:52.189511 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 926/1880] 10.222.0.25 () {58 vars in 1153 bytes} [Sun Aug 7 10:54:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 69 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 10:54:52.195608 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9489b526-4149-4a88-9aa7-3c63f98dc0a4 service cinder] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 10:54:52.214212 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9489b526-4149-4a88-9aa7-3c63f98dc0a4 service cinder] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 10:54:52.224872 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b526160-bd29-4955-aa97-91299605589a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5ac47e3b09f43d19a23a0f457fe9e23', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 10:54:52.225405 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b526160-bd29-4955-aa97-91299605589a admin admin] REQUEST_METHOD: `POST` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:54:52.225691 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b526160-bd29-4955-aa97-91299605589a admin admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:54:52.225803 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40d01e71-11ad-4b3f-8098-bfe9beb92a2c admin admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 10:54:52.226281 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b526160-bd29-4955-aa97-91299605589a admin admin] PATH_INFO: `/v3/projects` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:54:52.226593 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b526160-bd29-4955-aa97-91299605589a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 10:54:52.227509 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-40d01e71-11ad-4b3f-8098-bfe9beb92a2c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 10:54:52.228617 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b526160-bd29-4955-aa97-91299605589a admin admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 10:54:52.243752 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40d01e71-11ad-4b3f-8098-bfe9beb92a2c admin admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 10:54:52.248229 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9fe42cb-8042-41a7-83d2-e1fb9624d248 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88dbd6dfd1164994a3e16da553561910', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6de1f3cac02c4bcdb05dd2337176033d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 10:54:52.248913 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9fe42cb-8042-41a7-83d2-e1fb9624d248 service cinder] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:54:52.249261 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9fe42cb-8042-41a7-83d2-e1fb9624d248 service cinder] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:54:52.249603 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9fe42cb-8042-41a7-83d2-e1fb9624d248 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:54:52.253238 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-d9fe42cb-8042-41a7-83d2-e1fb9624d248 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 10:54:52.308871 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 955/1881] 10.222.0.25 () {60 vars in 1153 bytes} [Sun Aug 7 10:54:52 2022] POST /identity/v3/projects => generated 385 bytes in 115 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 07 10:54:52.318791 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b526160-bd29-4955-aa97-91299605589a admin admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 10:54:52.358499 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 10:54:52.396336 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9fe42cb-8042-41a7-83d2-e1fb9624d248 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 10:54:52.398065 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9fe42cb-8042-41a7-83d2-e1fb9624d248 service cinder] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 10:54:52.410915 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 927/1882] 10.222.0.25 () {60 vars in 1365 bytes} [Sun Aug 7 10:54:52 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 186 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 07 10:54:52.438844 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b526160-bd29-4955-aa97-91299605589a admin admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 10:54:52.444592 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd0fe2bb-4bbe-4a09-ae2b-52fd48968141 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5ac47e3b09f43d19a23a0f457fe9e23', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 10:54:52.445283 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd0fe2bb-4bbe-4a09-ae2b-52fd48968141 admin admin] REQUEST_METHOD: `POST` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:54:52.445576 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd0fe2bb-4bbe-4a09-ae2b-52fd48968141 admin admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:54:52.445872 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd0fe2bb-4bbe-4a09-ae2b-52fd48968141 admin admin] PATH_INFO: `/v3/users` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:54:52.446738 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd0fe2bb-4bbe-4a09-ae2b-52fd48968141 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 10:54:52.449362 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd0fe2bb-4bbe-4a09-ae2b-52fd48968141 admin admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 10:54:52.484214 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 956/1883] 10.222.0.25 () {60 vars in 1144 bytes} [Sun Aug 7 10:54:52 2022] POST /identity/v3/users => generated 293 bytes in 163 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 07 10:54:52.487007 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9fe42cb-8042-41a7-83d2-e1fb9624d248 service cinder] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 10:54:52.518076 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9fe42cb-8042-41a7-83d2-e1fb9624d248 service cinder] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 10:54:52.530827 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d41128f6-fc0c-40a2-a29c-30a5763c7fd6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5ac47e3b09f43d19a23a0f457fe9e23', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 10:54:52.531553 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d41128f6-fc0c-40a2-a29c-30a5763c7fd6 admin admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:54:52.532721 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d41128f6-fc0c-40a2-a29c-30a5763c7fd6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:54:52.533445 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d41128f6-fc0c-40a2-a29c-30a5763c7fd6 admin admin] PATH_INFO: `/v3/roles` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:54:52.534143 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d41128f6-fc0c-40a2-a29c-30a5763c7fd6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 10:54:52.536220 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d41128f6-fc0c-40a2-a29c-30a5763c7fd6 admin admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 10:54:52.553968 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 928/1884] 10.222.0.25 () {58 vars in 1122 bytes} [Sun Aug 7 10:54:52 2022] GET /identity/v3/roles => generated 1420 bytes in 69 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 07 10:54:52.557638 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd0fe2bb-4bbe-4a09-ae2b-52fd48968141 admin admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 10:54:52.569498 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd0fe2bb-4bbe-4a09-ae2b-52fd48968141 admin admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 10:54:52.576070 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8056831-1660-423e-a6a6-87a1fc4e632e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5ac47e3b09f43d19a23a0f457fe9e23', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 10:54:52.576907 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8056831-1660-423e-a6a6-87a1fc4e632e admin admin] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:54:52.577304 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8056831-1660-423e-a6a6-87a1fc4e632e admin admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:54:52.579937 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8056831-1660-423e-a6a6-87a1fc4e632e admin admin] PATH_INFO: `/v3/roles` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:54:52.582701 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8056831-1660-423e-a6a6-87a1fc4e632e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 10:54:52.584901 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8056831-1660-423e-a6a6-87a1fc4e632e admin admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 10:54:52.613133 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 957/1885] 10.222.0.25 () {58 vars in 1122 bytes} [Sun Aug 7 10:54:52 2022] GET /identity/v3/roles => generated 1420 bytes in 56 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 07 10:54:52.613736 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d41128f6-fc0c-40a2-a29c-30a5763c7fd6 admin admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 10:54:52.624741 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d41128f6-fc0c-40a2-a29c-30a5763c7fd6 admin admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 10:54:52.645823 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34ca24b3-dc1e-47fd-9621-3990d9828668 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5ac47e3b09f43d19a23a0f457fe9e23', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 10:54:52.646606 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34ca24b3-dc1e-47fd-9621-3990d9828668 admin admin] REQUEST_METHOD: `PUT` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:54:52.646887 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34ca24b3-dc1e-47fd-9621-3990d9828668 admin admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:54:52.647189 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34ca24b3-dc1e-47fd-9621-3990d9828668 admin admin] PATH_INFO: `/v3/projects/a3d26e4432c742c8b69fe73882c23b1a/users/9032af01895a47d08540b9c60226f640/roles/b98dfb3b9fee4644af2887a3ac3fd2e6` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:54:52.681001 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34ca24b3-dc1e-47fd-9621-3990d9828668 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a3d26e4432c742c8b69fe73882c23b1a, user_id=9032af01895a47d08540b9c60226f640, role_id=b98dfb3b9fee4644af2887a3ac3fd2e6)` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 10:54:52.686200 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34ca24b3-dc1e-47fd-9621-3990d9828668 admin admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 10:54:52.708124 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 929/1886] 10.222.0.25 () {60 vars in 1483 bytes} [Sun Aug 7 10:54:52 2022] PUT /identity/v3/projects/a3d26e4432c742c8b69fe73882c23b1a/users/9032af01895a47d08540b9c60226f640/roles/b98dfb3b9fee4644af2887a3ac3fd2e6 => generated 0 bytes in 95 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 07 10:54:52.718558 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ad0b1c9-fb7d-47bd-be42-692ebaf6d790 None None] REQUEST_METHOD: `POST` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:54:52.719678 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ad0b1c9-fb7d-47bd-be42-692ebaf6d790 None None] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:54:52.720064 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ad0b1c9-fb7d-47bd-be42-692ebaf6d790 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:54:52.819909 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.auth.core [None req-2ad0b1c9-fb7d-47bd-be42-692ebaf6d790 None None] MFA Rules not processed for user `9032af01895a47d08540b9c60226f640`. Rule list: `[]` (Enabled: `True`). {{(pid=80144) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 10:54:52.821244 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.fernet_utils [None req-2ad0b1c9-fb7d-47bd-be42-692ebaf6d790 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80144) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 10:54:53.098230 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 958/1887] 10.222.0.25 () {58 vars in 958 bytes} [Sun Aug 7 10:54:52 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 381 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 07 10:54:53.103565 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34ca24b3-dc1e-47fd-9621-3990d9828668 admin admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 10:54:53.121889 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:DELETE=5,keystone:SELECT=93,keystone:INSERT=2 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 10:54:53.270622 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ad0b1c9-fb7d-47bd-be42-692ebaf6d790 None None] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 10:54:53.272151 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.fernet_utils [None req-2ad0b1c9-fb7d-47bd-be42-692ebaf6d790 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80144) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 10:54:53.284832 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34ca24b3-dc1e-47fd-9621-3990d9828668 admin admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 10:54:53.290984 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ad0b1c9-fb7d-47bd-be42-692ebaf6d790 None None] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 10:54:53.296985 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a177bbaa-18c8-4b7c-9205-fd944a00526f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5ac47e3b09f43d19a23a0f457fe9e23', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 10:54:53.298419 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a177bbaa-18c8-4b7c-9205-fd944a00526f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:54:53.298905 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a177bbaa-18c8-4b7c-9205-fd944a00526f admin admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:54:53.299062 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d682ac4-8ea4-4c8f-a83d-f99adb172356 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5ac47e3b09f43d19a23a0f457fe9e23', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 10:54:53.299274 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a177bbaa-18c8-4b7c-9205-fd944a00526f admin admin] PATH_INFO: `/v3/projects/105d41e438e6421898223c9f12bebd9a` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:54:53.299603 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d682ac4-8ea4-4c8f-a83d-f99adb172356 admin admin] REQUEST_METHOD: `POST` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:54:53.299923 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d682ac4-8ea4-4c8f-a83d-f99adb172356 admin admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:54:53.300156 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d682ac4-8ea4-4c8f-a83d-f99adb172356 admin admin] PATH_INFO: `/v3/projects` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:54:53.300571 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a177bbaa-18c8-4b7c-9205-fd944a00526f admin admin] RBAC: Authorizing `identity:delete_project(project_id=105d41e438e6421898223c9f12bebd9a)` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 10:54:53.300790 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d682ac4-8ea4-4c8f-a83d-f99adb172356 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 10:54:53.303453 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a177bbaa-18c8-4b7c-9205-fd944a00526f admin admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 10:54:53.317575 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d682ac4-8ea4-4c8f-a83d-f99adb172356 admin admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 10:54:53.364096 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 930/1888] 10.222.0.25 () {60 vars in 1153 bytes} [Sun Aug 7 10:54:53 2022] POST /identity/v3/projects => generated 385 bytes in 261 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 07 10:54:53.367529 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d682ac4-8ea4-4c8f-a83d-f99adb172356 admin admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 10:54:53.385678 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.notifications [None req-a177bbaa-18c8-4b7c-9205-fd944a00526f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '105d41e438e6421898223c9f12bebd9a', 'request_id': 'req-a177bbaa-18c8-4b7c-9205-fd944a00526f'} {{(pid=80144) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 10:54:53.392893 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.notifications [None req-a177bbaa-18c8-4b7c-9205-fd944a00526f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '105d41e438e6421898223c9f12bebd9a', 'request_id': 'req-a177bbaa-18c8-4b7c-9205-fd944a00526f'} {{(pid=80144) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 10:54:53.405340 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.notifications [None req-a177bbaa-18c8-4b7c-9205-fd944a00526f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '105d41e438e6421898223c9f12bebd9a', 'request_id': 'req-a177bbaa-18c8-4b7c-9205-fd944a00526f'} {{(pid=80144) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 10:54:53.450174 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.notifications [None req-a177bbaa-18c8-4b7c-9205-fd944a00526f admin admin] The token cache is being invalidate because project 105d41e438e6421898223c9f12bebd9a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80144) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 07 10:54:53.450489 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.notifications [None req-a177bbaa-18c8-4b7c-9205-fd944a00526f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80144) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 10:54:53.453112 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.notifications [None req-a177bbaa-18c8-4b7c-9205-fd944a00526f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80144) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 10:54:53.456732 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 959/1889] 10.222.0.25 () {58 vars in 1233 bytes} [Sun Aug 7 10:54:53 2022] DELETE /identity/v3/projects/105d41e438e6421898223c9f12bebd9a => generated 0 bytes in 187 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 07 10:54:53.462193 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a177bbaa-18c8-4b7c-9205-fd944a00526f admin admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 10:54:53.464006 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.fernet_utils [None req-a177bbaa-18c8-4b7c-9205-fd944a00526f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80144) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 10:54:53.481443 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=89,keystone:INSERT=4,keystone:DELETE=7 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 10:54:53.515329 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d682ac4-8ea4-4c8f-a83d-f99adb172356 admin admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 10:54:53.613773 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a177bbaa-18c8-4b7c-9205-fd944a00526f admin admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 10:54:53.629834 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f5e02de-fef6-43aa-bd6c-6c02eabd9556 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5ac47e3b09f43d19a23a0f457fe9e23', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 10:54:53.643894 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f5e02de-fef6-43aa-bd6c-6c02eabd9556 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:54:53.644273 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f5e02de-fef6-43aa-bd6c-6c02eabd9556 admin admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:54:53.644963 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f5e02de-fef6-43aa-bd6c-6c02eabd9556 admin admin] PATH_INFO: `/v3/users/38eee0c6e1984fe7968706a30667e779` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:54:53.646645 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f5e02de-fef6-43aa-bd6c-6c02eabd9556 admin admin] RBAC: Authorizing `identity:delete_user(user_id=38eee0c6e1984fe7968706a30667e779)` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 10:54:53.648882 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f5e02de-fef6-43aa-bd6c-6c02eabd9556 admin admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 10:54:53.676381 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6445656a-5bed-4346-9a21-b871f8ebd7ae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5ac47e3b09f43d19a23a0f457fe9e23', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 10:54:53.677250 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6445656a-5bed-4346-9a21-b871f8ebd7ae admin admin] REQUEST_METHOD: `POST` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:54:53.677534 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6445656a-5bed-4346-9a21-b871f8ebd7ae admin admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:54:53.677809 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6445656a-5bed-4346-9a21-b871f8ebd7ae admin admin] PATH_INFO: `/v3/users` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:54:53.679689 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6445656a-5bed-4346-9a21-b871f8ebd7ae admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 10:54:53.681647 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6445656a-5bed-4346-9a21-b871f8ebd7ae admin admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 10:54:53.727715 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.notifications [None req-3f5e02de-fef6-43aa-bd6c-6c02eabd9556 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '38eee0c6e1984fe7968706a30667e779'} {{(pid=80144) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 10:54:53.732552 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 931/1890] 10.222.0.25 () {60 vars in 1144 bytes} [Sun Aug 7 10:54:53 2022] POST /identity/v3/users => generated 299 bytes in 365 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 07 10:54:53.733791 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6445656a-5bed-4346-9a21-b871f8ebd7ae admin admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 10:54:53.740063 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-6445656a-5bed-4346-9a21-b871f8ebd7ae admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 10:54:53.749312 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.notifications [None req-3f5e02de-fef6-43aa-bd6c-6c02eabd9556 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '38eee0c6e1984fe7968706a30667e779'} {{(pid=80144) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 10:54:53.760691 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6445656a-5bed-4346-9a21-b871f8ebd7ae admin admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 10:54:53.763573 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.notifications [None req-3f5e02de-fef6-43aa-bd6c-6c02eabd9556 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '38eee0c6e1984fe7968706a30667e779'} {{(pid=80144) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 10:54:53.764261 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.notifications [None req-3f5e02de-fef6-43aa-bd6c-6c02eabd9556 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '38eee0c6e1984fe7968706a30667e779'} {{(pid=80144) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 10:54:53.765105 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.notifications [None req-3f5e02de-fef6-43aa-bd6c-6c02eabd9556 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '38eee0c6e1984fe7968706a30667e779'} {{(pid=80144) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 10:54:53.787324 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-289d73f6-86a9-4654-846a-3198fac1a674 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5ac47e3b09f43d19a23a0f457fe9e23', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 10:54:53.790909 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-289d73f6-86a9-4654-846a-3198fac1a674 admin admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:54:53.791820 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-289d73f6-86a9-4654-846a-3198fac1a674 admin admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:54:53.792542 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-289d73f6-86a9-4654-846a-3198fac1a674 admin admin] PATH_INFO: `/v3/roles` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:54:53.795902 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-289d73f6-86a9-4654-846a-3198fac1a674 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 10:54:53.803574 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-289d73f6-86a9-4654-846a-3198fac1a674 admin admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 10:54:53.825981 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 960/1891] 10.222.0.25 () {58 vars in 1224 bytes} [Sun Aug 7 10:54:53 2022] DELETE /identity/v3/users/38eee0c6e1984fe7968706a30667e779 => generated 0 bytes in 361 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 07 10:54:53.847946 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 932/1892] 10.222.0.25 () {58 vars in 1122 bytes} [Sun Aug 7 10:54:53 2022] GET /identity/v3/roles => generated 1420 bytes in 115 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 07 10:54:53.852585 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f5e02de-fef6-43aa-bd6c-6c02eabd9556 admin admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 10:54:53.853995 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.fernet_utils [None req-3f5e02de-fef6-43aa-bd6c-6c02eabd9556 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80144) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 10:54:53.965849 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f5e02de-fef6-43aa-bd6c-6c02eabd9556 admin admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 10:54:53.981357 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c24e8d32-1f18-4fd0-8d7d-3d8b73b87cd1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5ac47e3b09f43d19a23a0f457fe9e23', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 10:54:53.982385 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c24e8d32-1f18-4fd0-8d7d-3d8b73b87cd1 admin admin] REQUEST_METHOD: `PUT` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:54:53.982840 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c24e8d32-1f18-4fd0-8d7d-3d8b73b87cd1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:54:53.983623 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c24e8d32-1f18-4fd0-8d7d-3d8b73b87cd1 admin admin] PATH_INFO: `/v3/projects/25beed62fc9a44049a5693fc25c519f7/users/8bf3c99fd787437bb0178f1f3b9b77a4/roles/81b1f1bcee2a473088fbbd327c1c5b23` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:54:54.008975 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c24e8d32-1f18-4fd0-8d7d-3d8b73b87cd1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=25beed62fc9a44049a5693fc25c519f7, user_id=8bf3c99fd787437bb0178f1f3b9b77a4, role_id=81b1f1bcee2a473088fbbd327c1c5b23)` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 10:54:54.012772 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c24e8d32-1f18-4fd0-8d7d-3d8b73b87cd1 admin admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 10:54:54.039746 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 961/1893] 10.222.0.25 () {60 vars in 1483 bytes} [Sun Aug 7 10:54:53 2022] PUT /identity/v3/projects/25beed62fc9a44049a5693fc25c519f7/users/8bf3c99fd787437bb0178f1f3b9b77a4/roles/81b1f1bcee2a473088fbbd327c1c5b23 => generated 0 bytes in 188 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 07 10:54:54.043782 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-289d73f6-86a9-4654-846a-3198fac1a674 admin admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 10:54:54.056267 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-289d73f6-86a9-4654-846a-3198fac1a674 admin admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 10:54:54.074209 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 10:54:54.150265 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ae46586-7b45-42d9-8f88-84a6cb844666 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5ac47e3b09f43d19a23a0f457fe9e23', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 10:54:54.151076 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ae46586-7b45-42d9-8f88-84a6cb844666 admin admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:54:54.151557 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ae46586-7b45-42d9-8f88-84a6cb844666 admin admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:54:54.151845 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ae46586-7b45-42d9-8f88-84a6cb844666 admin admin] PATH_INFO: `/v3/roles` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:54:54.153337 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ae46586-7b45-42d9-8f88-84a6cb844666 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 10:54:54.155036 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ae46586-7b45-42d9-8f88-84a6cb844666 admin admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 10:54:54.169830 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 933/1894] 10.222.0.25 () {58 vars in 1122 bytes} [Sun Aug 7 10:54:54 2022] GET /identity/v3/roles => generated 1420 bytes in 126 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 07 10:54:54.175076 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c24e8d32-1f18-4fd0-8d7d-3d8b73b87cd1 admin admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 10:54:54.194103 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c24e8d32-1f18-4fd0-8d7d-3d8b73b87cd1 admin admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 10:54:54.200111 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf848fe9-b6c3-4664-a5cf-da93296bcb3e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5ac47e3b09f43d19a23a0f457fe9e23', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 10:54:54.206229 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf848fe9-b6c3-4664-a5cf-da93296bcb3e admin admin] REQUEST_METHOD: `PUT` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:54:54.206522 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf848fe9-b6c3-4664-a5cf-da93296bcb3e admin admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:54:54.206807 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf848fe9-b6c3-4664-a5cf-da93296bcb3e admin admin] PATH_INFO: `/v3/projects/25beed62fc9a44049a5693fc25c519f7/users/8bf3c99fd787437bb0178f1f3b9b77a4/roles/81b1f1bcee2a473088fbbd327c1c5b23` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:54:54.209015 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf848fe9-b6c3-4664-a5cf-da93296bcb3e admin admin] RBAC: Authorizing `identity:create_grant(project_id=25beed62fc9a44049a5693fc25c519f7, user_id=8bf3c99fd787437bb0178f1f3b9b77a4, role_id=81b1f1bcee2a473088fbbd327c1c5b23)` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 10:54:54.211857 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf848fe9-b6c3-4664-a5cf-da93296bcb3e admin admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 10:54:54.223043 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=87,keystone:DELETE=10,keystone:INSERT=3 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 10:54:54.234363 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 962/1895] 10.222.0.25 () {60 vars in 1483 bytes} [Sun Aug 7 10:54:54 2022] PUT /identity/v3/projects/25beed62fc9a44049a5693fc25c519f7/users/8bf3c99fd787437bb0178f1f3b9b77a4/roles/81b1f1bcee2a473088fbbd327c1c5b23 => generated 0 bytes in 58 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 07 10:54:54.244636 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f0c8a2a-4612-4491-a470-2b6a247e3133 None None] REQUEST_METHOD: `POST` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:54:54.244927 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f0c8a2a-4612-4491-a470-2b6a247e3133 None None] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:54:54.245210 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f0c8a2a-4612-4491-a470-2b6a247e3133 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:54:54.307557 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.auth.core [None req-8f0c8a2a-4612-4491-a470-2b6a247e3133 None None] MFA Rules not processed for user `8bf3c99fd787437bb0178f1f3b9b77a4`. Rule list: `[]` (Enabled: `True`). {{(pid=80143) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 10:54:54.317021 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-8f0c8a2a-4612-4491-a470-2b6a247e3133 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 10:54:54.463411 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 934/1896] 10.222.0.25 () {58 vars in 958 bytes} [Sun Aug 7 10:54:54 2022] POST /identity/v3/auth/tokens => generated 3329 bytes in 220 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 07 10:54:54.477140 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf848fe9-b6c3-4664-a5cf-da93296bcb3e admin admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 10:54:54.480523 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.fernet_utils [None req-cf848fe9-b6c3-4664-a5cf-da93296bcb3e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80144) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 10:54:54.569475 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf848fe9-b6c3-4664-a5cf-da93296bcb3e admin admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 10:54:54.579852 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27d080b5-1438-49ba-9051-a8f2034d21fb service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88dbd6dfd1164994a3e16da553561910', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6de1f3cac02c4bcdb05dd2337176033d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 10:54:54.580897 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27d080b5-1438-49ba-9051-a8f2034d21fb service cinder] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:54:54.581904 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27d080b5-1438-49ba-9051-a8f2034d21fb service cinder] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:54:54.582249 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27d080b5-1438-49ba-9051-a8f2034d21fb service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:54:54.584338 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.fernet_utils [None req-27d080b5-1438-49ba-9051-a8f2034d21fb service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80144) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 10:54:54.761295 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f0c8a2a-4612-4491-a470-2b6a247e3133 None None] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 10:54:54.762754 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-8f0c8a2a-4612-4491-a470-2b6a247e3133 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 10:54:54.764538 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27d080b5-1438-49ba-9051-a8f2034d21fb service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 10:54:54.780151 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27d080b5-1438-49ba-9051-a8f2034d21fb service cinder] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 10:54:54.815039 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 963/1897] 10.222.0.25 () {60 vars in 1365 bytes} [Sun Aug 7 10:54:54 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 339 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 07 10:54:54.888584 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f0c8a2a-4612-4491-a470-2b6a247e3133 None None] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 10:54:54.896934 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bb56975-0efa-4193-9f72-f5260d7d22d6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5ac47e3b09f43d19a23a0f457fe9e23', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 10:54:54.897716 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bb56975-0efa-4193-9f72-f5260d7d22d6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:54:54.898343 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bb56975-0efa-4193-9f72-f5260d7d22d6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:54:54.898743 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bb56975-0efa-4193-9f72-f5260d7d22d6 admin admin] PATH_INFO: `/v3/projects/f9e32fe227d947a889edc33cde5525f1` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:54:54.901610 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bb56975-0efa-4193-9f72-f5260d7d22d6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f9e32fe227d947a889edc33cde5525f1)` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 10:54:54.903229 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bb56975-0efa-4193-9f72-f5260d7d22d6 admin admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 10:54:54.980733 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=97,keystone:DELETE=3 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 10:54:55.002609 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-6bb56975-0efa-4193-9f72-f5260d7d22d6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f9e32fe227d947a889edc33cde5525f1', 'request_id': 'req-6bb56975-0efa-4193-9f72-f5260d7d22d6'} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 10:54:55.009446 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-6bb56975-0efa-4193-9f72-f5260d7d22d6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f9e32fe227d947a889edc33cde5525f1', 'request_id': 'req-6bb56975-0efa-4193-9f72-f5260d7d22d6'} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 10:54:55.017497 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-6bb56975-0efa-4193-9f72-f5260d7d22d6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f9e32fe227d947a889edc33cde5525f1', 'request_id': 'req-6bb56975-0efa-4193-9f72-f5260d7d22d6'} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 10:54:55.049978 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-6bb56975-0efa-4193-9f72-f5260d7d22d6 admin admin] The token cache is being invalidate because project f9e32fe227d947a889edc33cde5525f1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80143) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 07 10:54:55.049978 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-6bb56975-0efa-4193-9f72-f5260d7d22d6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 10:54:55.050513 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-6bb56975-0efa-4193-9f72-f5260d7d22d6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 10:54:55.052395 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 935/1898] 10.222.0.25 () {58 vars in 1233 bytes} [Sun Aug 7 10:54:54 2022] DELETE /identity/v3/projects/f9e32fe227d947a889edc33cde5525f1 => generated 0 bytes in 291 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 07 10:54:55.057143 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-490205c6-b648-481d-a48c-a3d0e7c30be7 None None] REQUEST_METHOD: `POST` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:54:55.057453 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-490205c6-b648-481d-a48c-a3d0e7c30be7 None None] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:54:55.057736 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-490205c6-b648-481d-a48c-a3d0e7c30be7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:54:55.091278 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.auth.core [None req-490205c6-b648-481d-a48c-a3d0e7c30be7 None None] MFA Rules not processed for user `bac4e6b76fe64217ac8fa44e71cffe6a`. Rule list: `[]` (Enabled: `True`). {{(pid=80144) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 10:54:55.092445 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.fernet_utils [None req-490205c6-b648-481d-a48c-a3d0e7c30be7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80144) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 10:54:55.178967 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 964/1899] 10.222.0.25 () {58 vars in 958 bytes} [Sun Aug 7 10:54:55 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 07 10:54:55.189076 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-553f863a-f39b-443b-b4c9-cd4dd0b653c6 None None] REQUEST_METHOD: `POST` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:54:55.189604 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-553f863a-f39b-443b-b4c9-cd4dd0b653c6 None None] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:54:55.189909 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-553f863a-f39b-443b-b4c9-cd4dd0b653c6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:54:55.226341 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.auth.core [None req-553f863a-f39b-443b-b4c9-cd4dd0b653c6 None None] MFA Rules not processed for user `bac4e6b76fe64217ac8fa44e71cffe6a`. Rule list: `[]` (Enabled: `True`). {{(pid=80143) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 10:54:55.227885 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-553f863a-f39b-443b-b4c9-cd4dd0b653c6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 10:54:55.238450 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 936/1900] 10.222.0.25 () {58 vars in 958 bytes} [Sun Aug 7 10:54:55 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 51 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 07 10:54:55.243062 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-490205c6-b648-481d-a48c-a3d0e7c30be7 None None] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 10:54:55.253773 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-490205c6-b648-481d-a48c-a3d0e7c30be7 None None] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 10:54:55.259416 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fd5eadb-1ad1-4a11-b6d0-28b1093cd6ce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5ac47e3b09f43d19a23a0f457fe9e23', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 10:54:55.260107 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fd5eadb-1ad1-4a11-b6d0-28b1093cd6ce admin admin] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:54:55.260394 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fd5eadb-1ad1-4a11-b6d0-28b1093cd6ce admin admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:54:55.271226 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fd5eadb-1ad1-4a11-b6d0-28b1093cd6ce admin admin] PATH_INFO: `/v3/domains` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:54:55.272016 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fd5eadb-1ad1-4a11-b6d0-28b1093cd6ce admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80144) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 10:54:55.272391 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fd5eadb-1ad1-4a11-b6d0-28b1093cd6ce admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 10:54:55.278650 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fd5eadb-1ad1-4a11-b6d0-28b1093cd6ce admin admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 10:54:55.314810 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 965/1901] 10.222.0.25 () {58 vars in 1153 bytes} [Sun Aug 7 10:54:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 69 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 10:54:55.316020 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-553f863a-f39b-443b-b4c9-cd4dd0b653c6 None None] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 10:54:55.326757 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-553f863a-f39b-443b-b4c9-cd4dd0b653c6 None None] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 10:54:55.335137 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a72344ce-4c3b-4e1e-af05-922ed8be6f00 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5ac47e3b09f43d19a23a0f457fe9e23', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 10:54:55.335137 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a72344ce-4c3b-4e1e-af05-922ed8be6f00 admin admin] REQUEST_METHOD: `POST` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:54:55.335137 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a72344ce-4c3b-4e1e-af05-922ed8be6f00 admin admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:54:55.335137 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a72344ce-4c3b-4e1e-af05-922ed8be6f00 admin admin] PATH_INFO: `/v3/projects` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:54:55.335547 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a72344ce-4c3b-4e1e-af05-922ed8be6f00 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 10:54:55.336662 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a72344ce-4c3b-4e1e-af05-922ed8be6f00 admin admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 10:54:55.366738 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 937/1902] 10.222.0.25 () {60 vars in 1153 bytes} [Sun Aug 7 10:54:55 2022] POST /identity/v3/projects => generated 377 bytes in 51 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 07 10:54:55.371033 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fd5eadb-1ad1-4a11-b6d0-28b1093cd6ce admin admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 10:54:55.383361 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fd5eadb-1ad1-4a11-b6d0-28b1093cd6ce admin admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 10:54:55.441379 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=100 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 10:54:55.461620 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a27e3a0c-c58a-4a97-aa12-daad5801c0a7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5ac47e3b09f43d19a23a0f457fe9e23', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 10:54:55.462303 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a27e3a0c-c58a-4a97-aa12-daad5801c0a7 admin admin] REQUEST_METHOD: `POST` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:54:55.462594 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a27e3a0c-c58a-4a97-aa12-daad5801c0a7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:54:55.462873 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a27e3a0c-c58a-4a97-aa12-daad5801c0a7 admin admin] PATH_INFO: `/v3/users` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:54:55.463547 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a27e3a0c-c58a-4a97-aa12-daad5801c0a7 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 10:54:55.466025 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a27e3a0c-c58a-4a97-aa12-daad5801c0a7 admin admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 10:54:55.488130 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 966/1903] 10.222.0.25 () {60 vars in 1144 bytes} [Sun Aug 7 10:54:55 2022] POST /identity/v3/users => generated 289 bytes in 118 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 07 10:54:55.492178 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a72344ce-4c3b-4e1e-af05-922ed8be6f00 admin admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 10:54:55.503404 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a72344ce-4c3b-4e1e-af05-922ed8be6f00 admin admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 10:54:55.508033 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc82f87a-73c4-4ce1-a8e1-7d42c88e622e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5ac47e3b09f43d19a23a0f457fe9e23', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 10:54:55.508637 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc82f87a-73c4-4ce1-a8e1-7d42c88e622e admin admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:54:55.508913 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc82f87a-73c4-4ce1-a8e1-7d42c88e622e admin admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:54:55.509187 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc82f87a-73c4-4ce1-a8e1-7d42c88e622e admin admin] PATH_INFO: `/v3/roles` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:54:55.509821 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc82f87a-73c4-4ce1-a8e1-7d42c88e622e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 10:54:55.511439 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc82f87a-73c4-4ce1-a8e1-7d42c88e622e admin admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 10:54:55.523047 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 938/1904] 10.222.0.25 () {58 vars in 1122 bytes} [Sun Aug 7 10:54:55 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 07 10:54:55.527491 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a27e3a0c-c58a-4a97-aa12-daad5801c0a7 admin admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 10:54:55.538457 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a27e3a0c-c58a-4a97-aa12-daad5801c0a7 admin admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 10:54:55.543452 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e66ca1a3-a6f3-4357-9529-91b4d239bac0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5ac47e3b09f43d19a23a0f457fe9e23', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 10:54:55.544039 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e66ca1a3-a6f3-4357-9529-91b4d239bac0 admin admin] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:54:55.544317 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e66ca1a3-a6f3-4357-9529-91b4d239bac0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:54:55.544589 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e66ca1a3-a6f3-4357-9529-91b4d239bac0 admin admin] PATH_INFO: `/v3/roles` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:54:55.545255 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e66ca1a3-a6f3-4357-9529-91b4d239bac0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 10:54:55.546836 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e66ca1a3-a6f3-4357-9529-91b4d239bac0 admin admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 10:54:55.558290 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 967/1905] 10.222.0.25 () {58 vars in 1122 bytes} [Sun Aug 7 10:54:55 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 07 10:54:55.562537 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc82f87a-73c4-4ce1-a8e1-7d42c88e622e admin admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 10:54:55.573957 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc82f87a-73c4-4ce1-a8e1-7d42c88e622e admin admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 10:54:55.578707 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-457d252c-ccd7-42ff-9e0b-7dad8eb49919 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5ac47e3b09f43d19a23a0f457fe9e23', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 10:54:55.579431 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-457d252c-ccd7-42ff-9e0b-7dad8eb49919 admin admin] REQUEST_METHOD: `PUT` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:54:55.579732 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-457d252c-ccd7-42ff-9e0b-7dad8eb49919 admin admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:54:55.580034 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-457d252c-ccd7-42ff-9e0b-7dad8eb49919 admin admin] PATH_INFO: `/v3/projects/3e02268d7a254dc08ae29fc6b691bb9a/users/9c442cfdc7d3434486887d36250a7976/roles/b98dfb3b9fee4644af2887a3ac3fd2e6` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:54:55.597812 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-457d252c-ccd7-42ff-9e0b-7dad8eb49919 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3e02268d7a254dc08ae29fc6b691bb9a, user_id=9c442cfdc7d3434486887d36250a7976, role_id=b98dfb3b9fee4644af2887a3ac3fd2e6)` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 10:54:55.601990 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-457d252c-ccd7-42ff-9e0b-7dad8eb49919 admin admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 10:54:55.623502 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 939/1906] 10.222.0.25 () {60 vars in 1483 bytes} [Sun Aug 7 10:54:55 2022] PUT /identity/v3/projects/3e02268d7a254dc08ae29fc6b691bb9a/users/9c442cfdc7d3434486887d36250a7976/roles/b98dfb3b9fee4644af2887a3ac3fd2e6 => generated 0 bytes in 61 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 07 10:54:55.635090 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a574b1a-3e56-4b1e-8eed-f41439378d54 None None] REQUEST_METHOD: `POST` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:54:55.635387 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a574b1a-3e56-4b1e-8eed-f41439378d54 None None] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:54:55.635668 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a574b1a-3e56-4b1e-8eed-f41439378d54 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:54:55.668308 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.auth.core [None req-7a574b1a-3e56-4b1e-8eed-f41439378d54 None None] MFA Rules not processed for user `9c442cfdc7d3434486887d36250a7976`. Rule list: `[]` (Enabled: `True`). {{(pid=80144) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 10:54:55.671646 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.fernet_utils [None req-7a574b1a-3e56-4b1e-8eed-f41439378d54 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80144) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 10:54:55.765513 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 968/1907] 10.222.0.25 () {58 vars in 958 bytes} [Sun Aug 7 10:54:55 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 07 10:54:55.771156 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-457d252c-ccd7-42ff-9e0b-7dad8eb49919 admin admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 10:54:55.905572 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-457d252c-ccd7-42ff-9e0b-7dad8eb49919 admin admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 10:54:55.912120 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b73db919-322d-4d27-b662-61492f5acc56 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5ac47e3b09f43d19a23a0f457fe9e23', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 10:54:55.912958 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b73db919-322d-4d27-b662-61492f5acc56 admin admin] REQUEST_METHOD: `POST` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:54:55.913460 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b73db919-322d-4d27-b662-61492f5acc56 admin admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:54:55.914407 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b73db919-322d-4d27-b662-61492f5acc56 admin admin] PATH_INFO: `/v3/projects` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:54:55.915010 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b73db919-322d-4d27-b662-61492f5acc56 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 10:54:55.917206 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b73db919-322d-4d27-b662-61492f5acc56 admin admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 10:54:55.949594 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 940/1908] 10.222.0.25 () {60 vars in 1153 bytes} [Sun Aug 7 10:54:55 2022] POST /identity/v3/projects => generated 375 bytes in 179 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 07 10:54:55.954508 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a574b1a-3e56-4b1e-8eed-f41439378d54 None None] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 10:54:55.973894 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a574b1a-3e56-4b1e-8eed-f41439378d54 None None] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 10:54:56.051667 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1d6a9f0-d4a9-4fcb-86e9-c8ff833fab88 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5ac47e3b09f43d19a23a0f457fe9e23', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 10:54:56.052281 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1d6a9f0-d4a9-4fcb-86e9-c8ff833fab88 admin admin] REQUEST_METHOD: `POST` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:54:56.052560 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1d6a9f0-d4a9-4fcb-86e9-c8ff833fab88 admin admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:54:56.052830 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1d6a9f0-d4a9-4fcb-86e9-c8ff833fab88 admin admin] PATH_INFO: `/v3/users` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:54:56.053445 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1d6a9f0-d4a9-4fcb-86e9-c8ff833fab88 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 10:54:56.055400 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1d6a9f0-d4a9-4fcb-86e9-c8ff833fab88 admin admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 10:54:56.081218 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 969/1909] 10.222.0.25 () {60 vars in 1144 bytes} [Sun Aug 7 10:54:55 2022] POST /identity/v3/users => generated 294 bytes in 126 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 07 10:54:56.083254 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b73db919-322d-4d27-b662-61492f5acc56 admin admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 10:54:56.094497 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b73db919-322d-4d27-b662-61492f5acc56 admin admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 10:54:56.100369 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4acc13ce-d719-4b63-b03a-6aa7d88280db admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5ac47e3b09f43d19a23a0f457fe9e23', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 10:54:56.101273 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4acc13ce-d719-4b63-b03a-6aa7d88280db admin admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:54:56.101630 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4acc13ce-d719-4b63-b03a-6aa7d88280db admin admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:54:56.101946 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4acc13ce-d719-4b63-b03a-6aa7d88280db admin admin] PATH_INFO: `/v3/roles` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:54:56.103027 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4acc13ce-d719-4b63-b03a-6aa7d88280db admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 10:54:56.105246 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4acc13ce-d719-4b63-b03a-6aa7d88280db admin admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 10:54:56.118282 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 941/1910] 10.222.0.25 () {58 vars in 1122 bytes} [Sun Aug 7 10:54:56 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 07 10:54:56.122524 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1d6a9f0-d4a9-4fcb-86e9-c8ff833fab88 admin admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 10:54:56.134748 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1d6a9f0-d4a9-4fcb-86e9-c8ff833fab88 admin admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 10:54:56.139459 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fb29de9-917f-4c22-ad72-40716b81d6c6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5ac47e3b09f43d19a23a0f457fe9e23', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 10:54:56.140151 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fb29de9-917f-4c22-ad72-40716b81d6c6 admin admin] REQUEST_METHOD: `PUT` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:54:56.140427 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fb29de9-917f-4c22-ad72-40716b81d6c6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:54:56.140751 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fb29de9-917f-4c22-ad72-40716b81d6c6 admin admin] PATH_INFO: `/v3/projects/c733617fe0ff49e28c5641f347b61228/users/8fad464767a9417ca3116a1a1eb82d70/roles/81b1f1bcee2a473088fbbd327c1c5b23` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:54:56.161923 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fb29de9-917f-4c22-ad72-40716b81d6c6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c733617fe0ff49e28c5641f347b61228, user_id=8fad464767a9417ca3116a1a1eb82d70, role_id=81b1f1bcee2a473088fbbd327c1c5b23)` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 10:54:56.164589 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fb29de9-917f-4c22-ad72-40716b81d6c6 admin admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 10:54:56.179732 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 970/1911] 10.222.0.25 () {60 vars in 1483 bytes} [Sun Aug 7 10:54:56 2022] PUT /identity/v3/projects/c733617fe0ff49e28c5641f347b61228/users/8fad464767a9417ca3116a1a1eb82d70/roles/81b1f1bcee2a473088fbbd327c1c5b23 => generated 0 bytes in 57 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 07 10:54:56.183676 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4acc13ce-d719-4b63-b03a-6aa7d88280db admin admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 10:54:56.195018 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4acc13ce-d719-4b63-b03a-6aa7d88280db admin admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 10:54:56.263370 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fb29de9-917f-4c22-ad72-40716b81d6c6 admin admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 10:54:56.266075 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.fernet_utils [None req-4fb29de9-917f-4c22-ad72-40716b81d6c6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80144) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 10:54:56.276041 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=92,keystone:DELETE=4,keystone:INSERT=4 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 10:54:56.294713 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 10:54:56.304127 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5854fb75-8975-419e-9db5-fa3cd79e4b83 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5ac47e3b09f43d19a23a0f457fe9e23', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 10:54:56.304835 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5854fb75-8975-419e-9db5-fa3cd79e4b83 admin admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:54:56.305178 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5854fb75-8975-419e-9db5-fa3cd79e4b83 admin admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:54:56.305523 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5854fb75-8975-419e-9db5-fa3cd79e4b83 admin admin] PATH_INFO: `/v3/roles` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:54:56.306285 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5854fb75-8975-419e-9db5-fa3cd79e4b83 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 10:54:56.308003 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5854fb75-8975-419e-9db5-fa3cd79e4b83 admin admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 10:54:56.327079 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 942/1912] 10.222.0.25 () {58 vars in 1122 bytes} [Sun Aug 7 10:54:56 2022] GET /identity/v3/roles => generated 1420 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 07 10:54:56.327655 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5854fb75-8975-419e-9db5-fa3cd79e4b83 admin admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 10:54:56.339978 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5854fb75-8975-419e-9db5-fa3cd79e4b83 admin admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 10:54:56.350992 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a42cf58c-6b2f-430f-af55-3f9fed7833df admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5ac47e3b09f43d19a23a0f457fe9e23', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 10:54:56.351853 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a42cf58c-6b2f-430f-af55-3f9fed7833df admin admin] REQUEST_METHOD: `PUT` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:54:56.352303 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a42cf58c-6b2f-430f-af55-3f9fed7833df admin admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:54:56.352586 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a42cf58c-6b2f-430f-af55-3f9fed7833df admin admin] PATH_INFO: `/v3/projects/c733617fe0ff49e28c5641f347b61228/users/8fad464767a9417ca3116a1a1eb82d70/roles/81b1f1bcee2a473088fbbd327c1c5b23` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:54:56.355701 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a42cf58c-6b2f-430f-af55-3f9fed7833df admin admin] RBAC: Authorizing `identity:create_grant(project_id=c733617fe0ff49e28c5641f347b61228, user_id=8fad464767a9417ca3116a1a1eb82d70, role_id=81b1f1bcee2a473088fbbd327c1c5b23)` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 10:54:56.357896 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fb29de9-917f-4c22-ad72-40716b81d6c6 admin admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 10:54:56.360421 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a42cf58c-6b2f-430f-af55-3f9fed7833df admin admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 10:54:56.364254 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a72e05bf-87f3-4781-8dad-834ec297537c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5ac47e3b09f43d19a23a0f457fe9e23', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 10:54:56.365016 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a72e05bf-87f3-4781-8dad-834ec297537c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:54:56.365371 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a72e05bf-87f3-4781-8dad-834ec297537c admin admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:54:56.365719 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a72e05bf-87f3-4781-8dad-834ec297537c admin admin] PATH_INFO: `/v3/users/ba31bfb88d074806a7c4f79af716a3b2` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:54:56.366965 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a72e05bf-87f3-4781-8dad-834ec297537c admin admin] RBAC: Authorizing `identity:delete_user(user_id=ba31bfb88d074806a7c4f79af716a3b2)` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 10:54:56.368959 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a72e05bf-87f3-4781-8dad-834ec297537c admin admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 10:54:56.387255 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 943/1913] 10.222.0.25 () {60 vars in 1483 bytes} [Sun Aug 7 10:54:56 2022] PUT /identity/v3/projects/c733617fe0ff49e28c5641f347b61228/users/8fad464767a9417ca3116a1a1eb82d70/roles/81b1f1bcee2a473088fbbd327c1c5b23 => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 07 10:54:56.393808 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ddee370-2d02-4c1f-9d96-612122158400 None None] REQUEST_METHOD: `POST` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:54:56.395587 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ddee370-2d02-4c1f-9d96-612122158400 None None] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:54:56.396302 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ddee370-2d02-4c1f-9d96-612122158400 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:54:56.455182 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.auth.core [None req-5ddee370-2d02-4c1f-9d96-612122158400 None None] MFA Rules not processed for user `8fad464767a9417ca3116a1a1eb82d70`. Rule list: `[]` (Enabled: `True`). {{(pid=80143) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 10:54:56.456516 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-5ddee370-2d02-4c1f-9d96-612122158400 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 10:54:56.468849 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.notifications [None req-a72e05bf-87f3-4781-8dad-834ec297537c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ba31bfb88d074806a7c4f79af716a3b2'} {{(pid=80144) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 10:54:56.477297 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.notifications [None req-a72e05bf-87f3-4781-8dad-834ec297537c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ba31bfb88d074806a7c4f79af716a3b2'} {{(pid=80144) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 10:54:56.487812 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.notifications [None req-a72e05bf-87f3-4781-8dad-834ec297537c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ba31bfb88d074806a7c4f79af716a3b2'} {{(pid=80144) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 10:54:56.488796 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.notifications [None req-a72e05bf-87f3-4781-8dad-834ec297537c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ba31bfb88d074806a7c4f79af716a3b2'} {{(pid=80144) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 10:54:56.490326 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.notifications [None req-a72e05bf-87f3-4781-8dad-834ec297537c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ba31bfb88d074806a7c4f79af716a3b2'} {{(pid=80144) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 10:54:56.555830 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 971/1914] 10.222.0.25 () {58 vars in 1224 bytes} [Sun Aug 7 10:54:56 2022] DELETE /identity/v3/users/ba31bfb88d074806a7c4f79af716a3b2 => generated 0 bytes in 289 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 07 10:54:56.571075 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-045f9425-5ca7-4f53-8744-1373f6ba85e0 None None] REQUEST_METHOD: `POST` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:54:56.571373 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-045f9425-5ca7-4f53-8744-1373f6ba85e0 None None] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:54:56.571658 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-045f9425-5ca7-4f53-8744-1373f6ba85e0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:54:56.615721 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.auth.core [None req-045f9425-5ca7-4f53-8744-1373f6ba85e0 None None] MFA Rules not processed for user `bac4e6b76fe64217ac8fa44e71cffe6a`. Rule list: `[]` (Enabled: `True`). {{(pid=80144) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 10:54:56.627295 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.fernet_utils [None req-045f9425-5ca7-4f53-8744-1373f6ba85e0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80144) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 10:54:56.697139 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 944/1915] 10.222.0.25 () {58 vars in 958 bytes} [Sun Aug 7 10:54:56 2022] POST /identity/v3/auth/tokens => generated 3319 bytes in 308 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 07 10:54:56.710300 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ddee370-2d02-4c1f-9d96-612122158400 None None] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 10:54:56.714592 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 972/1916] 10.222.0.25 () {58 vars in 958 bytes} [Sun Aug 7 10:54:56 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 156 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 07 10:54:56.715910 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-5ddee370-2d02-4c1f-9d96-612122158400 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 10:54:56.746175 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-045f9425-5ca7-4f53-8744-1373f6ba85e0 None None] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 10:54:56.748519 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.fernet_utils [None req-045f9425-5ca7-4f53-8744-1373f6ba85e0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80144) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 10:54:56.817519 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 10:54:56.823405 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ddee370-2d02-4c1f-9d96-612122158400 None None] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 10:54:56.862030 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-045f9425-5ca7-4f53-8744-1373f6ba85e0 None None] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 10:54:56.876531 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc178a6d-8e8f-41f3-bccf-514e5f4f8e07 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be1283c9bf47424c9aa3ad27cab03027', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6de1f3cac02c4bcdb05dd2337176033d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 10:54:56.877359 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc178a6d-8e8f-41f3-bccf-514e5f4f8e07 service neutron] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:54:56.877699 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc178a6d-8e8f-41f3-bccf-514e5f4f8e07 service neutron] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:54:56.877977 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc178a6d-8e8f-41f3-bccf-514e5f4f8e07 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:54:56.904682 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc178a6d-8e8f-41f3-bccf-514e5f4f8e07 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 10:54:56.909013 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc178a6d-8e8f-41f3-bccf-514e5f4f8e07 service neutron] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 10:54:56.931202 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daf030a5-b835-436d-b21e-18202fa35a4a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88dbd6dfd1164994a3e16da553561910', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6de1f3cac02c4bcdb05dd2337176033d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 10:54:56.933546 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 973/1917] 10.222.0.25 () {60 vars in 1365 bytes} [Sun Aug 7 10:54:56 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 189 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 07 10:54:56.933728 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daf030a5-b835-436d-b21e-18202fa35a4a service cinder] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:54:56.934043 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daf030a5-b835-436d-b21e-18202fa35a4a service cinder] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:54:56.934394 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daf030a5-b835-436d-b21e-18202fa35a4a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:54:56.937217 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-daf030a5-b835-436d-b21e-18202fa35a4a service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 10:54:56.966850 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 10:54:57.093729 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daf030a5-b835-436d-b21e-18202fa35a4a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 10:54:57.103258 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daf030a5-b835-436d-b21e-18202fa35a4a service cinder] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 10:54:57.124020 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 945/1918] 10.222.0.25 () {60 vars in 1365 bytes} [Sun Aug 7 10:54:56 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 414 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 07 10:54:57.464832 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc178a6d-8e8f-41f3-bccf-514e5f4f8e07 service neutron] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 10:54:57.466576 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.fernet_utils [None req-dc178a6d-8e8f-41f3-bccf-514e5f4f8e07 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80144) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 10:54:57.494689 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc178a6d-8e8f-41f3-bccf-514e5f4f8e07 service neutron] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 10:54:57.501761 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eccf3f10-91f4-48c0-82cf-8071ff07862d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5ac47e3b09f43d19a23a0f457fe9e23', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 10:54:57.502484 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eccf3f10-91f4-48c0-82cf-8071ff07862d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:54:57.502779 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eccf3f10-91f4-48c0-82cf-8071ff07862d admin admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:54:57.504322 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eccf3f10-91f4-48c0-82cf-8071ff07862d admin admin] PATH_INFO: `/v3/projects/db541d7336684717aacdd90e77341550` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:54:57.506448 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eccf3f10-91f4-48c0-82cf-8071ff07862d admin admin] RBAC: Authorizing `identity:delete_project(project_id=db541d7336684717aacdd90e77341550)` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 10:54:57.508568 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eccf3f10-91f4-48c0-82cf-8071ff07862d admin admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 10:54:57.627537 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.notifications [None req-eccf3f10-91f4-48c0-82cf-8071ff07862d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'db541d7336684717aacdd90e77341550', 'request_id': 'req-eccf3f10-91f4-48c0-82cf-8071ff07862d'} {{(pid=80144) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 10:54:57.637154 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.notifications [None req-eccf3f10-91f4-48c0-82cf-8071ff07862d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'db541d7336684717aacdd90e77341550', 'request_id': 'req-eccf3f10-91f4-48c0-82cf-8071ff07862d'} {{(pid=80144) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 10:54:57.648265 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.notifications [None req-eccf3f10-91f4-48c0-82cf-8071ff07862d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'db541d7336684717aacdd90e77341550', 'request_id': 'req-eccf3f10-91f4-48c0-82cf-8071ff07862d'} {{(pid=80144) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 10:54:57.666654 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.notifications [None req-eccf3f10-91f4-48c0-82cf-8071ff07862d admin admin] The token cache is being invalidate because project db541d7336684717aacdd90e77341550 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80144) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 07 10:54:57.667153 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.notifications [None req-eccf3f10-91f4-48c0-82cf-8071ff07862d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80144) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 10:54:57.668094 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.notifications [None req-eccf3f10-91f4-48c0-82cf-8071ff07862d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80144) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 10:54:57.670507 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 974/1919] 10.222.0.25 () {58 vars in 1233 bytes} [Sun Aug 7 10:54:57 2022] DELETE /identity/v3/projects/db541d7336684717aacdd90e77341550 => generated 0 bytes in 207 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 07 10:54:57.675000 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daf030a5-b835-436d-b21e-18202fa35a4a service cinder] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 10:54:57.693852 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daf030a5-b835-436d-b21e-18202fa35a4a service cinder] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 10:54:57.794480 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c51b3d5-f18a-4783-acc8-0f97d0f513a4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5ac47e3b09f43d19a23a0f457fe9e23', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 10:54:57.795169 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c51b3d5-f18a-4783-acc8-0f97d0f513a4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:54:57.795471 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c51b3d5-f18a-4783-acc8-0f97d0f513a4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:54:57.795746 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c51b3d5-f18a-4783-acc8-0f97d0f513a4 admin admin] PATH_INFO: `/v3/users/1f550b79e60a405cacb953d861bad2b7` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:54:57.798100 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c51b3d5-f18a-4783-acc8-0f97d0f513a4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=1f550b79e60a405cacb953d861bad2b7)` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 10:54:57.800263 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c51b3d5-f18a-4783-acc8-0f97d0f513a4 admin admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 10:54:57.871730 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-2c51b3d5-f18a-4783-acc8-0f97d0f513a4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1f550b79e60a405cacb953d861bad2b7'} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 10:54:57.880499 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-2c51b3d5-f18a-4783-acc8-0f97d0f513a4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1f550b79e60a405cacb953d861bad2b7'} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 10:54:57.893844 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-2c51b3d5-f18a-4783-acc8-0f97d0f513a4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1f550b79e60a405cacb953d861bad2b7'} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 10:54:57.894591 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-2c51b3d5-f18a-4783-acc8-0f97d0f513a4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1f550b79e60a405cacb953d861bad2b7'} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 10:54:57.904621 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-2c51b3d5-f18a-4783-acc8-0f97d0f513a4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1f550b79e60a405cacb953d861bad2b7'} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 10:54:57.913618 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 946/1920] 10.222.0.25 () {58 vars in 1224 bytes} [Sun Aug 7 10:54:57 2022] DELETE /identity/v3/users/1f550b79e60a405cacb953d861bad2b7 => generated 0 bytes in 239 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 07 10:54:58.204229 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eccf3f10-91f4-48c0-82cf-8071ff07862d admin admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 10:54:58.205958 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.fernet_utils [None req-eccf3f10-91f4-48c0-82cf-8071ff07862d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80144) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 10:54:58.221695 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eccf3f10-91f4-48c0-82cf-8071ff07862d admin admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 10:54:58.297950 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-137eb412-a2ba-4708-b89d-6db3a7a5c6ad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5ac47e3b09f43d19a23a0f457fe9e23', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 10:54:58.298590 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-137eb412-a2ba-4708-b89d-6db3a7a5c6ad admin admin] REQUEST_METHOD: `DELETE` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:54:58.298874 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-137eb412-a2ba-4708-b89d-6db3a7a5c6ad admin admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:54:58.299313 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-137eb412-a2ba-4708-b89d-6db3a7a5c6ad admin admin] PATH_INFO: `/v3/projects/758d124435054ee0a53b42100561728f` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:54:58.301127 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-137eb412-a2ba-4708-b89d-6db3a7a5c6ad admin admin] RBAC: Authorizing `identity:delete_project(project_id=758d124435054ee0a53b42100561728f)` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 10:54:58.303060 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-137eb412-a2ba-4708-b89d-6db3a7a5c6ad admin admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 10:54:58.351194 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.notifications [None req-137eb412-a2ba-4708-b89d-6db3a7a5c6ad admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '758d124435054ee0a53b42100561728f', 'request_id': 'req-137eb412-a2ba-4708-b89d-6db3a7a5c6ad'} {{(pid=80144) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 10:54:58.356769 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.notifications [None req-137eb412-a2ba-4708-b89d-6db3a7a5c6ad admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '758d124435054ee0a53b42100561728f', 'request_id': 'req-137eb412-a2ba-4708-b89d-6db3a7a5c6ad'} {{(pid=80144) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 10:54:58.361023 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 10:54:58.365547 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.notifications [None req-137eb412-a2ba-4708-b89d-6db3a7a5c6ad admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '758d124435054ee0a53b42100561728f', 'request_id': 'req-137eb412-a2ba-4708-b89d-6db3a7a5c6ad'} {{(pid=80144) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 10:54:58.380891 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.notifications [None req-137eb412-a2ba-4708-b89d-6db3a7a5c6ad admin admin] The token cache is being invalidate because project 758d124435054ee0a53b42100561728f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80144) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 07 10:54:58.381310 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.notifications [None req-137eb412-a2ba-4708-b89d-6db3a7a5c6ad admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80144) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 10:54:58.382021 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.notifications [None req-137eb412-a2ba-4708-b89d-6db3a7a5c6ad admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80144) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 10:54:58.383547 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 975/1921] 10.222.0.25 () {58 vars in 1233 bytes} [Sun Aug 7 10:54:58 2022] DELETE /identity/v3/projects/758d124435054ee0a53b42100561728f => generated 0 bytes in 180 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 07 10:54:58.387479 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c51b3d5-f18a-4783-acc8-0f97d0f513a4 admin admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 10:54:58.388903 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-2c51b3d5-f18a-4783-acc8-0f97d0f513a4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 10:54:58.404093 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c51b3d5-f18a-4783-acc8-0f97d0f513a4 admin admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 10:54:58.447836 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 10:54:58.480925 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-194b1e77-1462-49a0-9eb2-da2337670907 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5ac47e3b09f43d19a23a0f457fe9e23', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 10:54:58.481837 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-194b1e77-1462-49a0-9eb2-da2337670907 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:54:58.482325 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-194b1e77-1462-49a0-9eb2-da2337670907 admin admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:54:58.483413 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-194b1e77-1462-49a0-9eb2-da2337670907 admin admin] PATH_INFO: `/v3/users/7bbd634e87404804bf411c2385f0666b` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:54:58.484756 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-194b1e77-1462-49a0-9eb2-da2337670907 admin admin] RBAC: Authorizing `identity:delete_user(user_id=7bbd634e87404804bf411c2385f0666b)` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 10:54:58.486955 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-194b1e77-1462-49a0-9eb2-da2337670907 admin admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 10:54:58.561342 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-194b1e77-1462-49a0-9eb2-da2337670907 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7bbd634e87404804bf411c2385f0666b'} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 10:54:58.571555 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-194b1e77-1462-49a0-9eb2-da2337670907 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7bbd634e87404804bf411c2385f0666b'} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 10:54:58.587514 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-194b1e77-1462-49a0-9eb2-da2337670907 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7bbd634e87404804bf411c2385f0666b'} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 10:54:58.588321 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-194b1e77-1462-49a0-9eb2-da2337670907 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7bbd634e87404804bf411c2385f0666b'} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 10:54:58.596706 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-194b1e77-1462-49a0-9eb2-da2337670907 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7bbd634e87404804bf411c2385f0666b'} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 10:54:58.604466 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 947/1922] 10.222.0.25 () {58 vars in 1224 bytes} [Sun Aug 7 10:54:58 2022] DELETE /identity/v3/users/7bbd634e87404804bf411c2385f0666b => generated 0 bytes in 217 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 07 10:54:58.952576 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-137eb412-a2ba-4708-b89d-6db3a7a5c6ad admin admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 10:54:58.954316 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.fernet_utils [None req-137eb412-a2ba-4708-b89d-6db3a7a5c6ad admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80144) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 10:54:58.967110 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-137eb412-a2ba-4708-b89d-6db3a7a5c6ad admin admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 10:54:59.028799 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c0ddc06-9520-4dc1-b517-7a6176039ec3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5ac47e3b09f43d19a23a0f457fe9e23', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 10:54:59.029446 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c0ddc06-9520-4dc1-b517-7a6176039ec3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:54:59.029726 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c0ddc06-9520-4dc1-b517-7a6176039ec3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:54:59.030015 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c0ddc06-9520-4dc1-b517-7a6176039ec3 admin admin] PATH_INFO: `/v3/projects/d20fc70dfb7e4e96a6b5d6f06f7058ed` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:54:59.031156 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c0ddc06-9520-4dc1-b517-7a6176039ec3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d20fc70dfb7e4e96a6b5d6f06f7058ed)` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 10:54:59.033321 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c0ddc06-9520-4dc1-b517-7a6176039ec3 admin admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 10:54:59.080618 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.notifications [None req-5c0ddc06-9520-4dc1-b517-7a6176039ec3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd20fc70dfb7e4e96a6b5d6f06f7058ed', 'request_id': 'req-5c0ddc06-9520-4dc1-b517-7a6176039ec3'} {{(pid=80144) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 10:54:59.086225 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.notifications [None req-5c0ddc06-9520-4dc1-b517-7a6176039ec3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd20fc70dfb7e4e96a6b5d6f06f7058ed', 'request_id': 'req-5c0ddc06-9520-4dc1-b517-7a6176039ec3'} {{(pid=80144) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 10:54:59.093569 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.notifications [None req-5c0ddc06-9520-4dc1-b517-7a6176039ec3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd20fc70dfb7e4e96a6b5d6f06f7058ed', 'request_id': 'req-5c0ddc06-9520-4dc1-b517-7a6176039ec3'} {{(pid=80144) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 10:54:59.110949 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.notifications [None req-5c0ddc06-9520-4dc1-b517-7a6176039ec3 admin admin] The token cache is being invalidate because project d20fc70dfb7e4e96a6b5d6f06f7058ed was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80144) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 07 10:54:59.110949 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.notifications [None req-5c0ddc06-9520-4dc1-b517-7a6176039ec3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80144) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 10:54:59.110949 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.notifications [None req-5c0ddc06-9520-4dc1-b517-7a6176039ec3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80144) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 10:54:59.111790 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 976/1923] 10.222.0.25 () {58 vars in 1233 bytes} [Sun Aug 7 10:54:58 2022] DELETE /identity/v3/projects/d20fc70dfb7e4e96a6b5d6f06f7058ed => generated 0 bytes in 159 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 07 10:54:59.117049 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90380c49-6e3f-4406-840f-afef96f15a76 None None] REQUEST_METHOD: `POST` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:54:59.117531 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90380c49-6e3f-4406-840f-afef96f15a76 None None] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:54:59.117958 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90380c49-6e3f-4406-840f-afef96f15a76 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:54:59.148579 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.auth.core [None req-90380c49-6e3f-4406-840f-afef96f15a76 None None] MFA Rules not processed for user `bac4e6b76fe64217ac8fa44e71cffe6a`. Rule list: `[]` (Enabled: `True`). {{(pid=80143) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 10:54:59.150413 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-90380c49-6e3f-4406-840f-afef96f15a76 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 10:54:59.216174 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 948/1924] 10.222.0.25 () {58 vars in 958 bytes} [Sun Aug 7 10:54:59 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 07 10:54:59.225809 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6770864b-e790-4fc7-a723-d6f2e4d1ee1b None None] REQUEST_METHOD: `POST` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:54:59.226192 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6770864b-e790-4fc7-a723-d6f2e4d1ee1b None None] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:54:59.226480 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6770864b-e790-4fc7-a723-d6f2e4d1ee1b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:54:59.247557 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.auth.core [None req-6770864b-e790-4fc7-a723-d6f2e4d1ee1b None None] MFA Rules not processed for user `bac4e6b76fe64217ac8fa44e71cffe6a`. Rule list: `[]` (Enabled: `True`). {{(pid=80144) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 10:54:59.248453 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.fernet_utils [None req-6770864b-e790-4fc7-a723-d6f2e4d1ee1b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80144) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 10:54:59.257790 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 977/1925] 10.222.0.25 () {58 vars in 958 bytes} [Sun Aug 7 10:54:59 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 07 10:54:59.263479 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90380c49-6e3f-4406-840f-afef96f15a76 None None] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 10:54:59.277691 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90380c49-6e3f-4406-840f-afef96f15a76 None None] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 10:54:59.283130 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-742798f6-28bd-4718-9beb-c405d4764632 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5ac47e3b09f43d19a23a0f457fe9e23', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 10:54:59.283855 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-742798f6-28bd-4718-9beb-c405d4764632 admin admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:54:59.284203 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-742798f6-28bd-4718-9beb-c405d4764632 admin admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:54:59.284718 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-742798f6-28bd-4718-9beb-c405d4764632 admin admin] PATH_INFO: `/v3/domains` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:54:59.285460 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-742798f6-28bd-4718-9beb-c405d4764632 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80143) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 10:54:59.285863 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-742798f6-28bd-4718-9beb-c405d4764632 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 10:54:59.287630 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-742798f6-28bd-4718-9beb-c405d4764632 admin admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 10:54:59.302352 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 949/1926] 10.222.0.25 () {58 vars in 1153 bytes} [Sun Aug 7 10:54:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 10:54:59.306595 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6770864b-e790-4fc7-a723-d6f2e4d1ee1b None None] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 10:54:59.322776 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6770864b-e790-4fc7-a723-d6f2e4d1ee1b None None] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 10:54:59.328917 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e2a43ce-cd58-4066-ace5-cc76400ff685 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5ac47e3b09f43d19a23a0f457fe9e23', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 10:54:59.329646 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e2a43ce-cd58-4066-ace5-cc76400ff685 admin admin] REQUEST_METHOD: `POST` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:54:59.329932 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e2a43ce-cd58-4066-ace5-cc76400ff685 admin admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:54:59.330211 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e2a43ce-cd58-4066-ace5-cc76400ff685 admin admin] PATH_INFO: `/v3/projects` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:54:59.330847 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e2a43ce-cd58-4066-ace5-cc76400ff685 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 10:54:59.332846 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e2a43ce-cd58-4066-ace5-cc76400ff685 admin admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 10:54:59.362988 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 978/1927] 10.222.0.25 () {60 vars in 1153 bytes} [Sun Aug 7 10:54:59 2022] POST /identity/v3/projects => generated 371 bytes in 57 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 07 10:54:59.367164 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-742798f6-28bd-4718-9beb-c405d4764632 admin admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 10:54:59.377642 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-742798f6-28bd-4718-9beb-c405d4764632 admin admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 10:54:59.425373 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 10:54:59.440989 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a9b2a11-a08a-483c-b516-bc8ad9ad534d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5ac47e3b09f43d19a23a0f457fe9e23', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 10:54:59.441597 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a9b2a11-a08a-483c-b516-bc8ad9ad534d admin admin] REQUEST_METHOD: `POST` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:54:59.441875 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a9b2a11-a08a-483c-b516-bc8ad9ad534d admin admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:54:59.442143 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a9b2a11-a08a-483c-b516-bc8ad9ad534d admin admin] PATH_INFO: `/v3/users` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:54:59.442756 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a9b2a11-a08a-483c-b516-bc8ad9ad534d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 10:54:59.444685 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a9b2a11-a08a-483c-b516-bc8ad9ad534d admin admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 10:54:59.464531 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 950/1928] 10.222.0.25 () {60 vars in 1144 bytes} [Sun Aug 7 10:54:59 2022] POST /identity/v3/users => generated 286 bytes in 98 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 07 10:54:59.469471 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e2a43ce-cd58-4066-ace5-cc76400ff685 admin admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 10:54:59.480008 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e2a43ce-cd58-4066-ace5-cc76400ff685 admin admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 10:54:59.485334 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39014c0b-9292-4ecf-ae8a-14fd88884edd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5ac47e3b09f43d19a23a0f457fe9e23', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 10:54:59.485936 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39014c0b-9292-4ecf-ae8a-14fd88884edd admin admin] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:54:59.486215 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39014c0b-9292-4ecf-ae8a-14fd88884edd admin admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:54:59.486486 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39014c0b-9292-4ecf-ae8a-14fd88884edd admin admin] PATH_INFO: `/v3/roles` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:54:59.487167 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39014c0b-9292-4ecf-ae8a-14fd88884edd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 10:54:59.488737 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39014c0b-9292-4ecf-ae8a-14fd88884edd admin admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 10:54:59.498463 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 979/1929] 10.222.0.25 () {58 vars in 1122 bytes} [Sun Aug 7 10:54:59 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 07 10:54:59.503113 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a9b2a11-a08a-483c-b516-bc8ad9ad534d admin admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 10:54:59.513110 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a9b2a11-a08a-483c-b516-bc8ad9ad534d admin admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 10:54:59.518231 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01affdfc-62ad-4b22-a7c1-2887592f1b8d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5ac47e3b09f43d19a23a0f457fe9e23', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 10:54:59.518823 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01affdfc-62ad-4b22-a7c1-2887592f1b8d admin admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:54:59.519157 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01affdfc-62ad-4b22-a7c1-2887592f1b8d admin admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:54:59.519434 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01affdfc-62ad-4b22-a7c1-2887592f1b8d admin admin] PATH_INFO: `/v3/roles` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:54:59.520085 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01affdfc-62ad-4b22-a7c1-2887592f1b8d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 10:54:59.522845 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01affdfc-62ad-4b22-a7c1-2887592f1b8d admin admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 10:54:59.534525 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 951/1930] 10.222.0.25 () {58 vars in 1122 bytes} [Sun Aug 7 10:54:59 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 07 10:54:59.538584 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39014c0b-9292-4ecf-ae8a-14fd88884edd admin admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 10:54:59.548618 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39014c0b-9292-4ecf-ae8a-14fd88884edd admin admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 10:54:59.553475 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c87aa20-7552-4e8b-8328-26613f4b2562 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5ac47e3b09f43d19a23a0f457fe9e23', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 10:54:59.554126 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c87aa20-7552-4e8b-8328-26613f4b2562 admin admin] REQUEST_METHOD: `PUT` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:54:59.554409 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c87aa20-7552-4e8b-8328-26613f4b2562 admin admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:54:59.554684 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c87aa20-7552-4e8b-8328-26613f4b2562 admin admin] PATH_INFO: `/v3/projects/65c513d55b2b4ab6b2617ce0dc066206/users/f7fbe7410300451e8048f594d0d02880/roles/b98dfb3b9fee4644af2887a3ac3fd2e6` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:54:59.569903 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c87aa20-7552-4e8b-8328-26613f4b2562 admin admin] RBAC: Authorizing `identity:create_grant(project_id=65c513d55b2b4ab6b2617ce0dc066206, user_id=f7fbe7410300451e8048f594d0d02880, role_id=b98dfb3b9fee4644af2887a3ac3fd2e6)` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 10:54:59.572698 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c87aa20-7552-4e8b-8328-26613f4b2562 admin admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 10:54:59.591775 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 980/1931] 10.222.0.25 () {60 vars in 1483 bytes} [Sun Aug 7 10:54:59 2022] PUT /identity/v3/projects/65c513d55b2b4ab6b2617ce0dc066206/users/f7fbe7410300451e8048f594d0d02880/roles/b98dfb3b9fee4644af2887a3ac3fd2e6 => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 07 10:54:59.602983 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b1848d5-1cfd-44ca-9db4-06471e0e305f None None] REQUEST_METHOD: `POST` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:54:59.603274 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b1848d5-1cfd-44ca-9db4-06471e0e305f None None] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:54:59.603550 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b1848d5-1cfd-44ca-9db4-06471e0e305f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:54:59.631799 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.auth.core [None req-5b1848d5-1cfd-44ca-9db4-06471e0e305f None None] MFA Rules not processed for user `f7fbe7410300451e8048f594d0d02880`. Rule list: `[]` (Enabled: `True`). {{(pid=80143) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 10:54:59.633020 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-5b1848d5-1cfd-44ca-9db4-06471e0e305f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 10:54:59.682906 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c87aa20-7552-4e8b-8328-26613f4b2562 admin admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 10:54:59.684780 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.fernet_utils [None req-5c87aa20-7552-4e8b-8328-26613f4b2562 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80144) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 10:54:59.706199 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 952/1932] 10.222.0.25 () {58 vars in 958 bytes} [Sun Aug 7 10:54:59 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 07 10:54:59.708901 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 10:54:59.754007 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b1848d5-1cfd-44ca-9db4-06471e0e305f None None] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 10:54:59.762975 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-5b1848d5-1cfd-44ca-9db4-06471e0e305f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 10:54:59.763216 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c87aa20-7552-4e8b-8328-26613f4b2562 admin admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 10:54:59.764409 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8a8416e-72d9-476e-a826-a3e4483d7dd1 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88dbd6dfd1164994a3e16da553561910', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6de1f3cac02c4bcdb05dd2337176033d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 10:54:59.765206 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8a8416e-72d9-476e-a826-a3e4483d7dd1 service cinder] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:54:59.766172 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8a8416e-72d9-476e-a826-a3e4483d7dd1 service cinder] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:54:59.766648 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8a8416e-72d9-476e-a826-a3e4483d7dd1 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:54:59.769251 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.fernet_utils [None req-b8a8416e-72d9-476e-a826-a3e4483d7dd1 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80144) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 10:54:59.770797 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b1848d5-1cfd-44ca-9db4-06471e0e305f None None] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 10:54:59.774631 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68e7148e-3de2-4acf-a3c8-1d85c076925f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88dbd6dfd1164994a3e16da553561910', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6de1f3cac02c4bcdb05dd2337176033d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 10:54:59.775290 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68e7148e-3de2-4acf-a3c8-1d85c076925f service cinder] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:54:59.775575 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68e7148e-3de2-4acf-a3c8-1d85c076925f service cinder] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:54:59.775849 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68e7148e-3de2-4acf-a3c8-1d85c076925f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:54:59.786489 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68e7148e-3de2-4acf-a3c8-1d85c076925f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 10:54:59.788186 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68e7148e-3de2-4acf-a3c8-1d85c076925f service cinder] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 10:54:59.800054 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 953/1933] 10.222.0.25 () {60 vars in 1365 bytes} [Sun Aug 7 10:54:59 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 47 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 07 10:54:59.823069 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68e7148e-3de2-4acf-a3c8-1d85c076925f service cinder] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 10:54:59.844867 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8a8416e-72d9-476e-a826-a3e4483d7dd1 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 10:54:59.846491 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8a8416e-72d9-476e-a826-a3e4483d7dd1 service cinder] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 10:54:59.858360 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 981/1934] 10.222.0.25 () {60 vars in 1365 bytes} [Sun Aug 7 10:54:59 2022] GET /identity/v3/auth/tokens => generated 3329 bytes in 176 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 07 10:54:59.898624 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68e7148e-3de2-4acf-a3c8-1d85c076925f service cinder] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 10:54:59.904516 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4750fe41-aff2-4320-add9-c1d25c9f8252 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5ac47e3b09f43d19a23a0f457fe9e23', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 10:54:59.905964 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4750fe41-aff2-4320-add9-c1d25c9f8252 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:54:59.906914 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4750fe41-aff2-4320-add9-c1d25c9f8252 admin admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:54:59.907327 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4750fe41-aff2-4320-add9-c1d25c9f8252 admin admin] PATH_INFO: `/v3/users/f7fbe7410300451e8048f594d0d02880` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:54:59.908859 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4750fe41-aff2-4320-add9-c1d25c9f8252 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f7fbe7410300451e8048f594d0d02880)` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 10:54:59.911680 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4750fe41-aff2-4320-add9-c1d25c9f8252 admin admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 10:54:59.979413 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=91,keystone:INSERT=3,keystone:DELETE=6 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 10:54:59.985201 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-4750fe41-aff2-4320-add9-c1d25c9f8252 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f7fbe7410300451e8048f594d0d02880'} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 10:54:59.999113 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-4750fe41-aff2-4320-add9-c1d25c9f8252 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f7fbe7410300451e8048f594d0d02880'} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 10:55:00.013870 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-4750fe41-aff2-4320-add9-c1d25c9f8252 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f7fbe7410300451e8048f594d0d02880'} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 10:55:00.014722 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-4750fe41-aff2-4320-add9-c1d25c9f8252 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f7fbe7410300451e8048f594d0d02880'} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 10:55:00.023296 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-4750fe41-aff2-4320-add9-c1d25c9f8252 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f7fbe7410300451e8048f594d0d02880'} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 10:55:00.030147 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 954/1935] 10.222.0.25 () {58 vars in 1224 bytes} [Sun Aug 7 10:54:59 2022] DELETE /identity/v3/users/f7fbe7410300451e8048f594d0d02880 => generated 0 bytes in 208 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 07 10:55:00.034882 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76628efe-d3ee-41ae-8249-ba8e122023ce None None] REQUEST_METHOD: `POST` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:55:00.035415 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76628efe-d3ee-41ae-8249-ba8e122023ce None None] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:55:00.035513 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76628efe-d3ee-41ae-8249-ba8e122023ce None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:55:00.058600 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.auth.core [None req-76628efe-d3ee-41ae-8249-ba8e122023ce None None] MFA Rules not processed for user `bac4e6b76fe64217ac8fa44e71cffe6a`. Rule list: `[]` (Enabled: `True`). {{(pid=80144) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 10:55:00.059837 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.fernet_utils [None req-76628efe-d3ee-41ae-8249-ba8e122023ce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80144) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 10:55:00.138698 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 982/1936] 10.222.0.25 () {58 vars in 958 bytes} [Sun Aug 7 10:55:00 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 07 10:55:00.150676 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4750fe41-aff2-4320-add9-c1d25c9f8252 admin admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 10:55:00.152840 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-4750fe41-aff2-4320-add9-c1d25c9f8252 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 10:55:00.244810 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4750fe41-aff2-4320-add9-c1d25c9f8252 admin admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 10:55:00.248644 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ce833e3-cd81-4d31-b6a5-ecbd78d9df80 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be1283c9bf47424c9aa3ad27cab03027', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6de1f3cac02c4bcdb05dd2337176033d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 10:55:00.249315 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ce833e3-cd81-4d31-b6a5-ecbd78d9df80 service neutron] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:55:00.249604 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ce833e3-cd81-4d31-b6a5-ecbd78d9df80 service neutron] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:55:00.249879 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ce833e3-cd81-4d31-b6a5-ecbd78d9df80 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:55:00.260299 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ce833e3-cd81-4d31-b6a5-ecbd78d9df80 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 10:55:00.261732 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ce833e3-cd81-4d31-b6a5-ecbd78d9df80 service neutron] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 10:55:00.284706 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 955/1937] 10.222.0.25 () {60 vars in 1365 bytes} [Sun Aug 7 10:55:00 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 135 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 07 10:55:00.659546 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76628efe-d3ee-41ae-8249-ba8e122023ce None None] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 10:55:00.661020 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.fernet_utils [None req-76628efe-d3ee-41ae-8249-ba8e122023ce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80144) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 10:55:00.673546 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76628efe-d3ee-41ae-8249-ba8e122023ce None None] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 10:55:00.678183 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-781dfa39-2f1b-44b0-8780-02a50b03bf4a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5ac47e3b09f43d19a23a0f457fe9e23', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 10:55:00.678819 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-781dfa39-2f1b-44b0-8780-02a50b03bf4a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:55:00.679135 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-781dfa39-2f1b-44b0-8780-02a50b03bf4a admin admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:55:00.679418 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-781dfa39-2f1b-44b0-8780-02a50b03bf4a admin admin] PATH_INFO: `/v3/projects/65c513d55b2b4ab6b2617ce0dc066206` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:55:00.680546 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-781dfa39-2f1b-44b0-8780-02a50b03bf4a admin admin] RBAC: Authorizing `identity:delete_project(project_id=65c513d55b2b4ab6b2617ce0dc066206)` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 10:55:00.682437 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-781dfa39-2f1b-44b0-8780-02a50b03bf4a admin admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 10:55:00.730157 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.notifications [None req-781dfa39-2f1b-44b0-8780-02a50b03bf4a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '65c513d55b2b4ab6b2617ce0dc066206', 'request_id': 'req-781dfa39-2f1b-44b0-8780-02a50b03bf4a'} {{(pid=80144) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 10:55:00.736281 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.notifications [None req-781dfa39-2f1b-44b0-8780-02a50b03bf4a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '65c513d55b2b4ab6b2617ce0dc066206', 'request_id': 'req-781dfa39-2f1b-44b0-8780-02a50b03bf4a'} {{(pid=80144) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 10:55:00.739863 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 10:55:00.743215 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.notifications [None req-781dfa39-2f1b-44b0-8780-02a50b03bf4a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '65c513d55b2b4ab6b2617ce0dc066206', 'request_id': 'req-781dfa39-2f1b-44b0-8780-02a50b03bf4a'} {{(pid=80144) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 10:55:00.755638 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.notifications [None req-781dfa39-2f1b-44b0-8780-02a50b03bf4a admin admin] The token cache is being invalidate because project 65c513d55b2b4ab6b2617ce0dc066206 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80144) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 07 10:55:00.755923 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.notifications [None req-781dfa39-2f1b-44b0-8780-02a50b03bf4a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80144) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 10:55:00.756545 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.notifications [None req-781dfa39-2f1b-44b0-8780-02a50b03bf4a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80144) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 10:55:00.758019 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 983/1938] 10.222.0.25 () {58 vars in 1233 bytes} [Sun Aug 7 10:55:00 2022] DELETE /identity/v3/projects/65c513d55b2b4ab6b2617ce0dc066206 => generated 0 bytes in 99 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 07 10:55:00.762831 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5819e924-0d23-4667-a018-385603511a97 None None] REQUEST_METHOD: `POST` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:55:00.763215 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5819e924-0d23-4667-a018-385603511a97 None None] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:55:00.763497 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5819e924-0d23-4667-a018-385603511a97 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:55:00.785511 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.auth.core [None req-5819e924-0d23-4667-a018-385603511a97 None None] MFA Rules not processed for user `bac4e6b76fe64217ac8fa44e71cffe6a`. Rule list: `[]` (Enabled: `True`). {{(pid=80143) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 10:55:00.786749 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-5819e924-0d23-4667-a018-385603511a97 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 10:55:00.849371 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 956/1939] 10.222.0.25 () {58 vars in 958 bytes} [Sun Aug 7 10:55:00 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 07 10:55:00.856827 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cf6ae1f-c6b9-4a35-89f2-b2cddb31a84f None None] REQUEST_METHOD: `POST` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:55:00.857117 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cf6ae1f-c6b9-4a35-89f2-b2cddb31a84f None None] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:55:00.857397 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cf6ae1f-c6b9-4a35-89f2-b2cddb31a84f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:55:00.878466 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.auth.core [None req-7cf6ae1f-c6b9-4a35-89f2-b2cddb31a84f None None] MFA Rules not processed for user `bac4e6b76fe64217ac8fa44e71cffe6a`. Rule list: `[]` (Enabled: `True`). {{(pid=80144) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 10:55:00.879699 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.fernet_utils [None req-7cf6ae1f-c6b9-4a35-89f2-b2cddb31a84f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80144) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 10:55:00.887553 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 984/1940] 10.222.0.25 () {58 vars in 958 bytes} [Sun Aug 7 10:55:00 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 07 10:55:00.892317 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5819e924-0d23-4667-a018-385603511a97 None None] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 10:55:00.902154 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5819e924-0d23-4667-a018-385603511a97 None None] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 10:55:00.907366 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93feab68-2487-4e6d-9583-b5f51c687694 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5ac47e3b09f43d19a23a0f457fe9e23', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 10:55:00.907976 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93feab68-2487-4e6d-9583-b5f51c687694 admin admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:55:00.908257 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93feab68-2487-4e6d-9583-b5f51c687694 admin admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:55:00.908536 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93feab68-2487-4e6d-9583-b5f51c687694 admin admin] PATH_INFO: `/v3/domains` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:55:00.909186 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93feab68-2487-4e6d-9583-b5f51c687694 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80143) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 10:55:00.909491 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93feab68-2487-4e6d-9583-b5f51c687694 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 10:55:00.911117 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93feab68-2487-4e6d-9583-b5f51c687694 admin admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 10:55:00.922584 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 957/1941] 10.222.0.25 () {58 vars in 1153 bytes} [Sun Aug 7 10:55:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 07 10:55:00.926792 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cf6ae1f-c6b9-4a35-89f2-b2cddb31a84f None None] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 10:55:00.936341 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cf6ae1f-c6b9-4a35-89f2-b2cddb31a84f None None] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 10:55:00.941756 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c39826e-77fe-4d12-9d6b-b1cb911f415f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5ac47e3b09f43d19a23a0f457fe9e23', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 10:55:00.942361 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c39826e-77fe-4d12-9d6b-b1cb911f415f admin admin] REQUEST_METHOD: `POST` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:55:00.942639 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c39826e-77fe-4d12-9d6b-b1cb911f415f admin admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:55:00.942909 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c39826e-77fe-4d12-9d6b-b1cb911f415f admin admin] PATH_INFO: `/v3/projects` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:55:00.943563 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c39826e-77fe-4d12-9d6b-b1cb911f415f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 10:55:00.945553 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c39826e-77fe-4d12-9d6b-b1cb911f415f admin admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 10:55:00.977585 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 985/1942] 10.222.0.25 () {60 vars in 1153 bytes} [Sun Aug 7 10:55:00 2022] POST /identity/v3/projects => generated 361 bytes in 48 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 07 10:55:00.978639 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93feab68-2487-4e6d-9583-b5f51c687694 admin admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 10:55:00.989156 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93feab68-2487-4e6d-9583-b5f51c687694 admin admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 10:55:01.034377 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=95,keystone:DELETE=4,keystone:INSERT=1 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 10:55:01.049243 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9aa5efcc-3419-47fe-86c9-2804243412de admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5ac47e3b09f43d19a23a0f457fe9e23', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 10:55:01.049865 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aa5efcc-3419-47fe-86c9-2804243412de admin admin] REQUEST_METHOD: `POST` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:55:01.050150 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aa5efcc-3419-47fe-86c9-2804243412de admin admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:55:01.050426 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aa5efcc-3419-47fe-86c9-2804243412de admin admin] PATH_INFO: `/v3/users` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:55:01.051134 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9aa5efcc-3419-47fe-86c9-2804243412de admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 10:55:01.053076 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9aa5efcc-3419-47fe-86c9-2804243412de admin admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 10:55:01.078311 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 958/1943] 10.222.0.25 () {60 vars in 1144 bytes} [Sun Aug 7 10:55:00 2022] POST /identity/v3/users => generated 281 bytes in 97 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 07 10:55:01.079523 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c39826e-77fe-4d12-9d6b-b1cb911f415f admin admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 10:55:01.089966 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c39826e-77fe-4d12-9d6b-b1cb911f415f admin admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 10:55:01.095202 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-673e43ae-87b4-4e80-8fa0-47debfedeb15 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5ac47e3b09f43d19a23a0f457fe9e23', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 10:55:01.095815 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-673e43ae-87b4-4e80-8fa0-47debfedeb15 admin admin] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:55:01.096095 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-673e43ae-87b4-4e80-8fa0-47debfedeb15 admin admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:55:01.096372 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-673e43ae-87b4-4e80-8fa0-47debfedeb15 admin admin] PATH_INFO: `/v3/roles` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:55:01.097021 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-673e43ae-87b4-4e80-8fa0-47debfedeb15 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 10:55:01.098596 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-673e43ae-87b4-4e80-8fa0-47debfedeb15 admin admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 10:55:01.109280 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 986/1944] 10.222.0.25 () {58 vars in 1122 bytes} [Sun Aug 7 10:55:01 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 07 10:55:01.113916 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9aa5efcc-3419-47fe-86c9-2804243412de admin admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 10:55:01.124052 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9aa5efcc-3419-47fe-86c9-2804243412de admin admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 10:55:01.129104 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cd044e1-7784-45aa-86c1-2e6f963db017 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5ac47e3b09f43d19a23a0f457fe9e23', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 10:55:01.129698 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cd044e1-7784-45aa-86c1-2e6f963db017 admin admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:55:01.129980 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cd044e1-7784-45aa-86c1-2e6f963db017 admin admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:55:01.130255 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cd044e1-7784-45aa-86c1-2e6f963db017 admin admin] PATH_INFO: `/v3/roles` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:55:01.130917 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cd044e1-7784-45aa-86c1-2e6f963db017 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 10:55:01.132558 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cd044e1-7784-45aa-86c1-2e6f963db017 admin admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 10:55:01.143452 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 959/1945] 10.222.0.25 () {58 vars in 1122 bytes} [Sun Aug 7 10:55:01 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 07 10:55:01.147460 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-673e43ae-87b4-4e80-8fa0-47debfedeb15 admin admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 10:55:01.158371 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-673e43ae-87b4-4e80-8fa0-47debfedeb15 admin admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 10:55:01.163055 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63c79041-a77f-49d6-9f18-c15e8ce11ee7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5ac47e3b09f43d19a23a0f457fe9e23', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 10:55:01.163744 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63c79041-a77f-49d6-9f18-c15e8ce11ee7 admin admin] REQUEST_METHOD: `PUT` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:55:01.164029 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63c79041-a77f-49d6-9f18-c15e8ce11ee7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:55:01.164306 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63c79041-a77f-49d6-9f18-c15e8ce11ee7 admin admin] PATH_INFO: `/v3/projects/444cefaab01b466291727865e92ca9e0/users/660fc113fc054abb8a44fa355e9328f5/roles/b98dfb3b9fee4644af2887a3ac3fd2e6` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:55:01.179270 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63c79041-a77f-49d6-9f18-c15e8ce11ee7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=444cefaab01b466291727865e92ca9e0, user_id=660fc113fc054abb8a44fa355e9328f5, role_id=b98dfb3b9fee4644af2887a3ac3fd2e6)` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 10:55:01.181900 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63c79041-a77f-49d6-9f18-c15e8ce11ee7 admin admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 10:55:01.194781 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 987/1946] 10.222.0.25 () {60 vars in 1483 bytes} [Sun Aug 7 10:55:01 2022] PUT /identity/v3/projects/444cefaab01b466291727865e92ca9e0/users/660fc113fc054abb8a44fa355e9328f5/roles/b98dfb3b9fee4644af2887a3ac3fd2e6 => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 07 10:55:01.206154 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dee60af-a98c-48e4-be71-55fdc1a3cef7 None None] REQUEST_METHOD: `POST` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:55:01.206649 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dee60af-a98c-48e4-be71-55fdc1a3cef7 None None] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:55:01.206758 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dee60af-a98c-48e4-be71-55fdc1a3cef7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:55:01.233770 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.auth.core [None req-6dee60af-a98c-48e4-be71-55fdc1a3cef7 None None] MFA Rules not processed for user `660fc113fc054abb8a44fa355e9328f5`. Rule list: `[]` (Enabled: `True`). {{(pid=80143) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 10:55:01.235192 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-6dee60af-a98c-48e4-be71-55fdc1a3cef7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 10:55:01.299529 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 960/1947] 10.222.0.25 () {58 vars in 958 bytes} [Sun Aug 7 10:55:01 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 07 10:55:01.335061 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63c79041-a77f-49d6-9f18-c15e8ce11ee7 admin admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 10:55:01.335866 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.fernet_utils [None req-63c79041-a77f-49d6-9f18-c15e8ce11ee7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80144) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 10:55:01.403975 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63c79041-a77f-49d6-9f18-c15e8ce11ee7 admin admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 10:55:01.408424 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f1aefac-16b8-4d1b-ad91-e9b4302a17ee service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88dbd6dfd1164994a3e16da553561910', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6de1f3cac02c4bcdb05dd2337176033d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 10:55:01.409052 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f1aefac-16b8-4d1b-ad91-e9b4302a17ee service cinder] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:55:01.409331 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f1aefac-16b8-4d1b-ad91-e9b4302a17ee service cinder] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:55:01.409604 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f1aefac-16b8-4d1b-ad91-e9b4302a17ee service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:55:01.419324 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f1aefac-16b8-4d1b-ad91-e9b4302a17ee service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 10:55:01.420835 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f1aefac-16b8-4d1b-ad91-e9b4302a17ee service cinder] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 10:55:01.430273 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 988/1948] 10.222.0.25 () {60 vars in 1365 bytes} [Sun Aug 7 10:55:01 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 07 10:55:01.445484 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dee60af-a98c-48e4-be71-55fdc1a3cef7 None None] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 10:55:01.524015 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dee60af-a98c-48e4-be71-55fdc1a3cef7 None None] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 10:55:01.528414 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa9dee45-c6ba-4d2b-a6e0-e29f7f38b6b4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5ac47e3b09f43d19a23a0f457fe9e23', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 10:55:01.529042 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa9dee45-c6ba-4d2b-a6e0-e29f7f38b6b4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:55:01.529320 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa9dee45-c6ba-4d2b-a6e0-e29f7f38b6b4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:55:01.529591 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa9dee45-c6ba-4d2b-a6e0-e29f7f38b6b4 admin admin] PATH_INFO: `/v3/users/660fc113fc054abb8a44fa355e9328f5` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:55:01.530741 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa9dee45-c6ba-4d2b-a6e0-e29f7f38b6b4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=660fc113fc054abb8a44fa355e9328f5)` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 10:55:01.533569 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa9dee45-c6ba-4d2b-a6e0-e29f7f38b6b4 admin admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 10:55:01.584115 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-aa9dee45-c6ba-4d2b-a6e0-e29f7f38b6b4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '660fc113fc054abb8a44fa355e9328f5'} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 10:55:01.591177 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-aa9dee45-c6ba-4d2b-a6e0-e29f7f38b6b4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '660fc113fc054abb8a44fa355e9328f5'} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 10:55:01.593082 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=90,keystone:INSERT=3,keystone:DELETE=7 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 10:55:01.606189 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-aa9dee45-c6ba-4d2b-a6e0-e29f7f38b6b4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '660fc113fc054abb8a44fa355e9328f5'} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 10:55:01.607354 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-aa9dee45-c6ba-4d2b-a6e0-e29f7f38b6b4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '660fc113fc054abb8a44fa355e9328f5'} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 10:55:01.617130 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-aa9dee45-c6ba-4d2b-a6e0-e29f7f38b6b4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '660fc113fc054abb8a44fa355e9328f5'} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 10:55:01.624369 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 961/1949] 10.222.0.25 () {58 vars in 1224 bytes} [Sun Aug 7 10:55:01 2022] DELETE /identity/v3/users/660fc113fc054abb8a44fa355e9328f5 => generated 0 bytes in 179 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 07 10:55:01.628844 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c65daea-dc75-4d89-aa0f-298c26fc0847 None None] REQUEST_METHOD: `POST` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:55:01.629138 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c65daea-dc75-4d89-aa0f-298c26fc0847 None None] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:55:01.629419 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c65daea-dc75-4d89-aa0f-298c26fc0847 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:55:01.661212 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.auth.core [None req-2c65daea-dc75-4d89-aa0f-298c26fc0847 None None] MFA Rules not processed for user `bac4e6b76fe64217ac8fa44e71cffe6a`. Rule list: `[]` (Enabled: `True`). {{(pid=80144) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 10:55:01.662361 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.fernet_utils [None req-2c65daea-dc75-4d89-aa0f-298c26fc0847 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80144) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 10:55:01.734937 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 989/1950] 10.222.0.25 () {58 vars in 958 bytes} [Sun Aug 7 10:55:01 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 07 10:55:01.747100 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa9dee45-c6ba-4d2b-a6e0-e29f7f38b6b4 admin admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 10:55:01.748834 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-aa9dee45-c6ba-4d2b-a6e0-e29f7f38b6b4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 10:55:01.821148 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa9dee45-c6ba-4d2b-a6e0-e29f7f38b6b4 admin admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 10:55:01.824743 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ddf6e38-d127-4db7-b487-7cd5b8627c19 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be1283c9bf47424c9aa3ad27cab03027', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6de1f3cac02c4bcdb05dd2337176033d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 10:55:01.825355 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ddf6e38-d127-4db7-b487-7cd5b8627c19 service neutron] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:55:01.825631 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ddf6e38-d127-4db7-b487-7cd5b8627c19 service neutron] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:55:01.825942 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ddf6e38-d127-4db7-b487-7cd5b8627c19 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:55:01.837911 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ddf6e38-d127-4db7-b487-7cd5b8627c19 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 10:55:01.839396 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ddf6e38-d127-4db7-b487-7cd5b8627c19 service neutron] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 10:55:01.854653 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 962/1951] 10.222.0.25 () {60 vars in 1365 bytes} [Sun Aug 7 10:55:01 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 108 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 07 10:55:02.144926 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c65daea-dc75-4d89-aa0f-298c26fc0847 None None] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 10:55:02.146764 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.fernet_utils [None req-2c65daea-dc75-4d89-aa0f-298c26fc0847 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80144) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 10:55:02.171027 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c65daea-dc75-4d89-aa0f-298c26fc0847 None None] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 10:55:02.178915 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf3d76bc-d3af-48cf-abec-b5f9ca5ddb31 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5ac47e3b09f43d19a23a0f457fe9e23', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 10:55:02.179872 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf3d76bc-d3af-48cf-abec-b5f9ca5ddb31 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:55:02.180422 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf3d76bc-d3af-48cf-abec-b5f9ca5ddb31 admin admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:55:02.180831 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf3d76bc-d3af-48cf-abec-b5f9ca5ddb31 admin admin] PATH_INFO: `/v3/projects/444cefaab01b466291727865e92ca9e0` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:55:02.182446 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf3d76bc-d3af-48cf-abec-b5f9ca5ddb31 admin admin] RBAC: Authorizing `identity:delete_project(project_id=444cefaab01b466291727865e92ca9e0)` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 10:55:02.184766 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf3d76bc-d3af-48cf-abec-b5f9ca5ddb31 admin admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 10:55:02.225781 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=97,keystone:INSERT=2,keystone:DELETE=1 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 10:55:02.246156 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.notifications [None req-bf3d76bc-d3af-48cf-abec-b5f9ca5ddb31 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '444cefaab01b466291727865e92ca9e0', 'request_id': 'req-bf3d76bc-d3af-48cf-abec-b5f9ca5ddb31'} {{(pid=80144) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 10:55:02.251996 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.notifications [None req-bf3d76bc-d3af-48cf-abec-b5f9ca5ddb31 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '444cefaab01b466291727865e92ca9e0', 'request_id': 'req-bf3d76bc-d3af-48cf-abec-b5f9ca5ddb31'} {{(pid=80144) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 10:55:02.260558 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.notifications [None req-bf3d76bc-d3af-48cf-abec-b5f9ca5ddb31 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '444cefaab01b466291727865e92ca9e0', 'request_id': 'req-bf3d76bc-d3af-48cf-abec-b5f9ca5ddb31'} {{(pid=80144) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 10:55:02.272396 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.notifications [None req-bf3d76bc-d3af-48cf-abec-b5f9ca5ddb31 admin admin] The token cache is being invalidate because project 444cefaab01b466291727865e92ca9e0 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80144) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 07 10:55:02.272689 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.notifications [None req-bf3d76bc-d3af-48cf-abec-b5f9ca5ddb31 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80144) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 10:55:02.273322 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.notifications [None req-bf3d76bc-d3af-48cf-abec-b5f9ca5ddb31 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80144) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 10:55:02.274982 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 990/1952] 10.222.0.25 () {58 vars in 1233 bytes} [Sun Aug 7 10:55:02 2022] DELETE /identity/v3/projects/444cefaab01b466291727865e92ca9e0 => generated 0 bytes in 131 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 07 10:55:02.279600 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1421bc99-8d53-478d-b7b6-cf3634aed8cb None None] REQUEST_METHOD: `POST` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:55:02.280088 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1421bc99-8d53-478d-b7b6-cf3634aed8cb None None] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:55:02.280178 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1421bc99-8d53-478d-b7b6-cf3634aed8cb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:55:02.300628 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.auth.core [None req-1421bc99-8d53-478d-b7b6-cf3634aed8cb None None] MFA Rules not processed for user `bac4e6b76fe64217ac8fa44e71cffe6a`. Rule list: `[]` (Enabled: `True`). {{(pid=80143) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 10:55:02.301834 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-1421bc99-8d53-478d-b7b6-cf3634aed8cb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 10:55:02.361557 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 963/1953] 10.222.0.25 () {58 vars in 958 bytes} [Sun Aug 7 10:55:02 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 07 10:55:02.369184 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96bbb526-e907-411c-af87-337e34d0366b None None] REQUEST_METHOD: `POST` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:55:02.369542 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96bbb526-e907-411c-af87-337e34d0366b None None] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:55:02.369823 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96bbb526-e907-411c-af87-337e34d0366b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:55:02.389833 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.auth.core [None req-96bbb526-e907-411c-af87-337e34d0366b None None] MFA Rules not processed for user `bac4e6b76fe64217ac8fa44e71cffe6a`. Rule list: `[]` (Enabled: `True`). {{(pid=80144) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 10:55:02.391827 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.fernet_utils [None req-96bbb526-e907-411c-af87-337e34d0366b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80144) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 10:55:02.400168 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 991/1954] 10.222.0.25 () {58 vars in 958 bytes} [Sun Aug 7 10:55:02 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 07 10:55:02.404482 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1421bc99-8d53-478d-b7b6-cf3634aed8cb None None] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 10:55:02.414518 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1421bc99-8d53-478d-b7b6-cf3634aed8cb None None] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 10:55:02.419633 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54da1d31-8350-4155-a7ae-0a1fec4518a9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5ac47e3b09f43d19a23a0f457fe9e23', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 10:55:02.420245 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54da1d31-8350-4155-a7ae-0a1fec4518a9 admin admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:55:02.420522 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54da1d31-8350-4155-a7ae-0a1fec4518a9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:55:02.420798 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54da1d31-8350-4155-a7ae-0a1fec4518a9 admin admin] PATH_INFO: `/v3/domains` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:55:02.421501 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54da1d31-8350-4155-a7ae-0a1fec4518a9 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80143) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 10:55:02.421801 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54da1d31-8350-4155-a7ae-0a1fec4518a9 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 10:55:02.423597 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54da1d31-8350-4155-a7ae-0a1fec4518a9 admin admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 10:55:02.435561 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 964/1955] 10.222.0.25 () {58 vars in 1153 bytes} [Sun Aug 7 10:55:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 10:55:02.440340 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96bbb526-e907-411c-af87-337e34d0366b None None] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 10:55:02.449966 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96bbb526-e907-411c-af87-337e34d0366b None None] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 10:55:02.455339 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-994febee-7fe6-40e8-8edb-b8a5f98092f8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5ac47e3b09f43d19a23a0f457fe9e23', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 10:55:02.455947 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-994febee-7fe6-40e8-8edb-b8a5f98092f8 admin admin] REQUEST_METHOD: `POST` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:55:02.456234 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-994febee-7fe6-40e8-8edb-b8a5f98092f8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:55:02.456517 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-994febee-7fe6-40e8-8edb-b8a5f98092f8 admin admin] PATH_INFO: `/v3/projects` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:55:02.457147 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-994febee-7fe6-40e8-8edb-b8a5f98092f8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 10:55:02.459051 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-994febee-7fe6-40e8-8edb-b8a5f98092f8 admin admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 10:55:02.490994 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 992/1956] 10.222.0.25 () {60 vars in 1153 bytes} [Sun Aug 7 10:55:02 2022] POST /identity/v3/projects => generated 371 bytes in 50 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 07 10:55:02.494310 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54da1d31-8350-4155-a7ae-0a1fec4518a9 admin admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 10:55:02.507103 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54da1d31-8350-4155-a7ae-0a1fec4518a9 admin admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 10:55:02.567895 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f65c63e4-1954-4e2e-8112-32b3bb0a65ac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5ac47e3b09f43d19a23a0f457fe9e23', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 10:55:02.568629 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f65c63e4-1954-4e2e-8112-32b3bb0a65ac admin admin] REQUEST_METHOD: `POST` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:55:02.568950 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f65c63e4-1954-4e2e-8112-32b3bb0a65ac admin admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:55:02.569314 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f65c63e4-1954-4e2e-8112-32b3bb0a65ac admin admin] PATH_INFO: `/v3/users` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:55:02.570060 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f65c63e4-1954-4e2e-8112-32b3bb0a65ac admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 10:55:02.572063 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f65c63e4-1954-4e2e-8112-32b3bb0a65ac admin admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 10:55:02.583770 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=93,keystone:DELETE=3,keystone:INSERT=4 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 10:55:02.598551 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 965/1957] 10.222.0.25 () {60 vars in 1144 bytes} [Sun Aug 7 10:55:02 2022] POST /identity/v3/users => generated 286 bytes in 103 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 07 10:55:02.600996 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-994febee-7fe6-40e8-8edb-b8a5f98092f8 admin admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 10:55:02.612008 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-994febee-7fe6-40e8-8edb-b8a5f98092f8 admin admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 10:55:02.616530 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d40d7207-56e7-487b-9b81-10151fe3d0ba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5ac47e3b09f43d19a23a0f457fe9e23', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 10:55:02.617128 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d40d7207-56e7-487b-9b81-10151fe3d0ba admin admin] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:55:02.617408 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d40d7207-56e7-487b-9b81-10151fe3d0ba admin admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:55:02.617680 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d40d7207-56e7-487b-9b81-10151fe3d0ba admin admin] PATH_INFO: `/v3/roles` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:55:02.618312 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d40d7207-56e7-487b-9b81-10151fe3d0ba admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 10:55:02.619910 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d40d7207-56e7-487b-9b81-10151fe3d0ba admin admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 10:55:02.630185 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 993/1958] 10.222.0.25 () {58 vars in 1122 bytes} [Sun Aug 7 10:55:02 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 07 10:55:02.634212 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f65c63e4-1954-4e2e-8112-32b3bb0a65ac admin admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 10:55:02.644829 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f65c63e4-1954-4e2e-8112-32b3bb0a65ac admin admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 10:55:02.649523 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ebd4bc9-edf8-4ce4-9c36-7d9a7579b54d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5ac47e3b09f43d19a23a0f457fe9e23', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 10:55:02.650115 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ebd4bc9-edf8-4ce4-9c36-7d9a7579b54d admin admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:55:02.650391 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ebd4bc9-edf8-4ce4-9c36-7d9a7579b54d admin admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:55:02.650663 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ebd4bc9-edf8-4ce4-9c36-7d9a7579b54d admin admin] PATH_INFO: `/v3/roles` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:55:02.651383 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ebd4bc9-edf8-4ce4-9c36-7d9a7579b54d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 10:55:02.652949 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ebd4bc9-edf8-4ce4-9c36-7d9a7579b54d admin admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 10:55:02.664571 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 966/1959] 10.222.0.25 () {58 vars in 1122 bytes} [Sun Aug 7 10:55:02 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 07 10:55:02.668634 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d40d7207-56e7-487b-9b81-10151fe3d0ba admin admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 10:55:02.679139 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d40d7207-56e7-487b-9b81-10151fe3d0ba admin admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 10:55:02.683655 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-899d1ca2-74cd-4e28-857b-b74b9f27d889 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5ac47e3b09f43d19a23a0f457fe9e23', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 10:55:02.684300 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-899d1ca2-74cd-4e28-857b-b74b9f27d889 admin admin] REQUEST_METHOD: `PUT` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:55:02.684606 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-899d1ca2-74cd-4e28-857b-b74b9f27d889 admin admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:55:02.684880 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-899d1ca2-74cd-4e28-857b-b74b9f27d889 admin admin] PATH_INFO: `/v3/projects/03e03e5f20c74c569a705e44db0f83ae/users/8c957864d10d4579834ff7ac2dd75ca1/roles/b98dfb3b9fee4644af2887a3ac3fd2e6` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:55:02.699482 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-899d1ca2-74cd-4e28-857b-b74b9f27d889 admin admin] RBAC: Authorizing `identity:create_grant(project_id=03e03e5f20c74c569a705e44db0f83ae, user_id=8c957864d10d4579834ff7ac2dd75ca1, role_id=b98dfb3b9fee4644af2887a3ac3fd2e6)` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 10:55:02.702099 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-899d1ca2-74cd-4e28-857b-b74b9f27d889 admin admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 10:55:02.714851 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 994/1960] 10.222.0.25 () {60 vars in 1483 bytes} [Sun Aug 7 10:55:02 2022] PUT /identity/v3/projects/03e03e5f20c74c569a705e44db0f83ae/users/8c957864d10d4579834ff7ac2dd75ca1/roles/b98dfb3b9fee4644af2887a3ac3fd2e6 => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 07 10:55:02.726137 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a66e8cb-f7e1-46c1-9c61-0b12a0ededca None None] REQUEST_METHOD: `POST` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:55:02.726423 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a66e8cb-f7e1-46c1-9c61-0b12a0ededca None None] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:55:02.726701 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a66e8cb-f7e1-46c1-9c61-0b12a0ededca None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:55:02.755108 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.auth.core [None req-6a66e8cb-f7e1-46c1-9c61-0b12a0ededca None None] MFA Rules not processed for user `8c957864d10d4579834ff7ac2dd75ca1`. Rule list: `[]` (Enabled: `True`). {{(pid=80143) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 10:55:02.756296 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-6a66e8cb-f7e1-46c1-9c61-0b12a0ededca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 10:55:02.818570 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 967/1961] 10.222.0.25 () {58 vars in 958 bytes} [Sun Aug 7 10:55:02 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 07 10:55:02.829732 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-899d1ca2-74cd-4e28-857b-b74b9f27d889 admin admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 10:55:02.831483 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.fernet_utils [None req-899d1ca2-74cd-4e28-857b-b74b9f27d889 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80144) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 10:55:02.893318 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-899d1ca2-74cd-4e28-857b-b74b9f27d889 admin admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 10:55:02.899740 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39207c93-b04f-4d25-a30b-3a017f6b9a99 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88dbd6dfd1164994a3e16da553561910', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6de1f3cac02c4bcdb05dd2337176033d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 10:55:02.900719 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39207c93-b04f-4d25-a30b-3a017f6b9a99 service cinder] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:55:02.901107 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39207c93-b04f-4d25-a30b-3a017f6b9a99 service cinder] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:55:02.901488 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39207c93-b04f-4d25-a30b-3a017f6b9a99 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:55:02.914156 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39207c93-b04f-4d25-a30b-3a017f6b9a99 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 10:55:02.917762 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39207c93-b04f-4d25-a30b-3a017f6b9a99 service cinder] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 10:55:02.929920 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 995/1962] 10.222.0.25 () {60 vars in 1365 bytes} [Sun Aug 7 10:55:02 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 100 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 07 10:55:02.960675 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a66e8cb-f7e1-46c1-9c61-0b12a0ededca None None] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 10:55:03.042513 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a66e8cb-f7e1-46c1-9c61-0b12a0ededca None None] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 10:55:03.050279 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a4ae22e-0e51-4a56-8ed8-6e076224340a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5ac47e3b09f43d19a23a0f457fe9e23', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 10:55:03.051078 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a4ae22e-0e51-4a56-8ed8-6e076224340a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:55:03.051456 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a4ae22e-0e51-4a56-8ed8-6e076224340a admin admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:55:03.051821 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a4ae22e-0e51-4a56-8ed8-6e076224340a admin admin] PATH_INFO: `/v3/users/8c957864d10d4579834ff7ac2dd75ca1` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:55:03.053219 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a4ae22e-0e51-4a56-8ed8-6e076224340a admin admin] RBAC: Authorizing `identity:delete_user(user_id=8c957864d10d4579834ff7ac2dd75ca1)` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 10:55:03.055522 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a4ae22e-0e51-4a56-8ed8-6e076224340a admin admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 10:55:03.116374 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-6a4ae22e-0e51-4a56-8ed8-6e076224340a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8c957864d10d4579834ff7ac2dd75ca1'} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 10:55:03.125818 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-6a4ae22e-0e51-4a56-8ed8-6e076224340a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8c957864d10d4579834ff7ac2dd75ca1'} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 10:55:03.141702 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-6a4ae22e-0e51-4a56-8ed8-6e076224340a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8c957864d10d4579834ff7ac2dd75ca1'} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 10:55:03.142454 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-6a4ae22e-0e51-4a56-8ed8-6e076224340a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8c957864d10d4579834ff7ac2dd75ca1'} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 10:55:03.146528 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 10:55:03.152123 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-6a4ae22e-0e51-4a56-8ed8-6e076224340a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8c957864d10d4579834ff7ac2dd75ca1'} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 10:55:03.163068 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 968/1963] 10.222.0.25 () {58 vars in 1224 bytes} [Sun Aug 7 10:55:02 2022] DELETE /identity/v3/users/8c957864d10d4579834ff7ac2dd75ca1 => generated 0 bytes in 203 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 07 10:55:03.167546 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b27fd39-8fbb-418d-821c-b903647877b5 None None] REQUEST_METHOD: `POST` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:55:03.168530 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b27fd39-8fbb-418d-821c-b903647877b5 None None] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:55:03.168814 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b27fd39-8fbb-418d-821c-b903647877b5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:55:03.190638 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.auth.core [None req-1b27fd39-8fbb-418d-821c-b903647877b5 None None] MFA Rules not processed for user `bac4e6b76fe64217ac8fa44e71cffe6a`. Rule list: `[]` (Enabled: `True`). {{(pid=80144) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 10:55:03.192086 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.fernet_utils [None req-1b27fd39-8fbb-418d-821c-b903647877b5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80144) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 10:55:03.247920 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:DELETE=6,keystone:SELECT=91,keystone:INSERT=3 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 10:55:03.267735 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 996/1964] 10.222.0.25 () {58 vars in 958 bytes} [Sun Aug 7 10:55:03 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 07 10:55:03.280050 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a4ae22e-0e51-4a56-8ed8-6e076224340a admin admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 10:55:03.282272 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-6a4ae22e-0e51-4a56-8ed8-6e076224340a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 10:55:03.356047 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a4ae22e-0e51-4a56-8ed8-6e076224340a admin admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 10:55:03.360668 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd2a3849-34b5-4fc2-95b1-c85c68661187 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be1283c9bf47424c9aa3ad27cab03027', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6de1f3cac02c4bcdb05dd2337176033d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 10:55:03.361276 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd2a3849-34b5-4fc2-95b1-c85c68661187 service neutron] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:55:03.361558 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd2a3849-34b5-4fc2-95b1-c85c68661187 service neutron] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:55:03.361833 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd2a3849-34b5-4fc2-95b1-c85c68661187 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:55:03.371119 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd2a3849-34b5-4fc2-95b1-c85c68661187 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 10:55:03.372539 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd2a3849-34b5-4fc2-95b1-c85c68661187 service neutron] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 10:55:03.382560 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 969/1965] 10.222.0.25 () {60 vars in 1365 bytes} [Sun Aug 7 10:55:03 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 103 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 07 10:55:03.662028 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b27fd39-8fbb-418d-821c-b903647877b5 None None] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 10:55:03.663566 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.fernet_utils [None req-1b27fd39-8fbb-418d-821c-b903647877b5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80144) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 10:55:03.675646 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b27fd39-8fbb-418d-821c-b903647877b5 None None] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 10:55:03.687461 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b738e33e-60ce-4174-b3ed-4b68fcf18830 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5ac47e3b09f43d19a23a0f457fe9e23', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 10:55:03.688156 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b738e33e-60ce-4174-b3ed-4b68fcf18830 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:55:03.688437 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b738e33e-60ce-4174-b3ed-4b68fcf18830 admin admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:55:03.688715 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b738e33e-60ce-4174-b3ed-4b68fcf18830 admin admin] PATH_INFO: `/v3/projects/03e03e5f20c74c569a705e44db0f83ae` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:55:03.689940 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b738e33e-60ce-4174-b3ed-4b68fcf18830 admin admin] RBAC: Authorizing `identity:delete_project(project_id=03e03e5f20c74c569a705e44db0f83ae)` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 10:55:03.692174 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b738e33e-60ce-4174-b3ed-4b68fcf18830 admin admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 10:55:03.746796 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.notifications [None req-b738e33e-60ce-4174-b3ed-4b68fcf18830 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '03e03e5f20c74c569a705e44db0f83ae', 'request_id': 'req-b738e33e-60ce-4174-b3ed-4b68fcf18830'} {{(pid=80144) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 10:55:03.756370 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.notifications [None req-b738e33e-60ce-4174-b3ed-4b68fcf18830 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '03e03e5f20c74c569a705e44db0f83ae', 'request_id': 'req-b738e33e-60ce-4174-b3ed-4b68fcf18830'} {{(pid=80144) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 10:55:03.769046 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.notifications [None req-b738e33e-60ce-4174-b3ed-4b68fcf18830 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '03e03e5f20c74c569a705e44db0f83ae', 'request_id': 'req-b738e33e-60ce-4174-b3ed-4b68fcf18830'} {{(pid=80144) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 10:55:03.784640 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.notifications [None req-b738e33e-60ce-4174-b3ed-4b68fcf18830 admin admin] The token cache is being invalidate because project 03e03e5f20c74c569a705e44db0f83ae was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80144) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 07 10:55:03.785048 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.notifications [None req-b738e33e-60ce-4174-b3ed-4b68fcf18830 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80144) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 10:55:03.785794 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.notifications [None req-b738e33e-60ce-4174-b3ed-4b68fcf18830 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80144) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 10:55:03.787623 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 997/1966] 10.222.0.25 () {58 vars in 1233 bytes} [Sun Aug 7 10:55:03 2022] DELETE /identity/v3/projects/03e03e5f20c74c569a705e44db0f83ae => generated 0 bytes in 126 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 07 10:55:03.793604 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33a1262d-6c52-4aed-82fc-92fa5c89c924 None None] REQUEST_METHOD: `POST` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:55:03.794474 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33a1262d-6c52-4aed-82fc-92fa5c89c924 None None] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:55:03.794860 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33a1262d-6c52-4aed-82fc-92fa5c89c924 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:55:03.820152 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.auth.core [None req-33a1262d-6c52-4aed-82fc-92fa5c89c924 None None] MFA Rules not processed for user `bac4e6b76fe64217ac8fa44e71cffe6a`. Rule list: `[]` (Enabled: `True`). {{(pid=80143) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 10:55:03.821555 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-33a1262d-6c52-4aed-82fc-92fa5c89c924 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 10:55:03.882891 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 970/1967] 10.222.0.25 () {58 vars in 958 bytes} [Sun Aug 7 10:55:03 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 07 10:55:03.890764 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d951c82c-5bb9-4bca-bcdb-7b1afe799fc0 None None] REQUEST_METHOD: `POST` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:55:03.891088 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d951c82c-5bb9-4bca-bcdb-7b1afe799fc0 None None] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:55:03.891369 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d951c82c-5bb9-4bca-bcdb-7b1afe799fc0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:55:03.913780 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.auth.core [None req-d951c82c-5bb9-4bca-bcdb-7b1afe799fc0 None None] MFA Rules not processed for user `bac4e6b76fe64217ac8fa44e71cffe6a`. Rule list: `[]` (Enabled: `True`). {{(pid=80144) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 10:55:03.915023 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.fernet_utils [None req-d951c82c-5bb9-4bca-bcdb-7b1afe799fc0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80144) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 10:55:03.923676 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 998/1968] 10.222.0.25 () {58 vars in 958 bytes} [Sun Aug 7 10:55:03 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 07 10:55:03.927987 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33a1262d-6c52-4aed-82fc-92fa5c89c924 None None] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 10:55:03.937787 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33a1262d-6c52-4aed-82fc-92fa5c89c924 None None] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 10:55:03.942424 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c5c10b5-1452-4de5-8149-ed17193525a6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5ac47e3b09f43d19a23a0f457fe9e23', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 10:55:03.943062 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c5c10b5-1452-4de5-8149-ed17193525a6 admin admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:55:03.943348 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c5c10b5-1452-4de5-8149-ed17193525a6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:55:03.943624 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c5c10b5-1452-4de5-8149-ed17193525a6 admin admin] PATH_INFO: `/v3/domains` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:55:03.944272 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c5c10b5-1452-4de5-8149-ed17193525a6 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80143) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 10:55:03.944579 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c5c10b5-1452-4de5-8149-ed17193525a6 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 10:55:03.946163 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c5c10b5-1452-4de5-8149-ed17193525a6 admin admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 10:55:03.957643 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 971/1969] 10.222.0.25 () {58 vars in 1153 bytes} [Sun Aug 7 10:55:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 10:55:03.961913 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d951c82c-5bb9-4bca-bcdb-7b1afe799fc0 None None] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 10:55:03.971688 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d951c82c-5bb9-4bca-bcdb-7b1afe799fc0 None None] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 10:55:03.976227 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c4ace61-091c-4557-b38c-613e9076912b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5ac47e3b09f43d19a23a0f457fe9e23', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 10:55:03.976819 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c4ace61-091c-4557-b38c-613e9076912b admin admin] REQUEST_METHOD: `POST` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:55:03.977098 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c4ace61-091c-4557-b38c-613e9076912b admin admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:55:03.977370 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c4ace61-091c-4557-b38c-613e9076912b admin admin] PATH_INFO: `/v3/projects` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:55:03.978000 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c4ace61-091c-4557-b38c-613e9076912b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 10:55:03.979915 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c4ace61-091c-4557-b38c-613e9076912b admin admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 10:55:04.010528 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 999/1970] 10.222.0.25 () {60 vars in 1153 bytes} [Sun Aug 7 10:55:03 2022] POST /identity/v3/projects => generated 383 bytes in 46 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 07 10:55:04.011728 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c5c10b5-1452-4de5-8149-ed17193525a6 admin admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 10:55:04.022732 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c5c10b5-1452-4de5-8149-ed17193525a6 admin admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 10:55:04.079188 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fac1159c-7532-45fa-98d7-de9f012a47ef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5ac47e3b09f43d19a23a0f457fe9e23', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 10:55:04.079803 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fac1159c-7532-45fa-98d7-de9f012a47ef admin admin] REQUEST_METHOD: `POST` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:55:04.080082 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fac1159c-7532-45fa-98d7-de9f012a47ef admin admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:55:04.080355 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fac1159c-7532-45fa-98d7-de9f012a47ef admin admin] PATH_INFO: `/v3/users` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:55:04.080974 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fac1159c-7532-45fa-98d7-de9f012a47ef admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 10:55:04.082863 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fac1159c-7532-45fa-98d7-de9f012a47ef admin admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 10:55:04.108830 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 972/1971] 10.222.0.25 () {60 vars in 1144 bytes} [Sun Aug 7 10:55:04 2022] POST /identity/v3/users => generated 292 bytes in 94 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 07 10:55:04.110601 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c4ace61-091c-4557-b38c-613e9076912b admin admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 10:55:04.121292 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c4ace61-091c-4557-b38c-613e9076912b admin admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 10:55:04.126068 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57dd975e-ba40-4a3e-8315-ffaa399a7adc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5ac47e3b09f43d19a23a0f457fe9e23', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 10:55:04.126825 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57dd975e-ba40-4a3e-8315-ffaa399a7adc admin admin] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:55:04.127146 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57dd975e-ba40-4a3e-8315-ffaa399a7adc admin admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:55:04.127445 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57dd975e-ba40-4a3e-8315-ffaa399a7adc admin admin] PATH_INFO: `/v3/roles` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:55:04.128076 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57dd975e-ba40-4a3e-8315-ffaa399a7adc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 10:55:04.129642 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57dd975e-ba40-4a3e-8315-ffaa399a7adc admin admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 10:55:04.139788 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 1000/1972] 10.222.0.25 () {58 vars in 1122 bytes} [Sun Aug 7 10:55:04 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 07 10:55:04.144041 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fac1159c-7532-45fa-98d7-de9f012a47ef admin admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 10:55:04.154767 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fac1159c-7532-45fa-98d7-de9f012a47ef admin admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 10:55:04.160150 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6d31776-e097-4605-a67c-de4a0ae02603 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5ac47e3b09f43d19a23a0f457fe9e23', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 10:55:04.160783 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6d31776-e097-4605-a67c-de4a0ae02603 admin admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:55:04.161063 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6d31776-e097-4605-a67c-de4a0ae02603 admin admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:55:04.161338 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6d31776-e097-4605-a67c-de4a0ae02603 admin admin] PATH_INFO: `/v3/roles` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:55:04.162015 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6d31776-e097-4605-a67c-de4a0ae02603 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 10:55:04.163636 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6d31776-e097-4605-a67c-de4a0ae02603 admin admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 10:55:04.174285 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 973/1973] 10.222.0.25 () {58 vars in 1122 bytes} [Sun Aug 7 10:55:04 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 07 10:55:04.178368 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57dd975e-ba40-4a3e-8315-ffaa399a7adc admin admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 10:55:04.188419 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57dd975e-ba40-4a3e-8315-ffaa399a7adc admin admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 10:55:04.193043 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44a7abd9-53de-4ec1-b378-71591f1deb87 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5ac47e3b09f43d19a23a0f457fe9e23', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 10:55:04.193698 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44a7abd9-53de-4ec1-b378-71591f1deb87 admin admin] REQUEST_METHOD: `PUT` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:55:04.193984 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44a7abd9-53de-4ec1-b378-71591f1deb87 admin admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:55:04.194260 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44a7abd9-53de-4ec1-b378-71591f1deb87 admin admin] PATH_INFO: `/v3/projects/112b9a6f7ead454695e7efd26c0d845b/users/ee9ca3b9c7284fd1a600cbd3add41f56/roles/b98dfb3b9fee4644af2887a3ac3fd2e6` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:55:04.209095 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44a7abd9-53de-4ec1-b378-71591f1deb87 admin admin] RBAC: Authorizing `identity:create_grant(project_id=112b9a6f7ead454695e7efd26c0d845b, user_id=ee9ca3b9c7284fd1a600cbd3add41f56, role_id=b98dfb3b9fee4644af2887a3ac3fd2e6)` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 10:55:04.211760 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44a7abd9-53de-4ec1-b378-71591f1deb87 admin admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 10:55:04.225006 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 1001/1974] 10.222.0.25 () {60 vars in 1483 bytes} [Sun Aug 7 10:55:04 2022] PUT /identity/v3/projects/112b9a6f7ead454695e7efd26c0d845b/users/ee9ca3b9c7284fd1a600cbd3add41f56/roles/b98dfb3b9fee4644af2887a3ac3fd2e6 => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 07 10:55:04.230037 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38b44817-2bda-4b5a-b2d2-8f20e84cc392 None None] REQUEST_METHOD: `POST` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:55:04.230330 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38b44817-2bda-4b5a-b2d2-8f20e84cc392 None None] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:55:04.230612 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38b44817-2bda-4b5a-b2d2-8f20e84cc392 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:55:04.238094 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:DELETE=1,keystone:SELECT=96,keystone:INSERT=3 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 10:55:04.251512 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.auth.core [None req-38b44817-2bda-4b5a-b2d2-8f20e84cc392 None None] MFA Rules not processed for user `bac4e6b76fe64217ac8fa44e71cffe6a`. Rule list: `[]` (Enabled: `True`). {{(pid=80143) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 10:55:04.252692 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-38b44817-2bda-4b5a-b2d2-8f20e84cc392 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 10:55:04.313740 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 974/1975] 10.222.0.25 () {58 vars in 958 bytes} [Sun Aug 7 10:55:04 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 07 10:55:04.323554 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44a7abd9-53de-4ec1-b378-71591f1deb87 admin admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 10:55:04.325328 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.fernet_utils [None req-44a7abd9-53de-4ec1-b378-71591f1deb87 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80144) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 10:55:04.391941 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44a7abd9-53de-4ec1-b378-71591f1deb87 admin admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 10:55:04.395809 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c64deb2-6131-4cfb-b8ea-819a6c381ed1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be1283c9bf47424c9aa3ad27cab03027', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6de1f3cac02c4bcdb05dd2337176033d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 10:55:04.396427 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c64deb2-6131-4cfb-b8ea-819a6c381ed1 service neutron] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:55:04.396709 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c64deb2-6131-4cfb-b8ea-819a6c381ed1 service neutron] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:55:04.396984 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c64deb2-6131-4cfb-b8ea-819a6c381ed1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:55:04.407015 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c64deb2-6131-4cfb-b8ea-819a6c381ed1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 10:55:04.408662 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c64deb2-6131-4cfb-b8ea-819a6c381ed1 service neutron] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 10:55:04.419402 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 1002/1976] 10.222.0.25 () {60 vars in 1365 bytes} [Sun Aug 7 10:55:04 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 07 10:55:05.089519 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38b44817-2bda-4b5a-b2d2-8f20e84cc392 None None] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 10:55:05.091290 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-38b44817-2bda-4b5a-b2d2-8f20e84cc392 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 10:55:05.169998 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38b44817-2bda-4b5a-b2d2-8f20e84cc392 None None] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 10:55:05.174016 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-212201bd-0194-4cda-8002-52573b9115b0 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88dbd6dfd1164994a3e16da553561910', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6de1f3cac02c4bcdb05dd2337176033d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 10:55:05.174653 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-212201bd-0194-4cda-8002-52573b9115b0 service cinder] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:55:05.174900 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-212201bd-0194-4cda-8002-52573b9115b0 service cinder] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:55:05.175238 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-212201bd-0194-4cda-8002-52573b9115b0 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:55:05.177113 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-212201bd-0194-4cda-8002-52573b9115b0 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 10:55:05.253907 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-212201bd-0194-4cda-8002-52573b9115b0 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 10:55:05.255436 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-212201bd-0194-4cda-8002-52573b9115b0 service cinder] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 10:55:05.273204 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 975/1977] 10.222.0.25 () {60 vars in 1365 bytes} [Sun Aug 7 10:55:05 2022] GET /identity/v3/auth/tokens => generated 3319 bytes in 184 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 07 10:55:09.862498 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cc18b8a-836d-4655-9435-5e2f3448df04 None None] REQUEST_METHOD: `POST` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:55:09.863066 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cc18b8a-836d-4655-9435-5e2f3448df04 None None] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:55:09.863066 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cc18b8a-836d-4655-9435-5e2f3448df04 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:55:09.896671 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.auth.core [None req-6cc18b8a-836d-4655-9435-5e2f3448df04 None None] MFA Rules not processed for user `ee9ca3b9c7284fd1a600cbd3add41f56`. Rule list: `[]` (Enabled: `True`). {{(pid=80144) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 10:55:09.898378 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.fernet_utils [None req-6cc18b8a-836d-4655-9435-5e2f3448df04 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80144) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 10:55:09.902670 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 10:55:09.976674 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 1003/1978] 10.222.0.25 () {58 vars in 958 bytes} [Sun Aug 7 10:55:09 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 07 10:55:09.981649 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-212201bd-0194-4cda-8002-52573b9115b0 service cinder] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 10:55:09.994975 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-212201bd-0194-4cda-8002-52573b9115b0 service cinder] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 10:55:10.000273 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c56d5795-c20e-4140-9a69-3b37bb4f79fa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5ac47e3b09f43d19a23a0f457fe9e23', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 10:55:10.001024 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c56d5795-c20e-4140-9a69-3b37bb4f79fa admin admin] REQUEST_METHOD: `POST` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:55:10.001382 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c56d5795-c20e-4140-9a69-3b37bb4f79fa admin admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:55:10.001731 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c56d5795-c20e-4140-9a69-3b37bb4f79fa admin admin] PATH_INFO: `/v3/projects` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:55:10.002624 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c56d5795-c20e-4140-9a69-3b37bb4f79fa admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 10:55:10.004506 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c56d5795-c20e-4140-9a69-3b37bb4f79fa admin admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 10:55:10.050771 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 976/1979] 10.222.0.25 () {60 vars in 1153 bytes} [Sun Aug 7 10:55:09 2022] POST /identity/v3/projects => generated 385 bytes in 69 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 07 10:55:10.054399 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cc18b8a-836d-4655-9435-5e2f3448df04 None None] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 10:55:10.068311 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cc18b8a-836d-4655-9435-5e2f3448df04 None None] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 10:55:10.150244 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1feaa89-eb22-4bdc-a793-82da4a77675f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5ac47e3b09f43d19a23a0f457fe9e23', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 10:55:10.151172 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1feaa89-eb22-4bdc-a793-82da4a77675f admin admin] REQUEST_METHOD: `POST` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:55:10.151645 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1feaa89-eb22-4bdc-a793-82da4a77675f admin admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:55:10.152039 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1feaa89-eb22-4bdc-a793-82da4a77675f admin admin] PATH_INFO: `/v3/users` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:55:10.152861 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1feaa89-eb22-4bdc-a793-82da4a77675f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 10:55:10.155450 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1feaa89-eb22-4bdc-a793-82da4a77675f admin admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 10:55:10.182356 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 1004/1980] 10.222.0.25 () {60 vars in 1144 bytes} [Sun Aug 7 10:55:10 2022] POST /identity/v3/users => generated 299 bytes in 128 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 07 10:55:10.186507 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c56d5795-c20e-4140-9a69-3b37bb4f79fa admin admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 10:55:10.197665 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c56d5795-c20e-4140-9a69-3b37bb4f79fa admin admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 10:55:10.203449 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb70bad2-f4e0-4323-863b-c9a317d999c4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5ac47e3b09f43d19a23a0f457fe9e23', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 10:55:10.204083 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb70bad2-f4e0-4323-863b-c9a317d999c4 admin admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:55:10.204367 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb70bad2-f4e0-4323-863b-c9a317d999c4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:55:10.204643 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb70bad2-f4e0-4323-863b-c9a317d999c4 admin admin] PATH_INFO: `/v3/roles` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:55:10.205288 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb70bad2-f4e0-4323-863b-c9a317d999c4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 10:55:10.206890 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb70bad2-f4e0-4323-863b-c9a317d999c4 admin admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 10:55:10.219623 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 977/1981] 10.222.0.25 () {58 vars in 1122 bytes} [Sun Aug 7 10:55:10 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 07 10:55:10.223930 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1feaa89-eb22-4bdc-a793-82da4a77675f admin admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 10:55:10.236033 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1feaa89-eb22-4bdc-a793-82da4a77675f admin admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 10:55:10.241499 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7037475-6519-496a-a8d4-c72940a57ed8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5ac47e3b09f43d19a23a0f457fe9e23', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 10:55:10.242175 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7037475-6519-496a-a8d4-c72940a57ed8 admin admin] REQUEST_METHOD: `PUT` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:55:10.242455 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7037475-6519-496a-a8d4-c72940a57ed8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:55:10.242746 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7037475-6519-496a-a8d4-c72940a57ed8 admin admin] PATH_INFO: `/v3/projects/7950a7f5cbac420b992ad97be30314cf/users/a0be3da3bf484da8b95fa985e43df9f5/roles/81b1f1bcee2a473088fbbd327c1c5b23` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:55:10.260550 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7037475-6519-496a-a8d4-c72940a57ed8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7950a7f5cbac420b992ad97be30314cf, user_id=a0be3da3bf484da8b95fa985e43df9f5, role_id=81b1f1bcee2a473088fbbd327c1c5b23)` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 10:55:10.262989 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7037475-6519-496a-a8d4-c72940a57ed8 admin admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 10:55:10.278850 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 1005/1982] 10.222.0.25 () {60 vars in 1483 bytes} [Sun Aug 7 10:55:10 2022] PUT /identity/v3/projects/7950a7f5cbac420b992ad97be30314cf/users/a0be3da3bf484da8b95fa985e43df9f5/roles/81b1f1bcee2a473088fbbd327c1c5b23 => generated 0 bytes in 55 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 07 10:55:10.283649 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb70bad2-f4e0-4323-863b-c9a317d999c4 admin admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 10:55:10.293992 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb70bad2-f4e0-4323-863b-c9a317d999c4 admin admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 10:55:10.310810 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 10:55:10.352543 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-130b6752-d88c-4ef8-bbc4-2e2734a56a75 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5ac47e3b09f43d19a23a0f457fe9e23', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 10:55:10.353155 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-130b6752-d88c-4ef8-bbc4-2e2734a56a75 admin admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:55:10.353441 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-130b6752-d88c-4ef8-bbc4-2e2734a56a75 admin admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:55:10.353716 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-130b6752-d88c-4ef8-bbc4-2e2734a56a75 admin admin] PATH_INFO: `/v3/roles` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:55:10.354360 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-130b6752-d88c-4ef8-bbc4-2e2734a56a75 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 10:55:10.355975 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-130b6752-d88c-4ef8-bbc4-2e2734a56a75 admin admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 10:55:10.365643 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 978/1983] 10.222.0.25 () {58 vars in 1122 bytes} [Sun Aug 7 10:55:10 2022] GET /identity/v3/roles => generated 1420 bytes in 83 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 07 10:55:10.369995 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7037475-6519-496a-a8d4-c72940a57ed8 admin admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 10:55:10.382844 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7037475-6519-496a-a8d4-c72940a57ed8 admin admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 10:55:10.387675 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b80915a8-7030-4b58-a84b-55c73d30f819 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5ac47e3b09f43d19a23a0f457fe9e23', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 10:55:10.388350 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b80915a8-7030-4b58-a84b-55c73d30f819 admin admin] REQUEST_METHOD: `PUT` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:55:10.388635 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b80915a8-7030-4b58-a84b-55c73d30f819 admin admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:55:10.388911 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b80915a8-7030-4b58-a84b-55c73d30f819 admin admin] PATH_INFO: `/v3/projects/7950a7f5cbac420b992ad97be30314cf/users/a0be3da3bf484da8b95fa985e43df9f5/roles/81b1f1bcee2a473088fbbd327c1c5b23` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:55:10.391453 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b80915a8-7030-4b58-a84b-55c73d30f819 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7950a7f5cbac420b992ad97be30314cf, user_id=a0be3da3bf484da8b95fa985e43df9f5, role_id=81b1f1bcee2a473088fbbd327c1c5b23)` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 10:55:10.394146 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b80915a8-7030-4b58-a84b-55c73d30f819 admin admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 10:55:10.408324 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 1006/1984] 10.222.0.25 () {60 vars in 1483 bytes} [Sun Aug 7 10:55:10 2022] PUT /identity/v3/projects/7950a7f5cbac420b992ad97be30314cf/users/a0be3da3bf484da8b95fa985e43df9f5/roles/81b1f1bcee2a473088fbbd327c1c5b23 => generated 0 bytes in 39 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 07 10:55:16.617171 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a014b34-4ea5-47f2-9e12-eb14d3f694eb None None] REQUEST_METHOD: `POST` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:55:16.617666 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a014b34-4ea5-47f2-9e12-eb14d3f694eb None None] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:55:16.617759 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a014b34-4ea5-47f2-9e12-eb14d3f694eb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:55:16.649082 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.auth.core [None req-1a014b34-4ea5-47f2-9e12-eb14d3f694eb None None] MFA Rules not processed for user `a0be3da3bf484da8b95fa985e43df9f5`. Rule list: `[]` (Enabled: `True`). {{(pid=80143) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 10:55:16.650511 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-1a014b34-4ea5-47f2-9e12-eb14d3f694eb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 10:55:16.776271 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 979/1985] 10.222.0.25 () {58 vars in 958 bytes} [Sun Aug 7 10:55:16 2022] POST /identity/v3/auth/tokens => generated 3329 bytes in 156 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 07 10:55:16.784533 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b80915a8-7030-4b58-a84b-55c73d30f819 admin admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 10:55:16.786292 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.fernet_utils [None req-b80915a8-7030-4b58-a84b-55c73d30f819 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80144) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 10:55:16.862016 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b80915a8-7030-4b58-a84b-55c73d30f819 admin admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 10:55:16.866329 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d60607fe-00ef-4d35-8ed3-af124c555f19 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a11186d9d0d6488b953ede2f1628148d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6de1f3cac02c4bcdb05dd2337176033d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 10:55:16.866987 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d60607fe-00ef-4d35-8ed3-af124c555f19 service glance] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:55:16.867345 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d60607fe-00ef-4d35-8ed3-af124c555f19 service glance] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:55:16.867771 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d60607fe-00ef-4d35-8ed3-af124c555f19 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:55:16.880935 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 10:55:16.936005 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d60607fe-00ef-4d35-8ed3-af124c555f19 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 10:55:16.937872 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d60607fe-00ef-4d35-8ed3-af124c555f19 service glance] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 10:55:16.951243 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 1007/1986] 10.222.0.25 () {60 vars in 1365 bytes} [Sun Aug 7 10:55:16 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 167 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 07 10:55:17.139193 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b31555f1-b112-47ac-b35d-991a19329ffb None None] REQUEST_METHOD: `POST` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:55:17.139495 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b31555f1-b112-47ac-b35d-991a19329ffb None None] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:55:17.139778 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b31555f1-b112-47ac-b35d-991a19329ffb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:55:17.176090 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.auth.core [None req-b31555f1-b112-47ac-b35d-991a19329ffb None None] MFA Rules not processed for user `de1d6c24b1a14d10a20d82a4c65793d8`. Rule list: `[]` (Enabled: `True`). {{(pid=80143) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 10:55:17.177329 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-b31555f1-b112-47ac-b35d-991a19329ffb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 10:55:17.258287 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 980/1987] 10.222.0.25 () {58 vars in 1009 bytes} [Sun Aug 7 10:55:17 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 07 10:55:17.271992 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d60607fe-00ef-4d35-8ed3-af124c555f19 service glance] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 10:55:17.273603 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.fernet_utils [None req-d60607fe-00ef-4d35-8ed3-af124c555f19 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80144) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 10:55:17.356248 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d60607fe-00ef-4d35-8ed3-af124c555f19 service glance] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 10:55:17.362351 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e0b3791-804a-40c9-aab7-4270204af244 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69490ea2a9a340bf91aca38bf5655b45', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6de1f3cac02c4bcdb05dd2337176033d', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 10:55:17.362994 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e0b3791-804a-40c9-aab7-4270204af244 service swift] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:55:17.363281 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e0b3791-804a-40c9-aab7-4270204af244 service swift] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:55:17.363557 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e0b3791-804a-40c9-aab7-4270204af244 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:55:17.372176 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e0b3791-804a-40c9-aab7-4270204af244 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 10:55:17.373679 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e0b3791-804a-40c9-aab7-4270204af244 service swift] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 10:55:17.382889 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 1008/1988] 10.222.0.25 () {60 vars in 1384 bytes} [Sun Aug 7 10:55:17 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 111 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 07 10:55:23.826563 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b31555f1-b112-47ac-b35d-991a19329ffb None None] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 10:55:23.828238 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-b31555f1-b112-47ac-b35d-991a19329ffb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 10:55:23.895098 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=100 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 10:55:23.934374 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b31555f1-b112-47ac-b35d-991a19329ffb None None] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 10:55:23.939775 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f109b3c1-bb20-45a3-82fa-4e4a2f9cad49 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5ac47e3b09f43d19a23a0f457fe9e23', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 10:55:23.940436 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f109b3c1-bb20-45a3-82fa-4e4a2f9cad49 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:55:23.940719 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f109b3c1-bb20-45a3-82fa-4e4a2f9cad49 admin admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:55:23.941133 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f109b3c1-bb20-45a3-82fa-4e4a2f9cad49 admin admin] PATH_INFO: `/v3/users/9032af01895a47d08540b9c60226f640` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:55:23.942371 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f109b3c1-bb20-45a3-82fa-4e4a2f9cad49 admin admin] RBAC: Authorizing `identity:delete_user(user_id=9032af01895a47d08540b9c60226f640)` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 10:55:23.944470 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f109b3c1-bb20-45a3-82fa-4e4a2f9cad49 admin admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 10:55:24.008617 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-f109b3c1-bb20-45a3-82fa-4e4a2f9cad49 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9032af01895a47d08540b9c60226f640'} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 10:55:24.018833 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-f109b3c1-bb20-45a3-82fa-4e4a2f9cad49 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9032af01895a47d08540b9c60226f640'} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 10:55:24.039077 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-f109b3c1-bb20-45a3-82fa-4e4a2f9cad49 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9032af01895a47d08540b9c60226f640'} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 10:55:24.039960 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-f109b3c1-bb20-45a3-82fa-4e4a2f9cad49 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9032af01895a47d08540b9c60226f640'} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 10:55:24.050104 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-f109b3c1-bb20-45a3-82fa-4e4a2f9cad49 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9032af01895a47d08540b9c60226f640'} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 10:55:24.059018 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 981/1989] 10.222.0.25 () {58 vars in 1224 bytes} [Sun Aug 7 10:55:23 2022] DELETE /identity/v3/users/9032af01895a47d08540b9c60226f640 => generated 0 bytes in 231 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 07 10:55:24.062847 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a975234-78c5-4cf2-a7fb-83ca162ed607 None None] REQUEST_METHOD: `POST` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:55:24.063406 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a975234-78c5-4cf2-a7fb-83ca162ed607 None None] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:55:24.063552 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a975234-78c5-4cf2-a7fb-83ca162ed607 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:55:24.086094 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.auth.core [None req-8a975234-78c5-4cf2-a7fb-83ca162ed607 None None] MFA Rules not processed for user `bac4e6b76fe64217ac8fa44e71cffe6a`. Rule list: `[]` (Enabled: `True`). {{(pid=80144) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 10:55:24.086094 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.fernet_utils [None req-8a975234-78c5-4cf2-a7fb-83ca162ed607 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80144) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 10:55:24.157941 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 1009/1990] 10.222.0.25 () {58 vars in 958 bytes} [Sun Aug 7 10:55:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 07 10:55:24.164657 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f109b3c1-bb20-45a3-82fa-4e4a2f9cad49 admin admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 10:55:24.166328 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-f109b3c1-bb20-45a3-82fa-4e4a2f9cad49 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 10:55:24.280636 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f109b3c1-bb20-45a3-82fa-4e4a2f9cad49 admin admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 10:55:24.286177 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f02d53c6-d47e-46fb-9535-dd4e40b28b85 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be1283c9bf47424c9aa3ad27cab03027', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6de1f3cac02c4bcdb05dd2337176033d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 10:55:24.286799 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f02d53c6-d47e-46fb-9535-dd4e40b28b85 service neutron] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:55:24.287122 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f02d53c6-d47e-46fb-9535-dd4e40b28b85 service neutron] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:55:24.287393 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f02d53c6-d47e-46fb-9535-dd4e40b28b85 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:55:24.304084 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f02d53c6-d47e-46fb-9535-dd4e40b28b85 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 10:55:24.306796 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f02d53c6-d47e-46fb-9535-dd4e40b28b85 service neutron] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 10:55:24.319714 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 982/1991] 10.222.0.25 () {60 vars in 1365 bytes} [Sun Aug 7 10:55:24 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 156 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 07 10:55:24.806231 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a975234-78c5-4cf2-a7fb-83ca162ed607 None None] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 10:55:24.807952 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.fernet_utils [None req-8a975234-78c5-4cf2-a7fb-83ca162ed607 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80144) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 10:55:24.826855 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a975234-78c5-4cf2-a7fb-83ca162ed607 None None] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 10:55:24.843809 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ea6ccbd-b05c-41e0-806a-d5ee32eed151 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5ac47e3b09f43d19a23a0f457fe9e23', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 10:55:24.844869 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ea6ccbd-b05c-41e0-806a-d5ee32eed151 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:55:24.845411 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ea6ccbd-b05c-41e0-806a-d5ee32eed151 admin admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:55:24.845945 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ea6ccbd-b05c-41e0-806a-d5ee32eed151 admin admin] PATH_INFO: `/v3/projects/a3d26e4432c742c8b69fe73882c23b1a` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:55:24.847403 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ea6ccbd-b05c-41e0-806a-d5ee32eed151 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a3d26e4432c742c8b69fe73882c23b1a)` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 10:55:24.850556 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ea6ccbd-b05c-41e0-806a-d5ee32eed151 admin admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 10:55:24.969795 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=97,keystone:DELETE=3 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 10:55:24.982130 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.notifications [None req-9ea6ccbd-b05c-41e0-806a-d5ee32eed151 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a3d26e4432c742c8b69fe73882c23b1a', 'request_id': 'req-9ea6ccbd-b05c-41e0-806a-d5ee32eed151'} {{(pid=80144) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 10:55:24.994667 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.notifications [None req-9ea6ccbd-b05c-41e0-806a-d5ee32eed151 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a3d26e4432c742c8b69fe73882c23b1a', 'request_id': 'req-9ea6ccbd-b05c-41e0-806a-d5ee32eed151'} {{(pid=80144) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 10:55:25.008325 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.notifications [None req-9ea6ccbd-b05c-41e0-806a-d5ee32eed151 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a3d26e4432c742c8b69fe73882c23b1a', 'request_id': 'req-9ea6ccbd-b05c-41e0-806a-d5ee32eed151'} {{(pid=80144) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 10:55:25.036137 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.notifications [None req-9ea6ccbd-b05c-41e0-806a-d5ee32eed151 admin admin] The token cache is being invalidate because project a3d26e4432c742c8b69fe73882c23b1a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80144) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 07 10:55:25.036903 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.notifications [None req-9ea6ccbd-b05c-41e0-806a-d5ee32eed151 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80144) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 10:55:25.039158 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.notifications [None req-9ea6ccbd-b05c-41e0-806a-d5ee32eed151 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80144) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 10:55:25.042941 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 1010/1992] 10.222.0.25 () {58 vars in 1233 bytes} [Sun Aug 7 10:55:24 2022] DELETE /identity/v3/projects/a3d26e4432c742c8b69fe73882c23b1a => generated 0 bytes in 234 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 07 10:55:25.045039 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f02d53c6-d47e-46fb-9535-dd4e40b28b85 service neutron] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 10:55:25.065888 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f02d53c6-d47e-46fb-9535-dd4e40b28b85 service neutron] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 10:55:25.187297 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a1912ec-e878-4afd-a8db-bc036352e348 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5ac47e3b09f43d19a23a0f457fe9e23', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 10:55:25.189892 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a1912ec-e878-4afd-a8db-bc036352e348 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:55:25.195277 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a1912ec-e878-4afd-a8db-bc036352e348 admin admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:55:25.195477 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a1912ec-e878-4afd-a8db-bc036352e348 admin admin] PATH_INFO: `/v3/users/8bf3c99fd787437bb0178f1f3b9b77a4` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:55:25.198107 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a1912ec-e878-4afd-a8db-bc036352e348 admin admin] RBAC: Authorizing `identity:delete_user(user_id=8bf3c99fd787437bb0178f1f3b9b77a4)` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 10:55:25.200387 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a1912ec-e878-4afd-a8db-bc036352e348 admin admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 10:55:25.233546 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 10:55:25.325119 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-1a1912ec-e878-4afd-a8db-bc036352e348 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8bf3c99fd787437bb0178f1f3b9b77a4'} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 10:55:25.341467 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-1a1912ec-e878-4afd-a8db-bc036352e348 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8bf3c99fd787437bb0178f1f3b9b77a4'} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 10:55:25.360752 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-1a1912ec-e878-4afd-a8db-bc036352e348 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8bf3c99fd787437bb0178f1f3b9b77a4'} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 10:55:25.361714 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-1a1912ec-e878-4afd-a8db-bc036352e348 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8bf3c99fd787437bb0178f1f3b9b77a4'} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 10:55:25.373467 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-1a1912ec-e878-4afd-a8db-bc036352e348 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8bf3c99fd787437bb0178f1f3b9b77a4'} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 10:55:25.402464 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 983/1993] 10.222.0.25 () {58 vars in 1224 bytes} [Sun Aug 7 10:55:25 2022] DELETE /identity/v3/users/8bf3c99fd787437bb0178f1f3b9b77a4 => generated 0 bytes in 354 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 07 10:55:25.840305 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ea6ccbd-b05c-41e0-806a-d5ee32eed151 admin admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 10:55:25.841873 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.fernet_utils [None req-9ea6ccbd-b05c-41e0-806a-d5ee32eed151 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80144) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 10:55:25.893866 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ea6ccbd-b05c-41e0-806a-d5ee32eed151 admin admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 10:55:26.002242 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-670ef170-20e3-44f8-92ff-f68a5fd25ea6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5ac47e3b09f43d19a23a0f457fe9e23', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 10:55:26.003618 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-670ef170-20e3-44f8-92ff-f68a5fd25ea6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:55:26.003818 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-670ef170-20e3-44f8-92ff-f68a5fd25ea6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:55:26.004140 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-670ef170-20e3-44f8-92ff-f68a5fd25ea6 admin admin] PATH_INFO: `/v3/projects/25beed62fc9a44049a5693fc25c519f7` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:55:26.005374 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-670ef170-20e3-44f8-92ff-f68a5fd25ea6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=25beed62fc9a44049a5693fc25c519f7)` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 10:55:26.007577 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-670ef170-20e3-44f8-92ff-f68a5fd25ea6 admin admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 10:55:26.090422 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.notifications [None req-670ef170-20e3-44f8-92ff-f68a5fd25ea6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '25beed62fc9a44049a5693fc25c519f7', 'request_id': 'req-670ef170-20e3-44f8-92ff-f68a5fd25ea6'} {{(pid=80144) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 10:55:26.098717 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.notifications [None req-670ef170-20e3-44f8-92ff-f68a5fd25ea6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '25beed62fc9a44049a5693fc25c519f7', 'request_id': 'req-670ef170-20e3-44f8-92ff-f68a5fd25ea6'} {{(pid=80144) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 10:55:26.111531 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.notifications [None req-670ef170-20e3-44f8-92ff-f68a5fd25ea6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '25beed62fc9a44049a5693fc25c519f7', 'request_id': 'req-670ef170-20e3-44f8-92ff-f68a5fd25ea6'} {{(pid=80144) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 10:55:26.124648 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.notifications [None req-670ef170-20e3-44f8-92ff-f68a5fd25ea6 admin admin] The token cache is being invalidate because project 25beed62fc9a44049a5693fc25c519f7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80144) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 07 10:55:26.125000 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.notifications [None req-670ef170-20e3-44f8-92ff-f68a5fd25ea6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80144) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 10:55:26.125769 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.notifications [None req-670ef170-20e3-44f8-92ff-f68a5fd25ea6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80144) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 10:55:26.131845 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 1011/1994] 10.222.0.25 () {58 vars in 1233 bytes} [Sun Aug 7 10:55:25 2022] DELETE /identity/v3/projects/25beed62fc9a44049a5693fc25c519f7 => generated 0 bytes in 288 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 07 10:55:26.134281 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca4b0384-e1cc-4b2e-b5a9-cb20662623b1 None None] REQUEST_METHOD: `POST` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:55:26.135450 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca4b0384-e1cc-4b2e-b5a9-cb20662623b1 None None] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:55:26.135836 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca4b0384-e1cc-4b2e-b5a9-cb20662623b1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:55:26.170454 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.auth.core [None req-ca4b0384-e1cc-4b2e-b5a9-cb20662623b1 None None] MFA Rules not processed for user `bac4e6b76fe64217ac8fa44e71cffe6a`. Rule list: `[]` (Enabled: `True`). {{(pid=80143) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 10:55:26.172186 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-ca4b0384-e1cc-4b2e-b5a9-cb20662623b1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 10:55:26.267670 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 984/1995] 10.222.0.25 () {58 vars in 958 bytes} [Sun Aug 7 10:55:26 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 134 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 07 10:55:26.276138 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1da4b2f6-8493-4de1-9213-18045bebaa92 None None] REQUEST_METHOD: `POST` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:55:26.276698 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1da4b2f6-8493-4de1-9213-18045bebaa92 None None] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:55:26.277222 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1da4b2f6-8493-4de1-9213-18045bebaa92 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:55:26.301630 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.auth.core [None req-1da4b2f6-8493-4de1-9213-18045bebaa92 None None] MFA Rules not processed for user `bac4e6b76fe64217ac8fa44e71cffe6a`. Rule list: `[]` (Enabled: `True`). {{(pid=80144) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 10:55:26.314605 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.fernet_utils [None req-1da4b2f6-8493-4de1-9213-18045bebaa92 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80144) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 10:55:26.326584 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 1012/1996] 10.222.0.25 () {58 vars in 958 bytes} [Sun Aug 7 10:55:26 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 49 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 07 10:55:26.328068 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca4b0384-e1cc-4b2e-b5a9-cb20662623b1 None None] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 10:55:26.340453 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca4b0384-e1cc-4b2e-b5a9-cb20662623b1 None None] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 10:55:26.347834 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdc941de-a06a-4f0b-873b-5182f4a4a03b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5ac47e3b09f43d19a23a0f457fe9e23', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 10:55:26.348974 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdc941de-a06a-4f0b-873b-5182f4a4a03b admin admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:55:26.349348 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdc941de-a06a-4f0b-873b-5182f4a4a03b admin admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:55:26.349690 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdc941de-a06a-4f0b-873b-5182f4a4a03b admin admin] PATH_INFO: `/v3/domains` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:55:26.350452 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdc941de-a06a-4f0b-873b-5182f4a4a03b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80143) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 10:55:26.350907 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdc941de-a06a-4f0b-873b-5182f4a4a03b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 10:55:26.352720 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdc941de-a06a-4f0b-873b-5182f4a4a03b admin admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 10:55:26.377384 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 985/1997] 10.222.0.25 () {58 vars in 1153 bytes} [Sun Aug 7 10:55:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 50 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 10:55:26.381678 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1da4b2f6-8493-4de1-9213-18045bebaa92 None None] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 10:55:26.399898 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1da4b2f6-8493-4de1-9213-18045bebaa92 None None] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 10:55:26.405645 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ebff4af-c1a2-4448-a2d4-81e88289ca23 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5ac47e3b09f43d19a23a0f457fe9e23', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 10:55:26.406683 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ebff4af-c1a2-4448-a2d4-81e88289ca23 admin admin] REQUEST_METHOD: `POST` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:55:26.428734 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ebff4af-c1a2-4448-a2d4-81e88289ca23 admin admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:55:26.429155 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ebff4af-c1a2-4448-a2d4-81e88289ca23 admin admin] PATH_INFO: `/v3/projects` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:55:26.429954 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ebff4af-c1a2-4448-a2d4-81e88289ca23 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 10:55:26.432259 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ebff4af-c1a2-4448-a2d4-81e88289ca23 admin admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 10:55:26.483822 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 1013/1998] 10.222.0.25 () {60 vars in 1153 bytes} [Sun Aug 7 10:55:26 2022] POST /identity/v3/projects => generated 375 bytes in 102 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 07 10:55:26.487773 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdc941de-a06a-4f0b-873b-5182f4a4a03b admin admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 10:55:26.500598 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdc941de-a06a-4f0b-873b-5182f4a4a03b admin admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 10:55:26.573596 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3b4dbfc-dbed-480b-8c53-acda26cb1154 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5ac47e3b09f43d19a23a0f457fe9e23', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 10:55:26.574213 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3b4dbfc-dbed-480b-8c53-acda26cb1154 admin admin] REQUEST_METHOD: `POST` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:55:26.574495 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3b4dbfc-dbed-480b-8c53-acda26cb1154 admin admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:55:26.574766 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3b4dbfc-dbed-480b-8c53-acda26cb1154 admin admin] PATH_INFO: `/v3/users` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:55:26.575440 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3b4dbfc-dbed-480b-8c53-acda26cb1154 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 10:55:26.578145 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3b4dbfc-dbed-480b-8c53-acda26cb1154 admin admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 10:55:26.607822 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 986/1999] 10.222.0.25 () {60 vars in 1144 bytes} [Sun Aug 7 10:55:26 2022] POST /identity/v3/users => generated 288 bytes in 121 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 07 10:55:26.612665 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ebff4af-c1a2-4448-a2d4-81e88289ca23 admin admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 10:55:26.631434 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ebff4af-c1a2-4448-a2d4-81e88289ca23 admin admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 10:55:26.632029 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ffae64d-d366-4f10-af67-990e5123c1d7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5ac47e3b09f43d19a23a0f457fe9e23', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 10:55:26.632373 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ffae64d-d366-4f10-af67-990e5123c1d7 admin admin] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:55:26.632910 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ffae64d-d366-4f10-af67-990e5123c1d7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:55:26.633392 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ffae64d-d366-4f10-af67-990e5123c1d7 admin admin] PATH_INFO: `/v3/roles` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:55:26.634142 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ffae64d-d366-4f10-af67-990e5123c1d7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 10:55:26.635882 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ffae64d-d366-4f10-af67-990e5123c1d7 admin admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 10:55:26.648645 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 1014/2000] 10.222.0.25 () {58 vars in 1122 bytes} [Sun Aug 7 10:55:26 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 07 10:55:26.652647 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3b4dbfc-dbed-480b-8c53-acda26cb1154 admin admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 10:55:26.662175 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:DELETE=10,keystone:SELECT=86,keystone:INSERT=4 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 10:55:26.667003 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3b4dbfc-dbed-480b-8c53-acda26cb1154 admin admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 10:55:26.676749 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b46afa1c-5ef4-4441-9269-02dae463cf19 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5ac47e3b09f43d19a23a0f457fe9e23', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 10:55:26.677498 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b46afa1c-5ef4-4441-9269-02dae463cf19 admin admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:55:26.677896 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b46afa1c-5ef4-4441-9269-02dae463cf19 admin admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:55:26.678676 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b46afa1c-5ef4-4441-9269-02dae463cf19 admin admin] PATH_INFO: `/v3/roles` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:55:26.679534 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b46afa1c-5ef4-4441-9269-02dae463cf19 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 10:55:26.683375 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b46afa1c-5ef4-4441-9269-02dae463cf19 admin admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 10:55:26.694835 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 987/2001] 10.222.0.25 () {58 vars in 1122 bytes} [Sun Aug 7 10:55:26 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 07 10:55:26.699026 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ffae64d-d366-4f10-af67-990e5123c1d7 admin admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 10:55:26.710670 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ffae64d-d366-4f10-af67-990e5123c1d7 admin admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 10:55:26.718751 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b18e228c-2a43-4d46-8fd1-c5c477522ce5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5ac47e3b09f43d19a23a0f457fe9e23', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 10:55:26.719483 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b18e228c-2a43-4d46-8fd1-c5c477522ce5 admin admin] REQUEST_METHOD: `PUT` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:55:26.719771 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b18e228c-2a43-4d46-8fd1-c5c477522ce5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:55:26.720054 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b18e228c-2a43-4d46-8fd1-c5c477522ce5 admin admin] PATH_INFO: `/v3/projects/bfcf7aebfb9e4963bc55842c72ed5f41/users/36fb5eeaf0184da5b86486d8320cfa14/roles/b98dfb3b9fee4644af2887a3ac3fd2e6` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:55:26.734832 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=86,keystone:DELETE=11,keystone:INSERT=3 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 10:55:26.738574 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b18e228c-2a43-4d46-8fd1-c5c477522ce5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=bfcf7aebfb9e4963bc55842c72ed5f41, user_id=36fb5eeaf0184da5b86486d8320cfa14, role_id=b98dfb3b9fee4644af2887a3ac3fd2e6)` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 10:55:26.743878 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b18e228c-2a43-4d46-8fd1-c5c477522ce5 admin admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 10:55:26.779775 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adebc599-db53-4ff2-ba07-18c4848e7ac8 None None] REQUEST_METHOD: `POST` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:55:26.780001 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 1015/2002] 10.222.0.25 () {60 vars in 1483 bytes} [Sun Aug 7 10:55:26 2022] PUT /identity/v3/projects/bfcf7aebfb9e4963bc55842c72ed5f41/users/36fb5eeaf0184da5b86486d8320cfa14/roles/b98dfb3b9fee4644af2887a3ac3fd2e6 => generated 0 bytes in 69 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 07 10:55:26.780163 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adebc599-db53-4ff2-ba07-18c4848e7ac8 None None] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:55:26.780447 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adebc599-db53-4ff2-ba07-18c4848e7ac8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:55:26.811163 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.auth.core [None req-adebc599-db53-4ff2-ba07-18c4848e7ac8 None None] MFA Rules not processed for user `36fb5eeaf0184da5b86486d8320cfa14`. Rule list: `[]` (Enabled: `True`). {{(pid=80143) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 10:55:26.813006 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-adebc599-db53-4ff2-ba07-18c4848e7ac8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 10:55:26.931417 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 988/2003] 10.222.0.25 () {58 vars in 958 bytes} [Sun Aug 7 10:55:26 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 152 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 07 10:55:26.963139 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b18e228c-2a43-4d46-8fd1-c5c477522ce5 admin admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 10:55:27.183946 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b18e228c-2a43-4d46-8fd1-c5c477522ce5 admin admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 10:55:27.193226 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1620d98d-1dcc-45fc-94f5-a360588d6626 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5ac47e3b09f43d19a23a0f457fe9e23', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 10:55:27.196088 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1620d98d-1dcc-45fc-94f5-a360588d6626 admin admin] REQUEST_METHOD: `POST` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:55:27.196656 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1620d98d-1dcc-45fc-94f5-a360588d6626 admin admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:55:27.196942 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1620d98d-1dcc-45fc-94f5-a360588d6626 admin admin] PATH_INFO: `/v3/projects` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:55:27.197614 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1620d98d-1dcc-45fc-94f5-a360588d6626 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 10:55:27.204368 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1620d98d-1dcc-45fc-94f5-a360588d6626 admin admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 10:55:27.296119 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 1016/2004] 10.222.0.25 () {60 vars in 1153 bytes} [Sun Aug 7 10:55:26 2022] POST /identity/v3/projects => generated 375 bytes in 333 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 07 10:55:27.299895 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adebc599-db53-4ff2-ba07-18c4848e7ac8 None None] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 10:55:27.313261 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adebc599-db53-4ff2-ba07-18c4848e7ac8 None None] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 10:55:27.429448 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8663de1d-64b6-425c-84bd-b56b7817901e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5ac47e3b09f43d19a23a0f457fe9e23', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 10:55:27.430110 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8663de1d-64b6-425c-84bd-b56b7817901e admin admin] REQUEST_METHOD: `POST` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:55:27.430379 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8663de1d-64b6-425c-84bd-b56b7817901e admin admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:55:27.430654 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8663de1d-64b6-425c-84bd-b56b7817901e admin admin] PATH_INFO: `/v3/users` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:55:27.431359 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8663de1d-64b6-425c-84bd-b56b7817901e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 10:55:27.434254 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8663de1d-64b6-425c-84bd-b56b7817901e admin admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 10:55:27.471256 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 989/2005] 10.222.0.25 () {60 vars in 1144 bytes} [Sun Aug 7 10:55:27 2022] POST /identity/v3/users => generated 294 bytes in 166 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 07 10:55:27.471485 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1620d98d-1dcc-45fc-94f5-a360588d6626 admin admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 10:55:27.485659 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1620d98d-1dcc-45fc-94f5-a360588d6626 admin admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 10:55:27.501565 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6713ccdf-b365-4d30-a77a-1d2a771e6617 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5ac47e3b09f43d19a23a0f457fe9e23', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 10:55:27.502314 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6713ccdf-b365-4d30-a77a-1d2a771e6617 admin admin] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:55:27.502653 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6713ccdf-b365-4d30-a77a-1d2a771e6617 admin admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:55:27.503046 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6713ccdf-b365-4d30-a77a-1d2a771e6617 admin admin] PATH_INFO: `/v3/roles` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:55:27.503769 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6713ccdf-b365-4d30-a77a-1d2a771e6617 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 10:55:27.505427 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6713ccdf-b365-4d30-a77a-1d2a771e6617 admin admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 10:55:27.526454 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 1017/2006] 10.222.0.25 () {58 vars in 1122 bytes} [Sun Aug 7 10:55:27 2022] GET /identity/v3/roles => generated 1420 bytes in 61 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 07 10:55:27.531149 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8663de1d-64b6-425c-84bd-b56b7817901e admin admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 10:55:27.544648 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8663de1d-64b6-425c-84bd-b56b7817901e admin admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 10:55:27.557363 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d34e4e56-2940-44e7-a6e6-e0fc71ab1fe8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5ac47e3b09f43d19a23a0f457fe9e23', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 10:55:27.558064 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d34e4e56-2940-44e7-a6e6-e0fc71ab1fe8 admin admin] REQUEST_METHOD: `PUT` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:55:27.558351 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d34e4e56-2940-44e7-a6e6-e0fc71ab1fe8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:55:27.558629 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d34e4e56-2940-44e7-a6e6-e0fc71ab1fe8 admin admin] PATH_INFO: `/v3/projects/9dfd360b7dab498cb9eafa84b89dd0b2/users/af1f1b4994b64444aa1bc10f17d681fd/roles/81b1f1bcee2a473088fbbd327c1c5b23` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:55:27.591193 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d34e4e56-2940-44e7-a6e6-e0fc71ab1fe8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9dfd360b7dab498cb9eafa84b89dd0b2, user_id=af1f1b4994b64444aa1bc10f17d681fd, role_id=81b1f1bcee2a473088fbbd327c1c5b23)` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 10:55:27.594557 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d34e4e56-2940-44e7-a6e6-e0fc71ab1fe8 admin admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 10:55:27.615640 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 990/2007] 10.222.0.25 () {60 vars in 1483 bytes} [Sun Aug 7 10:55:27 2022] PUT /identity/v3/projects/9dfd360b7dab498cb9eafa84b89dd0b2/users/af1f1b4994b64444aa1bc10f17d681fd/roles/81b1f1bcee2a473088fbbd327c1c5b23 => generated 0 bytes in 86 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 07 10:55:27.619801 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6713ccdf-b365-4d30-a77a-1d2a771e6617 admin admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 10:55:27.633865 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6713ccdf-b365-4d30-a77a-1d2a771e6617 admin admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 10:55:27.707592 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e96680c5-0845-49f8-a9a2-55182f6346f0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5ac47e3b09f43d19a23a0f457fe9e23', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 10:55:27.708239 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e96680c5-0845-49f8-a9a2-55182f6346f0 admin admin] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:55:27.708533 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e96680c5-0845-49f8-a9a2-55182f6346f0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:55:27.708860 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e96680c5-0845-49f8-a9a2-55182f6346f0 admin admin] PATH_INFO: `/v3/roles` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:55:27.709517 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e96680c5-0845-49f8-a9a2-55182f6346f0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 10:55:27.711364 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e96680c5-0845-49f8-a9a2-55182f6346f0 admin admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 10:55:27.723151 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 1018/2008] 10.222.0.25 () {58 vars in 1122 bytes} [Sun Aug 7 10:55:27 2022] GET /identity/v3/roles => generated 1420 bytes in 104 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 07 10:55:27.727333 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d34e4e56-2940-44e7-a6e6-e0fc71ab1fe8 admin admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 10:55:27.740029 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d34e4e56-2940-44e7-a6e6-e0fc71ab1fe8 admin admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 10:55:27.745569 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5281f3ff-1f30-450f-a09a-37ca87f47ce6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5ac47e3b09f43d19a23a0f457fe9e23', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 10:55:27.746731 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5281f3ff-1f30-450f-a09a-37ca87f47ce6 admin admin] REQUEST_METHOD: `PUT` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:55:27.746731 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5281f3ff-1f30-450f-a09a-37ca87f47ce6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:55:27.746950 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5281f3ff-1f30-450f-a09a-37ca87f47ce6 admin admin] PATH_INFO: `/v3/projects/9dfd360b7dab498cb9eafa84b89dd0b2/users/af1f1b4994b64444aa1bc10f17d681fd/roles/81b1f1bcee2a473088fbbd327c1c5b23` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:55:27.749492 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5281f3ff-1f30-450f-a09a-37ca87f47ce6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9dfd360b7dab498cb9eafa84b89dd0b2, user_id=af1f1b4994b64444aa1bc10f17d681fd, role_id=81b1f1bcee2a473088fbbd327c1c5b23)` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 10:55:27.752261 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5281f3ff-1f30-450f-a09a-37ca87f47ce6 admin admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 10:55:27.766718 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 991/2009] 10.222.0.25 () {60 vars in 1483 bytes} [Sun Aug 7 10:55:27 2022] PUT /identity/v3/projects/9dfd360b7dab498cb9eafa84b89dd0b2/users/af1f1b4994b64444aa1bc10f17d681fd/roles/81b1f1bcee2a473088fbbd327c1c5b23 => generated 0 bytes in 40 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 07 10:55:27.776806 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b744f14c-92dc-42c0-865b-8ee310343d89 None None] REQUEST_METHOD: `POST` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:55:27.777257 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b744f14c-92dc-42c0-865b-8ee310343d89 None None] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:55:27.777631 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b744f14c-92dc-42c0-865b-8ee310343d89 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:55:27.814720 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.auth.core [None req-b744f14c-92dc-42c0-865b-8ee310343d89 None None] MFA Rules not processed for user `af1f1b4994b64444aa1bc10f17d681fd`. Rule list: `[]` (Enabled: `True`). {{(pid=80144) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 10:55:27.817202 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.fernet_utils [None req-b744f14c-92dc-42c0-865b-8ee310343d89 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80144) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 10:55:27.888330 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 10:55:27.926201 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 1019/2010] 10.222.0.25 () {58 vars in 958 bytes} [Sun Aug 7 10:55:27 2022] POST /identity/v3/auth/tokens => generated 3319 bytes in 150 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 07 10:55:27.938220 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5281f3ff-1f30-450f-a09a-37ca87f47ce6 admin admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 10:55:27.942195 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-5281f3ff-1f30-450f-a09a-37ca87f47ce6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 10:55:28.011146 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 10:55:28.053090 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5281f3ff-1f30-450f-a09a-37ca87f47ce6 admin admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 10:55:28.073993 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72ebcaca-77fb-4ba2-b0da-73dafa300a7b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88dbd6dfd1164994a3e16da553561910', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6de1f3cac02c4bcdb05dd2337176033d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 10:55:28.073993 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72ebcaca-77fb-4ba2-b0da-73dafa300a7b service cinder] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:55:28.073993 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72ebcaca-77fb-4ba2-b0da-73dafa300a7b service cinder] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:55:28.073993 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72ebcaca-77fb-4ba2-b0da-73dafa300a7b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:55:28.141534 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72ebcaca-77fb-4ba2-b0da-73dafa300a7b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 10:55:28.143227 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72ebcaca-77fb-4ba2-b0da-73dafa300a7b service cinder] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 10:55:28.153797 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 992/2011] 10.222.0.25 () {60 vars in 1365 bytes} [Sun Aug 7 10:55:27 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 217 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 07 10:55:28.253890 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeeed1a3-edeb-4eb5-a30e-f19ac313297f None None] REQUEST_METHOD: `POST` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:55:28.254506 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeeed1a3-edeb-4eb5-a30e-f19ac313297f None None] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:55:28.254612 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeeed1a3-edeb-4eb5-a30e-f19ac313297f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:55:28.289670 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.auth.core [None req-aeeed1a3-edeb-4eb5-a30e-f19ac313297f None None] MFA Rules not processed for user `ee9ca3b9c7284fd1a600cbd3add41f56`. Rule list: `[]` (Enabled: `True`). {{(pid=80144) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 10:55:28.295790 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.fernet_utils [None req-aeeed1a3-edeb-4eb5-a30e-f19ac313297f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80144) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 10:55:28.413012 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 1020/2012] 10.222.0.25 () {58 vars in 958 bytes} [Sun Aug 7 10:55:28 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 160 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 07 10:55:38.149105 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=32 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 10:55:38.386013 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=31 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 10:55:40.401478 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72ebcaca-77fb-4ba2-b0da-73dafa300a7b service cinder] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 10:55:40.505844 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72ebcaca-77fb-4ba2-b0da-73dafa300a7b service cinder] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 10:55:40.510955 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d17c0afb-9f7a-4503-983a-5ef3995bc10a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5ac47e3b09f43d19a23a0f457fe9e23', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 10:55:40.512143 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d17c0afb-9f7a-4503-983a-5ef3995bc10a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:55:40.512143 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d17c0afb-9f7a-4503-983a-5ef3995bc10a admin admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:55:40.512630 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d17c0afb-9f7a-4503-983a-5ef3995bc10a admin admin] PATH_INFO: `/v3/users/36fb5eeaf0184da5b86486d8320cfa14` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:55:40.513835 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d17c0afb-9f7a-4503-983a-5ef3995bc10a admin admin] RBAC: Authorizing `identity:delete_user(user_id=36fb5eeaf0184da5b86486d8320cfa14)` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 10:55:40.516127 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d17c0afb-9f7a-4503-983a-5ef3995bc10a admin admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 10:55:40.586017 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-d17c0afb-9f7a-4503-983a-5ef3995bc10a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '36fb5eeaf0184da5b86486d8320cfa14'} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 10:55:40.595240 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-d17c0afb-9f7a-4503-983a-5ef3995bc10a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '36fb5eeaf0184da5b86486d8320cfa14'} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 10:55:40.609487 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-d17c0afb-9f7a-4503-983a-5ef3995bc10a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '36fb5eeaf0184da5b86486d8320cfa14'} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 10:55:40.610223 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-d17c0afb-9f7a-4503-983a-5ef3995bc10a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '36fb5eeaf0184da5b86486d8320cfa14'} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 10:55:40.617664 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-d17c0afb-9f7a-4503-983a-5ef3995bc10a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '36fb5eeaf0184da5b86486d8320cfa14'} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 10:55:40.624337 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 993/2013] 10.222.0.25 () {58 vars in 1224 bytes} [Sun Aug 7 10:55:40 2022] DELETE /identity/v3/users/36fb5eeaf0184da5b86486d8320cfa14 => generated 0 bytes in 223 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 07 10:55:40.628722 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b335431a-0795-4ac1-8784-cfab4fcb319f None None] REQUEST_METHOD: `POST` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:55:40.629229 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b335431a-0795-4ac1-8784-cfab4fcb319f None None] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:55:40.629347 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b335431a-0795-4ac1-8784-cfab4fcb319f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:55:40.648416 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.auth.core [None req-b335431a-0795-4ac1-8784-cfab4fcb319f None None] MFA Rules not processed for user `bac4e6b76fe64217ac8fa44e71cffe6a`. Rule list: `[]` (Enabled: `True`). {{(pid=80144) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 10:55:40.649819 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.fernet_utils [None req-b335431a-0795-4ac1-8784-cfab4fcb319f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80144) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 10:55:40.709690 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 1021/2014] 10.222.0.25 () {58 vars in 958 bytes} [Sun Aug 7 10:55:40 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 07 10:55:40.721196 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d17c0afb-9f7a-4503-983a-5ef3995bc10a admin admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 10:55:40.722823 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-d17c0afb-9f7a-4503-983a-5ef3995bc10a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 10:55:40.786868 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d17c0afb-9f7a-4503-983a-5ef3995bc10a admin admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 10:55:40.790442 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cd3f598-6d97-4c20-949e-8b40c002d5be service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be1283c9bf47424c9aa3ad27cab03027', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6de1f3cac02c4bcdb05dd2337176033d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 10:55:40.791119 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cd3f598-6d97-4c20-949e-8b40c002d5be service neutron] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:55:40.791400 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cd3f598-6d97-4c20-949e-8b40c002d5be service neutron] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:55:40.791677 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cd3f598-6d97-4c20-949e-8b40c002d5be service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:55:40.802192 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cd3f598-6d97-4c20-949e-8b40c002d5be service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 10:55:40.803653 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cd3f598-6d97-4c20-949e-8b40c002d5be service neutron] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 10:55:40.813892 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 994/2015] 10.222.0.25 () {60 vars in 1365 bytes} [Sun Aug 7 10:55:40 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 07 10:55:41.084074 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b335431a-0795-4ac1-8784-cfab4fcb319f None None] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 10:55:41.085633 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.fernet_utils [None req-b335431a-0795-4ac1-8784-cfab4fcb319f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80144) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 10:55:41.101956 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b335431a-0795-4ac1-8784-cfab4fcb319f None None] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 10:55:41.109439 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20e24866-9269-41be-92b7-8b313249ff60 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5ac47e3b09f43d19a23a0f457fe9e23', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 10:55:41.110339 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20e24866-9269-41be-92b7-8b313249ff60 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:55:41.110756 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20e24866-9269-41be-92b7-8b313249ff60 admin admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:55:41.111062 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20e24866-9269-41be-92b7-8b313249ff60 admin admin] PATH_INFO: `/v3/users/05a0d408980b49acafd2c9a93bf527f2` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:55:41.112354 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20e24866-9269-41be-92b7-8b313249ff60 admin admin] RBAC: Authorizing `identity:delete_user(user_id=05a0d408980b49acafd2c9a93bf527f2)` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 10:55:41.114453 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20e24866-9269-41be-92b7-8b313249ff60 admin admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 10:55:41.125545 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cd3f598-6d97-4c20-949e-8b40c002d5be service neutron] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 10:55:41.127827 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-0cd3f598-6d97-4c20-949e-8b40c002d5be service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 10:55:41.150401 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cd3f598-6d97-4c20-949e-8b40c002d5be service neutron] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 10:55:41.155576 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fde34c2c-5a87-4a22-8474-e4c60a705ca2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5ac47e3b09f43d19a23a0f457fe9e23', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 10:55:41.156318 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fde34c2c-5a87-4a22-8474-e4c60a705ca2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:55:41.156587 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fde34c2c-5a87-4a22-8474-e4c60a705ca2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:55:41.156972 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fde34c2c-5a87-4a22-8474-e4c60a705ca2 admin admin] PATH_INFO: `/v3/projects/bfcf7aebfb9e4963bc55842c72ed5f41` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:55:41.158261 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fde34c2c-5a87-4a22-8474-e4c60a705ca2 admin admin] RBAC: Authorizing `identity:delete_project(project_id=bfcf7aebfb9e4963bc55842c72ed5f41)` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 10:55:41.160092 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fde34c2c-5a87-4a22-8474-e4c60a705ca2 admin admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 10:55:41.172055 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.notifications [None req-20e24866-9269-41be-92b7-8b313249ff60 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '05a0d408980b49acafd2c9a93bf527f2'} {{(pid=80144) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 10:55:41.179150 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.notifications [None req-20e24866-9269-41be-92b7-8b313249ff60 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '05a0d408980b49acafd2c9a93bf527f2'} {{(pid=80144) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 10:55:41.179786 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 10:55:41.189266 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.notifications [None req-20e24866-9269-41be-92b7-8b313249ff60 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '05a0d408980b49acafd2c9a93bf527f2'} {{(pid=80144) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 10:55:41.189931 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.notifications [None req-20e24866-9269-41be-92b7-8b313249ff60 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '05a0d408980b49acafd2c9a93bf527f2'} {{(pid=80144) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 10:55:41.190800 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.notifications [None req-20e24866-9269-41be-92b7-8b313249ff60 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '05a0d408980b49acafd2c9a93bf527f2'} {{(pid=80144) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 10:55:41.210283 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 1022/2016] 10.222.0.25 () {58 vars in 1224 bytes} [Sun Aug 7 10:55:41 2022] DELETE /identity/v3/users/05a0d408980b49acafd2c9a93bf527f2 => generated 0 bytes in 127 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 07 10:55:41.214386 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d486d687-9162-4ddc-9a31-6f74b4508640 None None] REQUEST_METHOD: `POST` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:55:41.214741 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d486d687-9162-4ddc-9a31-6f74b4508640 None None] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:55:41.215046 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d486d687-9162-4ddc-9a31-6f74b4508640 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:55:41.215166 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-fde34c2c-5a87-4a22-8474-e4c60a705ca2 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'bfcf7aebfb9e4963bc55842c72ed5f41', 'request_id': 'req-fde34c2c-5a87-4a22-8474-e4c60a705ca2'} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 10:55:41.220797 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-fde34c2c-5a87-4a22-8474-e4c60a705ca2 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'bfcf7aebfb9e4963bc55842c72ed5f41', 'request_id': 'req-fde34c2c-5a87-4a22-8474-e4c60a705ca2'} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 10:55:41.227157 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-fde34c2c-5a87-4a22-8474-e4c60a705ca2 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'bfcf7aebfb9e4963bc55842c72ed5f41', 'request_id': 'req-fde34c2c-5a87-4a22-8474-e4c60a705ca2'} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 10:55:41.236136 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.auth.core [None req-d486d687-9162-4ddc-9a31-6f74b4508640 None None] MFA Rules not processed for user `bac4e6b76fe64217ac8fa44e71cffe6a`. Rule list: `[]` (Enabled: `True`). {{(pid=80144) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 10:55:41.238132 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.fernet_utils [None req-d486d687-9162-4ddc-9a31-6f74b4508640 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80144) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 10:55:41.241449 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-fde34c2c-5a87-4a22-8474-e4c60a705ca2 admin admin] The token cache is being invalidate because project bfcf7aebfb9e4963bc55842c72ed5f41 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80143) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 07 10:55:41.241746 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-fde34c2c-5a87-4a22-8474-e4c60a705ca2 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 10:55:41.242448 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-fde34c2c-5a87-4a22-8474-e4c60a705ca2 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 10:55:41.244040 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 995/2017] 10.222.0.25 () {58 vars in 1233 bytes} [Sun Aug 7 10:55:41 2022] DELETE /identity/v3/projects/bfcf7aebfb9e4963bc55842c72ed5f41 => generated 0 bytes in 120 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 07 10:55:41.247991 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fde34c2c-5a87-4a22-8474-e4c60a705ca2 admin admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 10:55:41.249846 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-fde34c2c-5a87-4a22-8474-e4c60a705ca2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 10:55:41.298543 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 1023/2018] 10.222.0.25 () {58 vars in 958 bytes} [Sun Aug 7 10:55:41 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 07 10:55:41.308049 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d486d687-9162-4ddc-9a31-6f74b4508640 None None] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 10:55:41.309489 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.fernet_utils [None req-d486d687-9162-4ddc-9a31-6f74b4508640 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80144) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 10:55:41.318575 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fde34c2c-5a87-4a22-8474-e4c60a705ca2 admin admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 10:55:41.324109 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 10:55:41.325004 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93c72e1c-8a4d-46c0-aeb5-2ecaafe27e32 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5ac47e3b09f43d19a23a0f457fe9e23', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 10:55:41.325645 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93c72e1c-8a4d-46c0-aeb5-2ecaafe27e32 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:55:41.325925 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93c72e1c-8a4d-46c0-aeb5-2ecaafe27e32 admin admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:55:41.327573 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93c72e1c-8a4d-46c0-aeb5-2ecaafe27e32 admin admin] PATH_INFO: `/v3/users/af1f1b4994b64444aa1bc10f17d681fd` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:55:41.328383 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93c72e1c-8a4d-46c0-aeb5-2ecaafe27e32 admin admin] RBAC: Authorizing `identity:delete_user(user_id=af1f1b4994b64444aa1bc10f17d681fd)` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 10:55:41.330476 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93c72e1c-8a4d-46c0-aeb5-2ecaafe27e32 admin admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 10:55:41.383114 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d486d687-9162-4ddc-9a31-6f74b4508640 None None] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 10:55:41.388072 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adfe358f-432e-43c6-affb-8b7761087db8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be1283c9bf47424c9aa3ad27cab03027', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6de1f3cac02c4bcdb05dd2337176033d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 10:55:41.388711 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adfe358f-432e-43c6-affb-8b7761087db8 service neutron] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:55:41.389262 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adfe358f-432e-43c6-affb-8b7761087db8 service neutron] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:55:41.389589 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adfe358f-432e-43c6-affb-8b7761087db8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:55:41.390745 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-93c72e1c-8a4d-46c0-aeb5-2ecaafe27e32 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'af1f1b4994b64444aa1bc10f17d681fd'} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 10:55:41.401991 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-93c72e1c-8a4d-46c0-aeb5-2ecaafe27e32 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'af1f1b4994b64444aa1bc10f17d681fd'} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 10:55:41.407993 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adfe358f-432e-43c6-affb-8b7761087db8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 10:55:41.409893 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adfe358f-432e-43c6-affb-8b7761087db8 service neutron] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 10:55:41.418383 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-93c72e1c-8a4d-46c0-aeb5-2ecaafe27e32 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'af1f1b4994b64444aa1bc10f17d681fd'} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 10:55:41.419212 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-93c72e1c-8a4d-46c0-aeb5-2ecaafe27e32 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'af1f1b4994b64444aa1bc10f17d681fd'} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 10:55:41.429182 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 1024/2019] 10.222.0.25 () {60 vars in 1365 bytes} [Sun Aug 7 10:55:41 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 122 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 07 10:55:41.431321 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-93c72e1c-8a4d-46c0-aeb5-2ecaafe27e32 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'af1f1b4994b64444aa1bc10f17d681fd'} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 10:55:41.449853 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 996/2020] 10.222.0.25 () {58 vars in 1224 bytes} [Sun Aug 7 10:55:41 2022] DELETE /identity/v3/users/af1f1b4994b64444aa1bc10f17d681fd => generated 0 bytes in 199 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 07 10:55:41.793283 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adfe358f-432e-43c6-affb-8b7761087db8 service neutron] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 10:55:41.818688 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adfe358f-432e-43c6-affb-8b7761087db8 service neutron] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 10:55:41.899188 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaad64c8-62a9-4efe-980a-444a4d1ca58a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5ac47e3b09f43d19a23a0f457fe9e23', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 10:55:41.900335 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaad64c8-62a9-4efe-980a-444a4d1ca58a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:55:41.900844 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaad64c8-62a9-4efe-980a-444a4d1ca58a admin admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:55:41.901311 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaad64c8-62a9-4efe-980a-444a4d1ca58a admin admin] PATH_INFO: `/v3/projects/9dfd360b7dab498cb9eafa84b89dd0b2` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:55:41.903879 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaad64c8-62a9-4efe-980a-444a4d1ca58a admin admin] RBAC: Authorizing `identity:delete_project(project_id=9dfd360b7dab498cb9eafa84b89dd0b2)` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 10:55:41.906675 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaad64c8-62a9-4efe-980a-444a4d1ca58a admin admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 10:55:41.969072 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.notifications [None req-aaad64c8-62a9-4efe-980a-444a4d1ca58a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9dfd360b7dab498cb9eafa84b89dd0b2', 'request_id': 'req-aaad64c8-62a9-4efe-980a-444a4d1ca58a'} {{(pid=80144) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 10:55:41.974690 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.notifications [None req-aaad64c8-62a9-4efe-980a-444a4d1ca58a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9dfd360b7dab498cb9eafa84b89dd0b2', 'request_id': 'req-aaad64c8-62a9-4efe-980a-444a4d1ca58a'} {{(pid=80144) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 10:55:41.984911 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.notifications [None req-aaad64c8-62a9-4efe-980a-444a4d1ca58a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9dfd360b7dab498cb9eafa84b89dd0b2', 'request_id': 'req-aaad64c8-62a9-4efe-980a-444a4d1ca58a'} {{(pid=80144) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 10:55:42.001343 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.notifications [None req-aaad64c8-62a9-4efe-980a-444a4d1ca58a admin admin] The token cache is being invalidate because project 9dfd360b7dab498cb9eafa84b89dd0b2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80144) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 07 10:55:42.001838 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.notifications [None req-aaad64c8-62a9-4efe-980a-444a4d1ca58a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80144) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 10:55:42.002759 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.notifications [None req-aaad64c8-62a9-4efe-980a-444a4d1ca58a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80144) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 10:55:42.007080 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 1025/2021] 10.222.0.25 () {58 vars in 1233 bytes} [Sun Aug 7 10:55:41 2022] DELETE /identity/v3/projects/9dfd360b7dab498cb9eafa84b89dd0b2 => generated 0 bytes in 212 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 07 10:55:42.010700 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acee0042-6981-44f3-98cd-c83fb560503e None None] REQUEST_METHOD: `POST` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:55:42.011050 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acee0042-6981-44f3-98cd-c83fb560503e None None] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:55:42.011337 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acee0042-6981-44f3-98cd-c83fb560503e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:55:42.032731 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.auth.core [None req-acee0042-6981-44f3-98cd-c83fb560503e None None] MFA Rules not processed for user `bac4e6b76fe64217ac8fa44e71cffe6a`. Rule list: `[]` (Enabled: `True`). {{(pid=80143) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 10:55:42.034048 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-acee0042-6981-44f3-98cd-c83fb560503e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 10:55:42.059711 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 10:55:42.105089 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 997/2022] 10.222.0.25 () {58 vars in 958 bytes} [Sun Aug 7 10:55:42 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 07 10:55:42.113599 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-704d53b6-10b7-4a9e-a46e-8b32172fd2f7 None None] REQUEST_METHOD: `POST` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:55:42.113918 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-704d53b6-10b7-4a9e-a46e-8b32172fd2f7 None None] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:55:42.114159 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-704d53b6-10b7-4a9e-a46e-8b32172fd2f7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:55:42.131497 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acee0042-6981-44f3-98cd-c83fb560503e None None] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 10:55:42.133257 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-acee0042-6981-44f3-98cd-c83fb560503e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 10:55:42.136616 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.auth.core [None req-704d53b6-10b7-4a9e-a46e-8b32172fd2f7 None None] MFA Rules not processed for user `bac4e6b76fe64217ac8fa44e71cffe6a`. Rule list: `[]` (Enabled: `True`). {{(pid=80144) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 10:55:42.138094 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.fernet_utils [None req-704d53b6-10b7-4a9e-a46e-8b32172fd2f7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80144) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 10:55:42.148515 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 1026/2023] 10.222.0.25 () {58 vars in 958 bytes} [Sun Aug 7 10:55:42 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 07 10:55:42.151459 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acee0042-6981-44f3-98cd-c83fb560503e None None] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 10:55:42.152865 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-704d53b6-10b7-4a9e-a46e-8b32172fd2f7 None None] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 10:55:42.158080 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-009452d2-14f7-4236-99e1-904b16c7011d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5ac47e3b09f43d19a23a0f457fe9e23', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 10:55:42.159030 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-009452d2-14f7-4236-99e1-904b16c7011d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:55:42.159377 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-009452d2-14f7-4236-99e1-904b16c7011d admin admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:55:42.159824 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-009452d2-14f7-4236-99e1-904b16c7011d admin admin] PATH_INFO: `/v3/projects/23e2b09610fd4ee0af16405257b2daf3` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:55:42.161042 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-009452d2-14f7-4236-99e1-904b16c7011d admin admin] RBAC: Authorizing `identity:delete_project(project_id=23e2b09610fd4ee0af16405257b2daf3)` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 10:55:42.162996 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-009452d2-14f7-4236-99e1-904b16c7011d admin admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 10:55:42.164387 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-704d53b6-10b7-4a9e-a46e-8b32172fd2f7 None None] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 10:55:42.171316 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d72bee3c-6e6b-4cc4-af12-936762677a45 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5ac47e3b09f43d19a23a0f457fe9e23', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 10:55:42.172028 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d72bee3c-6e6b-4cc4-af12-936762677a45 admin admin] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:55:42.172404 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d72bee3c-6e6b-4cc4-af12-936762677a45 admin admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:55:42.172777 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d72bee3c-6e6b-4cc4-af12-936762677a45 admin admin] PATH_INFO: `/v3/domains` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:55:42.173502 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d72bee3c-6e6b-4cc4-af12-936762677a45 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80144) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 10:55:42.173868 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d72bee3c-6e6b-4cc4-af12-936762677a45 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 10:55:42.175471 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d72bee3c-6e6b-4cc4-af12-936762677a45 admin admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 10:55:42.187249 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 1027/2024] 10.222.0.25 () {58 vars in 1153 bytes} [Sun Aug 7 10:55:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 07 10:55:42.191549 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d72bee3c-6e6b-4cc4-af12-936762677a45 admin admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 10:55:42.201741 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d72bee3c-6e6b-4cc4-af12-936762677a45 admin admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 10:55:42.221935 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-009452d2-14f7-4236-99e1-904b16c7011d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '23e2b09610fd4ee0af16405257b2daf3', 'request_id': 'req-009452d2-14f7-4236-99e1-904b16c7011d'} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 10:55:42.227916 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-009452d2-14f7-4236-99e1-904b16c7011d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '23e2b09610fd4ee0af16405257b2daf3', 'request_id': 'req-009452d2-14f7-4236-99e1-904b16c7011d'} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 10:55:42.228594 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 10:55:42.236484 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-009452d2-14f7-4236-99e1-904b16c7011d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '23e2b09610fd4ee0af16405257b2daf3', 'request_id': 'req-009452d2-14f7-4236-99e1-904b16c7011d'} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 10:55:42.250414 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-009452d2-14f7-4236-99e1-904b16c7011d admin admin] The token cache is being invalidate because project 23e2b09610fd4ee0af16405257b2daf3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80143) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 07 10:55:42.250947 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-009452d2-14f7-4236-99e1-904b16c7011d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 10:55:42.251601 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-009452d2-14f7-4236-99e1-904b16c7011d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 10:55:42.253336 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 998/2025] 10.222.0.25 () {58 vars in 1233 bytes} [Sun Aug 7 10:55:42 2022] DELETE /identity/v3/projects/23e2b09610fd4ee0af16405257b2daf3 => generated 0 bytes in 122 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 07 10:55:42.257781 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59d81e32-cf56-4539-8006-75431954a3c2 None None] REQUEST_METHOD: `POST` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:55:42.258070 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59d81e32-cf56-4539-8006-75431954a3c2 None None] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:55:42.258356 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59d81e32-cf56-4539-8006-75431954a3c2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:55:42.266571 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac2c24d4-fb77-4ee3-a207-68d6915b3df6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5ac47e3b09f43d19a23a0f457fe9e23', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 10:55:42.267311 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac2c24d4-fb77-4ee3-a207-68d6915b3df6 admin admin] REQUEST_METHOD: `POST` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:55:42.267598 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac2c24d4-fb77-4ee3-a207-68d6915b3df6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:55:42.268163 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac2c24d4-fb77-4ee3-a207-68d6915b3df6 admin admin] PATH_INFO: `/v3/projects` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:55:42.268550 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac2c24d4-fb77-4ee3-a207-68d6915b3df6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 10:55:42.270614 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac2c24d4-fb77-4ee3-a207-68d6915b3df6 admin admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 10:55:42.278990 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.auth.core [None req-59d81e32-cf56-4539-8006-75431954a3c2 None None] MFA Rules not processed for user `bac4e6b76fe64217ac8fa44e71cffe6a`. Rule list: `[]` (Enabled: `True`). {{(pid=80143) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 10:55:42.280866 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-59d81e32-cf56-4539-8006-75431954a3c2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 10:55:42.297360 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 999/2026] 10.222.0.25 () {58 vars in 958 bytes} [Sun Aug 7 10:55:42 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 40 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 07 10:55:42.298899 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d8fc7ed-9369-4c93-abac-baa9fd7b1d28 None None] REQUEST_METHOD: `POST` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:55:42.299491 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d8fc7ed-9369-4c93-abac-baa9fd7b1d28 None None] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:55:42.299720 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d8fc7ed-9369-4c93-abac-baa9fd7b1d28 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:55:42.328647 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 1028/2027] 10.222.0.25 () {60 vars in 1153 bytes} [Sun Aug 7 10:55:42 2022] POST /identity/v3/projects => generated 367 bytes in 137 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 07 10:55:42.329808 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac2c24d4-fb77-4ee3-a207-68d6915b3df6 admin admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 10:55:42.334495 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.auth.core [None req-5d8fc7ed-9369-4c93-abac-baa9fd7b1d28 None None] MFA Rules not processed for user `bac4e6b76fe64217ac8fa44e71cffe6a`. Rule list: `[]` (Enabled: `True`). {{(pid=80143) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 10:55:42.334758 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.fernet_utils [None req-ac2c24d4-fb77-4ee3-a207-68d6915b3df6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80144) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 10:55:42.337304 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-5d8fc7ed-9369-4c93-abac-baa9fd7b1d28 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 10:55:42.422793 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac2c24d4-fb77-4ee3-a207-68d6915b3df6 admin admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 10:55:42.427640 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-893d8243-aaf1-4228-8453-673616a7a6b8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5ac47e3b09f43d19a23a0f457fe9e23', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 10:55:42.428489 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-893d8243-aaf1-4228-8453-673616a7a6b8 admin admin] REQUEST_METHOD: `POST` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:55:42.428774 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-893d8243-aaf1-4228-8453-673616a7a6b8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:55:42.429054 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-893d8243-aaf1-4228-8453-673616a7a6b8 admin admin] PATH_INFO: `/v3/users` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:55:42.429685 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-893d8243-aaf1-4228-8453-673616a7a6b8 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 10:55:42.433993 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1000/2028] 10.222.0.25 () {58 vars in 958 bytes} [Sun Aug 7 10:55:42 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 07 10:55:42.434496 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-893d8243-aaf1-4228-8453-673616a7a6b8 admin admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 10:55:42.434593 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc5d98b1-5139-49c5-8a4c-e73f25abc854 None None] REQUEST_METHOD: `POST` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:55:42.434593 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc5d98b1-5139-49c5-8a4c-e73f25abc854 None None] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:55:42.434593 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc5d98b1-5139-49c5-8a4c-e73f25abc854 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:55:42.458955 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.auth.core [None req-bc5d98b1-5139-49c5-8a4c-e73f25abc854 None None] MFA Rules not processed for user `de1d6c24b1a14d10a20d82a4c65793d8`. Rule list: `[]` (Enabled: `True`). {{(pid=80143) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 10:55:42.460368 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-bc5d98b1-5139-49c5-8a4c-e73f25abc854 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 10:55:42.464623 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 1029/2029] 10.222.0.25 () {60 vars in 1144 bytes} [Sun Aug 7 10:55:42 2022] POST /identity/v3/users => generated 284 bytes in 132 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 07 10:55:42.464623 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-893d8243-aaf1-4228-8453-673616a7a6b8 admin admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 10:55:42.476743 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-893d8243-aaf1-4228-8453-673616a7a6b8 admin admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 10:55:42.481920 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08840033-e67b-4a79-b185-af5800ef2206 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5ac47e3b09f43d19a23a0f457fe9e23', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 10:55:42.482552 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08840033-e67b-4a79-b185-af5800ef2206 admin admin] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:55:42.483280 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08840033-e67b-4a79-b185-af5800ef2206 admin admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:55:42.483562 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08840033-e67b-4a79-b185-af5800ef2206 admin admin] PATH_INFO: `/v3/domains` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:55:42.484216 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08840033-e67b-4a79-b185-af5800ef2206 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80144) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 10:55:42.484528 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08840033-e67b-4a79-b185-af5800ef2206 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 10:55:42.486227 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08840033-e67b-4a79-b185-af5800ef2206 admin admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 10:55:42.500006 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 1030/2030] 10.222.0.25 () {58 vars in 1153 bytes} [Sun Aug 7 10:55:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 10:55:42.501084 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08840033-e67b-4a79-b185-af5800ef2206 admin admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 10:55:42.513417 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 10:55:42.515154 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08840033-e67b-4a79-b185-af5800ef2206 admin admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 10:55:42.528568 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85552812-a9fc-4a68-bc3b-ec265a44a10d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5ac47e3b09f43d19a23a0f457fe9e23', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 10:55:42.528568 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85552812-a9fc-4a68-bc3b-ec265a44a10d admin admin] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:55:42.528568 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85552812-a9fc-4a68-bc3b-ec265a44a10d admin admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:55:42.528568 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85552812-a9fc-4a68-bc3b-ec265a44a10d admin admin] PATH_INFO: `/v3/roles` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:55:42.529017 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85552812-a9fc-4a68-bc3b-ec265a44a10d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 10:55:42.529017 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85552812-a9fc-4a68-bc3b-ec265a44a10d admin admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 10:55:42.538674 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1001/2031] 10.222.0.25 () {58 vars in 1009 bytes} [Sun Aug 7 10:55:42 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 07 10:55:42.539880 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 1031/2032] 10.222.0.25 () {58 vars in 1122 bytes} [Sun Aug 7 10:55:42 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 07 10:55:42.540384 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc5d98b1-5139-49c5-8a4c-e73f25abc854 None None] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 10:55:42.543779 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85552812-a9fc-4a68-bc3b-ec265a44a10d admin admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 10:55:42.554716 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc5d98b1-5139-49c5-8a4c-e73f25abc854 None None] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 10:55:42.560230 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df1e8a88-73a6-4809-82d4-e2e9803b1c3e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5ac47e3b09f43d19a23a0f457fe9e23', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 10:55:42.561157 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df1e8a88-73a6-4809-82d4-e2e9803b1c3e admin admin] REQUEST_METHOD: `POST` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:55:42.561603 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df1e8a88-73a6-4809-82d4-e2e9803b1c3e admin admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:55:42.562069 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df1e8a88-73a6-4809-82d4-e2e9803b1c3e admin admin] PATH_INFO: `/v3/projects` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:55:42.563323 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df1e8a88-73a6-4809-82d4-e2e9803b1c3e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 10:55:42.566288 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df1e8a88-73a6-4809-82d4-e2e9803b1c3e admin admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 10:55:42.567812 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85552812-a9fc-4a68-bc3b-ec265a44a10d admin admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 10:55:42.572898 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f420db5-6a81-4849-99e0-49d3ed79414f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5ac47e3b09f43d19a23a0f457fe9e23', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 10:55:42.573438 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f420db5-6a81-4849-99e0-49d3ed79414f admin admin] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:55:42.573722 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f420db5-6a81-4849-99e0-49d3ed79414f admin admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:55:42.573998 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f420db5-6a81-4849-99e0-49d3ed79414f admin admin] PATH_INFO: `/v3/roles` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:55:42.574960 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f420db5-6a81-4849-99e0-49d3ed79414f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 10:55:42.577423 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f420db5-6a81-4849-99e0-49d3ed79414f admin admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 10:55:42.588924 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 1032/2033] 10.222.0.25 () {58 vars in 1122 bytes} [Sun Aug 7 10:55:42 2022] GET /identity/v3/roles => generated 1420 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 07 10:55:42.589801 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f420db5-6a81-4849-99e0-49d3ed79414f admin admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 10:55:42.596695 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.fernet_utils [None req-0f420db5-6a81-4849-99e0-49d3ed79414f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80144) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 10:55:42.611551 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1002/2034] 10.222.0.25 () {60 vars in 1153 bytes} [Sun Aug 7 10:55:42 2022] POST /identity/v3/projects => generated 385 bytes in 72 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 07 10:55:42.612488 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df1e8a88-73a6-4809-82d4-e2e9803b1c3e admin admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 10:55:42.671960 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f420db5-6a81-4849-99e0-49d3ed79414f admin admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 10:55:42.677001 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 10:55:42.680418 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df1e8a88-73a6-4809-82d4-e2e9803b1c3e admin admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 10:55:42.685974 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bf47ca5-aed9-4e97-8401-8aa22a9bad12 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5ac47e3b09f43d19a23a0f457fe9e23', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 10:55:42.686499 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bf47ca5-aed9-4e97-8401-8aa22a9bad12 admin admin] REQUEST_METHOD: `PUT` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:55:42.686818 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bf47ca5-aed9-4e97-8401-8aa22a9bad12 admin admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:55:42.687153 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bf47ca5-aed9-4e97-8401-8aa22a9bad12 admin admin] PATH_INFO: `/v3/projects/384bd09b1fcb4b6d838381ab958f1de3/users/9e1e727405d74f82bf192e6154241811/roles/b98dfb3b9fee4644af2887a3ac3fd2e6` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:55:42.702689 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bf47ca5-aed9-4e97-8401-8aa22a9bad12 admin admin] RBAC: Authorizing `identity:create_grant(project_id=384bd09b1fcb4b6d838381ab958f1de3, user_id=9e1e727405d74f82bf192e6154241811, role_id=b98dfb3b9fee4644af2887a3ac3fd2e6)` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 10:55:42.705384 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bf47ca5-aed9-4e97-8401-8aa22a9bad12 admin admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 10:55:42.719100 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1003/2035] 10.222.0.25 () {60 vars in 1483 bytes} [Sun Aug 7 10:55:42 2022] PUT /identity/v3/projects/384bd09b1fcb4b6d838381ab958f1de3/users/9e1e727405d74f82bf192e6154241811/roles/b98dfb3b9fee4644af2887a3ac3fd2e6 => generated 0 bytes in 107 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 07 10:55:42.720438 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bf47ca5-aed9-4e97-8401-8aa22a9bad12 admin admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 10:55:42.743205 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04106de5-31c2-4ab2-8085-1d572a19fbc1 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69490ea2a9a340bf91aca38bf5655b45', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6de1f3cac02c4bcdb05dd2337176033d', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 10:55:42.744388 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04106de5-31c2-4ab2-8085-1d572a19fbc1 service swift] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:55:42.744679 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04106de5-31c2-4ab2-8085-1d572a19fbc1 service swift] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:55:42.744955 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04106de5-31c2-4ab2-8085-1d572a19fbc1 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:55:42.786183 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bf47ca5-aed9-4e97-8401-8aa22a9bad12 admin admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 10:55:42.791141 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfc6dfbe-453a-45e0-8879-46c437217b2e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5ac47e3b09f43d19a23a0f457fe9e23', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 10:55:42.791898 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfc6dfbe-453a-45e0-8879-46c437217b2e admin admin] REQUEST_METHOD: `POST` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:55:42.792178 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfc6dfbe-453a-45e0-8879-46c437217b2e admin admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:55:42.792451 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfc6dfbe-453a-45e0-8879-46c437217b2e admin admin] PATH_INFO: `/v3/users` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:55:42.793102 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfc6dfbe-453a-45e0-8879-46c437217b2e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 10:55:42.795044 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfc6dfbe-453a-45e0-8879-46c437217b2e admin admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 10:55:42.802343 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04106de5-31c2-4ab2-8085-1d572a19fbc1 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 10:55:42.803970 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04106de5-31c2-4ab2-8085-1d572a19fbc1 service swift] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 10:55:42.814192 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 1033/2036] 10.222.0.25 () {60 vars in 1384 bytes} [Sun Aug 7 10:55:42 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 225 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 07 10:55:42.815728 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b677b651-4f01-4b13-b689-59cda7edfc71 None None] REQUEST_METHOD: `POST` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:55:42.816473 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b677b651-4f01-4b13-b689-59cda7edfc71 None None] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:55:42.816880 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b677b651-4f01-4b13-b689-59cda7edfc71 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:55:42.821996 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1004/2037] 10.222.0.25 () {60 vars in 1144 bytes} [Sun Aug 7 10:55:42 2022] POST /identity/v3/users => generated 293 bytes in 99 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 07 10:55:42.836027 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfc6dfbe-453a-45e0-8879-46c437217b2e admin admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 10:55:42.854985 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfc6dfbe-453a-45e0-8879-46c437217b2e admin admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 10:55:42.862060 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a75a08b-369e-4119-a50c-803bc0bc9e62 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5ac47e3b09f43d19a23a0f457fe9e23', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 10:55:42.862836 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a75a08b-369e-4119-a50c-803bc0bc9e62 admin admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:55:42.863239 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a75a08b-369e-4119-a50c-803bc0bc9e62 admin admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:55:42.863736 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a75a08b-369e-4119-a50c-803bc0bc9e62 admin admin] PATH_INFO: `/v3/roles` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:55:42.865079 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a75a08b-369e-4119-a50c-803bc0bc9e62 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 10:55:42.871711 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.auth.core [None req-b677b651-4f01-4b13-b689-59cda7edfc71 None None] MFA Rules not processed for user `9e1e727405d74f82bf192e6154241811`. Rule list: `[]` (Enabled: `True`). {{(pid=80144) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 10:55:42.873589 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.fernet_utils [None req-b677b651-4f01-4b13-b689-59cda7edfc71 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80144) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 10:55:42.877338 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a75a08b-369e-4119-a50c-803bc0bc9e62 admin admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 10:55:42.911770 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1005/2038] 10.222.0.25 () {58 vars in 1122 bytes} [Sun Aug 7 10:55:42 2022] GET /identity/v3/roles => generated 1420 bytes in 80 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 07 10:55:42.916553 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a75a08b-369e-4119-a50c-803bc0bc9e62 admin admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 10:55:42.937215 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a75a08b-369e-4119-a50c-803bc0bc9e62 admin admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 10:55:42.957372 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d32eefbe-9259-4fb3-9079-d114561476f0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5ac47e3b09f43d19a23a0f457fe9e23', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 10:55:42.958149 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d32eefbe-9259-4fb3-9079-d114561476f0 admin admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:55:42.959366 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d32eefbe-9259-4fb3-9079-d114561476f0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:55:42.959749 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d32eefbe-9259-4fb3-9079-d114561476f0 admin admin] PATH_INFO: `/v3/roles` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:55:42.962079 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d32eefbe-9259-4fb3-9079-d114561476f0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 10:55:42.963831 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d32eefbe-9259-4fb3-9079-d114561476f0 admin admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 10:55:42.990546 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1006/2039] 10.222.0.25 () {58 vars in 1122 bytes} [Sun Aug 7 10:55:42 2022] GET /identity/v3/roles => generated 1420 bytes in 74 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 07 10:55:42.993660 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d32eefbe-9259-4fb3-9079-d114561476f0 admin admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 10:55:43.014975 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d32eefbe-9259-4fb3-9079-d114561476f0 admin admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 10:55:43.020762 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 1034/2040] 10.222.0.25 () {58 vars in 958 bytes} [Sun Aug 7 10:55:42 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 206 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 07 10:55:43.029408 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-370f1425-3dea-4c28-b213-278bba44588a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5ac47e3b09f43d19a23a0f457fe9e23', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 10:55:43.031106 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-370f1425-3dea-4c28-b213-278bba44588a admin admin] REQUEST_METHOD: `PUT` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:55:43.031567 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-370f1425-3dea-4c28-b213-278bba44588a admin admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:55:43.031949 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-370f1425-3dea-4c28-b213-278bba44588a admin admin] PATH_INFO: `/v3/projects/23533bbdae174017bf68eb3072591809/users/79778b8885304daa940b3d03f7c3a862/roles/b98dfb3b9fee4644af2887a3ac3fd2e6` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:55:43.033797 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b677b651-4f01-4b13-b689-59cda7edfc71 None None] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 10:55:43.035851 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.fernet_utils [None req-b677b651-4f01-4b13-b689-59cda7edfc71 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80144) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 10:55:43.065345 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 10:55:43.087546 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-370f1425-3dea-4c28-b213-278bba44588a admin admin] RBAC: Authorizing `identity:create_grant(project_id=23533bbdae174017bf68eb3072591809, user_id=79778b8885304daa940b3d03f7c3a862, role_id=b98dfb3b9fee4644af2887a3ac3fd2e6)` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 10:55:43.093853 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-370f1425-3dea-4c28-b213-278bba44588a admin admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 10:55:43.112365 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=100 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 10:55:43.133669 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b677b651-4f01-4b13-b689-59cda7edfc71 None None] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 10:55:43.135570 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1007/2041] 10.222.0.25 () {60 vars in 1483 bytes} [Sun Aug 7 10:55:42 2022] PUT /identity/v3/projects/23533bbdae174017bf68eb3072591809/users/79778b8885304daa940b3d03f7c3a862/roles/b98dfb3b9fee4644af2887a3ac3fd2e6 => generated 0 bytes in 142 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 07 10:55:43.147085 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96a70d6f-ae54-4937-9b7d-f163084b69d8 None None] REQUEST_METHOD: `POST` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:55:43.147441 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96a70d6f-ae54-4937-9b7d-f163084b69d8 None None] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:55:43.147792 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96a70d6f-ae54-4937-9b7d-f163084b69d8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:55:43.205353 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.auth.core [None req-96a70d6f-ae54-4937-9b7d-f163084b69d8 None None] MFA Rules not processed for user `79778b8885304daa940b3d03f7c3a862`. Rule list: `[]` (Enabled: `True`). {{(pid=80143) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 10:55:43.206729 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-96a70d6f-ae54-4937-9b7d-f163084b69d8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 10:55:43.218186 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de2ddd0f-826f-4c67-9ecb-287b54e2f949 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a11186d9d0d6488b953ede2f1628148d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6de1f3cac02c4bcdb05dd2337176033d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 10:55:43.219127 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de2ddd0f-826f-4c67-9ecb-287b54e2f949 service glance] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:55:43.219426 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de2ddd0f-826f-4c67-9ecb-287b54e2f949 service glance] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:55:43.219689 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de2ddd0f-826f-4c67-9ecb-287b54e2f949 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:55:43.296411 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1008/2042] 10.222.0.25 () {58 vars in 958 bytes} [Sun Aug 7 10:55:43 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 150 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 07 10:55:43.297830 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de2ddd0f-826f-4c67-9ecb-287b54e2f949 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 10:55:43.299475 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de2ddd0f-826f-4c67-9ecb-287b54e2f949 service glance] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 10:55:43.301073 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96a70d6f-ae54-4937-9b7d-f163084b69d8 None None] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 10:55:43.310615 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 1035/2043] 10.222.0.25 () {60 vars in 1365 bytes} [Sun Aug 7 10:55:43 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 278 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 07 10:55:43.388978 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96a70d6f-ae54-4937-9b7d-f163084b69d8 None None] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 10:55:43.395509 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4fa8b96-1e2c-439a-908b-43626adf0ab3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5ac47e3b09f43d19a23a0f457fe9e23', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 10:55:43.396235 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4fa8b96-1e2c-439a-908b-43626adf0ab3 admin admin] REQUEST_METHOD: `POST` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:55:43.396570 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4fa8b96-1e2c-439a-908b-43626adf0ab3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:55:43.396877 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4fa8b96-1e2c-439a-908b-43626adf0ab3 admin admin] PATH_INFO: `/v3/projects` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:55:43.406973 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4fa8b96-1e2c-439a-908b-43626adf0ab3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 10:55:43.406973 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4fa8b96-1e2c-439a-908b-43626adf0ab3 admin admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 10:55:43.453171 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1009/2044] 10.222.0.25 () {60 vars in 1153 bytes} [Sun Aug 7 10:55:43 2022] POST /identity/v3/projects => generated 383 bytes in 153 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 07 10:55:43.457160 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de2ddd0f-826f-4c67-9ecb-287b54e2f949 service glance] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 10:55:43.468613 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de2ddd0f-826f-4c67-9ecb-287b54e2f949 service glance] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 10:55:43.533625 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67a3b000-92a9-47e0-b2b3-402a5c1e54d6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5ac47e3b09f43d19a23a0f457fe9e23', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 10:55:43.534250 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67a3b000-92a9-47e0-b2b3-402a5c1e54d6 admin admin] REQUEST_METHOD: `POST` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:55:43.534955 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67a3b000-92a9-47e0-b2b3-402a5c1e54d6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:55:43.534955 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67a3b000-92a9-47e0-b2b3-402a5c1e54d6 admin admin] PATH_INFO: `/v3/users` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:55:43.535502 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67a3b000-92a9-47e0-b2b3-402a5c1e54d6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 10:55:43.537424 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67a3b000-92a9-47e0-b2b3-402a5c1e54d6 admin admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 10:55:43.562965 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 1036/2045] 10.222.0.25 () {60 vars in 1144 bytes} [Sun Aug 7 10:55:43 2022] POST /identity/v3/users => generated 292 bytes in 106 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 07 10:55:43.567546 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4fa8b96-1e2c-439a-908b-43626adf0ab3 admin admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 10:55:43.581721 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4fa8b96-1e2c-439a-908b-43626adf0ab3 admin admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 10:55:43.587655 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46db6114-7d0a-4634-994a-9965d9196160 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5ac47e3b09f43d19a23a0f457fe9e23', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 10:55:43.588426 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46db6114-7d0a-4634-994a-9965d9196160 admin admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:55:43.588776 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46db6114-7d0a-4634-994a-9965d9196160 admin admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:55:43.589421 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46db6114-7d0a-4634-994a-9965d9196160 admin admin] PATH_INFO: `/v3/roles` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:55:43.590160 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46db6114-7d0a-4634-994a-9965d9196160 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 10:55:43.591836 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46db6114-7d0a-4634-994a-9965d9196160 admin admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 10:55:43.606286 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1010/2046] 10.222.0.25 () {58 vars in 1122 bytes} [Sun Aug 7 10:55:43 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 07 10:55:43.608509 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67a3b000-92a9-47e0-b2b3-402a5c1e54d6 admin admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 10:55:43.618310 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67a3b000-92a9-47e0-b2b3-402a5c1e54d6 admin admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 10:55:43.622832 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1020f842-8e7a-49d1-a6f3-15e1b8a67bf7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5ac47e3b09f43d19a23a0f457fe9e23', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 10:55:43.623488 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1020f842-8e7a-49d1-a6f3-15e1b8a67bf7 admin admin] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:55:43.623765 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1020f842-8e7a-49d1-a6f3-15e1b8a67bf7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:55:43.624037 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1020f842-8e7a-49d1-a6f3-15e1b8a67bf7 admin admin] PATH_INFO: `/v3/roles` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:55:43.624673 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1020f842-8e7a-49d1-a6f3-15e1b8a67bf7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 10:55:43.626331 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1020f842-8e7a-49d1-a6f3-15e1b8a67bf7 admin admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 10:55:43.636549 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 1037/2047] 10.222.0.25 () {58 vars in 1122 bytes} [Sun Aug 7 10:55:43 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 07 10:55:43.641124 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46db6114-7d0a-4634-994a-9965d9196160 admin admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 10:55:43.651584 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46db6114-7d0a-4634-994a-9965d9196160 admin admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 10:55:43.657042 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f3b02b4-f17b-452a-bb27-49a40a9aca00 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5ac47e3b09f43d19a23a0f457fe9e23', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 10:55:43.657708 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f3b02b4-f17b-452a-bb27-49a40a9aca00 admin admin] REQUEST_METHOD: `PUT` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:55:43.658005 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f3b02b4-f17b-452a-bb27-49a40a9aca00 admin admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:55:43.658283 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f3b02b4-f17b-452a-bb27-49a40a9aca00 admin admin] PATH_INFO: `/v3/projects/c3c499c3f7d642f0a4fb5f98489c8750/users/de0b670616264a1aa6b1269240cdf67a/roles/b98dfb3b9fee4644af2887a3ac3fd2e6` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:55:43.675028 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f3b02b4-f17b-452a-bb27-49a40a9aca00 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c3c499c3f7d642f0a4fb5f98489c8750, user_id=de0b670616264a1aa6b1269240cdf67a, role_id=b98dfb3b9fee4644af2887a3ac3fd2e6)` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 10:55:43.677672 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f3b02b4-f17b-452a-bb27-49a40a9aca00 admin admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 10:55:43.690885 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1011/2048] 10.222.0.25 () {60 vars in 1483 bytes} [Sun Aug 7 10:55:43 2022] PUT /identity/v3/projects/c3c499c3f7d642f0a4fb5f98489c8750/users/de0b670616264a1aa6b1269240cdf67a/roles/b98dfb3b9fee4644af2887a3ac3fd2e6 => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 07 10:55:43.703367 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1955548-3c61-4c94-b9d5-204eee54eecd None None] REQUEST_METHOD: `POST` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:55:43.703655 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1955548-3c61-4c94-b9d5-204eee54eecd None None] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:55:43.703931 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1955548-3c61-4c94-b9d5-204eee54eecd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:55:43.735308 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.auth.core [None req-b1955548-3c61-4c94-b9d5-204eee54eecd None None] MFA Rules not processed for user `de0b670616264a1aa6b1269240cdf67a`. Rule list: `[]` (Enabled: `True`). {{(pid=80144) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 10:55:43.736607 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.fernet_utils [None req-b1955548-3c61-4c94-b9d5-204eee54eecd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80144) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 10:55:43.762893 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 10:55:43.816402 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 1038/2049] 10.222.0.25 () {58 vars in 958 bytes} [Sun Aug 7 10:55:43 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 07 10:55:43.820880 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f3b02b4-f17b-452a-bb27-49a40a9aca00 admin admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 10:55:43.857228 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 10:55:43.888739 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f3b02b4-f17b-452a-bb27-49a40a9aca00 admin admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 10:55:43.894510 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3628bd88-2d0c-4890-b375-fcdf1deaf0b6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5ac47e3b09f43d19a23a0f457fe9e23', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 10:55:43.895168 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3628bd88-2d0c-4890-b375-fcdf1deaf0b6 admin admin] REQUEST_METHOD: `POST` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:55:43.895457 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3628bd88-2d0c-4890-b375-fcdf1deaf0b6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:55:43.895735 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3628bd88-2d0c-4890-b375-fcdf1deaf0b6 admin admin] PATH_INFO: `/v3/projects` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:55:43.896371 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3628bd88-2d0c-4890-b375-fcdf1deaf0b6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 10:55:43.898264 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3628bd88-2d0c-4890-b375-fcdf1deaf0b6 admin admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 10:55:43.925662 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1012/2050] 10.222.0.25 () {60 vars in 1153 bytes} [Sun Aug 7 10:55:43 2022] POST /identity/v3/projects => generated 385 bytes in 105 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 07 10:55:43.929736 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1955548-3c61-4c94-b9d5-204eee54eecd None None] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 10:55:43.940934 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1955548-3c61-4c94-b9d5-204eee54eecd None None] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 10:55:43.995052 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5113514d-1484-4aa8-818f-d585d6976527 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5ac47e3b09f43d19a23a0f457fe9e23', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 10:55:43.995657 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5113514d-1484-4aa8-818f-d585d6976527 admin admin] REQUEST_METHOD: `POST` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:55:43.995936 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5113514d-1484-4aa8-818f-d585d6976527 admin admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:55:43.996204 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5113514d-1484-4aa8-818f-d585d6976527 admin admin] PATH_INFO: `/v3/users` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:55:43.996814 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5113514d-1484-4aa8-818f-d585d6976527 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 10:55:43.999313 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5113514d-1484-4aa8-818f-d585d6976527 admin admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 10:55:44.019161 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 1039/2051] 10.222.0.25 () {60 vars in 1144 bytes} [Sun Aug 7 10:55:43 2022] POST /identity/v3/users => generated 299 bytes in 90 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 07 10:55:44.023153 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3628bd88-2d0c-4890-b375-fcdf1deaf0b6 admin admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 10:55:44.033382 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3628bd88-2d0c-4890-b375-fcdf1deaf0b6 admin admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 10:55:44.038407 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24ee9b86-9847-4546-a0c2-f7b7327ee686 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5ac47e3b09f43d19a23a0f457fe9e23', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 10:55:44.039040 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24ee9b86-9847-4546-a0c2-f7b7327ee686 admin admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:55:44.039327 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24ee9b86-9847-4546-a0c2-f7b7327ee686 admin admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:55:44.039604 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24ee9b86-9847-4546-a0c2-f7b7327ee686 admin admin] PATH_INFO: `/v3/roles` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:55:44.040244 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24ee9b86-9847-4546-a0c2-f7b7327ee686 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 10:55:44.041860 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24ee9b86-9847-4546-a0c2-f7b7327ee686 admin admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 10:55:44.053496 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1013/2052] 10.222.0.25 () {58 vars in 1122 bytes} [Sun Aug 7 10:55:44 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 07 10:55:44.057872 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5113514d-1484-4aa8-818f-d585d6976527 admin admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 10:55:44.069911 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5113514d-1484-4aa8-818f-d585d6976527 admin admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 10:55:44.074834 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-880cd0b0-eb8f-404d-9ba8-f85b4d0a5124 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5ac47e3b09f43d19a23a0f457fe9e23', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 10:55:44.075786 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-880cd0b0-eb8f-404d-9ba8-f85b4d0a5124 admin admin] REQUEST_METHOD: `PUT` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:55:44.076099 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-880cd0b0-eb8f-404d-9ba8-f85b4d0a5124 admin admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:55:44.076425 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-880cd0b0-eb8f-404d-9ba8-f85b4d0a5124 admin admin] PATH_INFO: `/v3/projects/a010c7af94fe44ac91756ad7878560df/users/becc5e7ed3d0486b8723495b6fd94cc3/roles/81b1f1bcee2a473088fbbd327c1c5b23` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:55:44.092359 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-880cd0b0-eb8f-404d-9ba8-f85b4d0a5124 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a010c7af94fe44ac91756ad7878560df, user_id=becc5e7ed3d0486b8723495b6fd94cc3, role_id=81b1f1bcee2a473088fbbd327c1c5b23)` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 10:55:44.095413 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-880cd0b0-eb8f-404d-9ba8-f85b4d0a5124 admin admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 10:55:44.109904 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 1040/2053] 10.222.0.25 () {60 vars in 1483 bytes} [Sun Aug 7 10:55:44 2022] PUT /identity/v3/projects/a010c7af94fe44ac91756ad7878560df/users/becc5e7ed3d0486b8723495b6fd94cc3/roles/81b1f1bcee2a473088fbbd327c1c5b23 => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 07 10:55:44.113807 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24ee9b86-9847-4546-a0c2-f7b7327ee686 admin admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 10:55:44.125286 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24ee9b86-9847-4546-a0c2-f7b7327ee686 admin admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 10:55:44.181752 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89e489b7-e823-49a9-a8cf-f501e27d8fd2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5ac47e3b09f43d19a23a0f457fe9e23', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 10:55:44.182379 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89e489b7-e823-49a9-a8cf-f501e27d8fd2 admin admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:55:44.182638 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89e489b7-e823-49a9-a8cf-f501e27d8fd2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:55:44.182908 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89e489b7-e823-49a9-a8cf-f501e27d8fd2 admin admin] PATH_INFO: `/v3/roles` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:55:44.183572 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89e489b7-e823-49a9-a8cf-f501e27d8fd2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 10:55:44.185146 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89e489b7-e823-49a9-a8cf-f501e27d8fd2 admin admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 10:55:44.195628 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1014/2054] 10.222.0.25 () {58 vars in 1122 bytes} [Sun Aug 7 10:55:44 2022] GET /identity/v3/roles => generated 1420 bytes in 82 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 07 10:55:44.199775 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-880cd0b0-eb8f-404d-9ba8-f85b4d0a5124 admin admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 10:55:44.210188 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-880cd0b0-eb8f-404d-9ba8-f85b4d0a5124 admin admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 10:55:44.214693 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57819606-70f2-46e1-a48e-2871ebe3d179 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5ac47e3b09f43d19a23a0f457fe9e23', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 10:55:44.215394 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57819606-70f2-46e1-a48e-2871ebe3d179 admin admin] REQUEST_METHOD: `PUT` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:55:44.215669 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57819606-70f2-46e1-a48e-2871ebe3d179 admin admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:55:44.215933 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57819606-70f2-46e1-a48e-2871ebe3d179 admin admin] PATH_INFO: `/v3/projects/a010c7af94fe44ac91756ad7878560df/users/becc5e7ed3d0486b8723495b6fd94cc3/roles/81b1f1bcee2a473088fbbd327c1c5b23` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:55:44.217863 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57819606-70f2-46e1-a48e-2871ebe3d179 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a010c7af94fe44ac91756ad7878560df, user_id=becc5e7ed3d0486b8723495b6fd94cc3, role_id=81b1f1bcee2a473088fbbd327c1c5b23)` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 10:55:44.220504 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57819606-70f2-46e1-a48e-2871ebe3d179 admin admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 10:55:44.230848 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 1041/2055] 10.222.0.25 () {60 vars in 1483 bytes} [Sun Aug 7 10:55:44 2022] PUT /identity/v3/projects/a010c7af94fe44ac91756ad7878560df/users/becc5e7ed3d0486b8723495b6fd94cc3/roles/81b1f1bcee2a473088fbbd327c1c5b23 => generated 0 bytes in 32 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 07 10:55:44.242141 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d69c797-50d2-4382-a31b-5d2347219bfe None None] REQUEST_METHOD: `POST` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:55:44.242515 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d69c797-50d2-4382-a31b-5d2347219bfe None None] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:55:44.242858 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d69c797-50d2-4382-a31b-5d2347219bfe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:55:44.275264 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.auth.core [None req-7d69c797-50d2-4382-a31b-5d2347219bfe None None] MFA Rules not processed for user `becc5e7ed3d0486b8723495b6fd94cc3`. Rule list: `[]` (Enabled: `True`). {{(pid=80143) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 10:55:44.276501 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-7d69c797-50d2-4382-a31b-5d2347219bfe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 10:55:44.280586 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57819606-70f2-46e1-a48e-2871ebe3d179 admin admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 10:55:44.284110 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.fernet_utils [None req-57819606-70f2-46e1-a48e-2871ebe3d179 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80144) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 10:55:44.351681 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1015/2056] 10.222.0.25 () {58 vars in 958 bytes} [Sun Aug 7 10:55:44 2022] POST /identity/v3/auth/tokens => generated 3329 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 07 10:55:44.366905 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d69c797-50d2-4382-a31b-5d2347219bfe None None] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 10:55:44.368348 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-7d69c797-50d2-4382-a31b-5d2347219bfe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 10:55:44.371998 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57819606-70f2-46e1-a48e-2871ebe3d179 admin admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 10:55:44.376730 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4824fd6e-b04d-4e33-abdc-7fbf54f6a742 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5ac47e3b09f43d19a23a0f457fe9e23', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 10:55:44.377362 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4824fd6e-b04d-4e33-abdc-7fbf54f6a742 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:55:44.377641 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4824fd6e-b04d-4e33-abdc-7fbf54f6a742 admin admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:55:44.377915 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4824fd6e-b04d-4e33-abdc-7fbf54f6a742 admin admin] PATH_INFO: `/v3/users/9c442cfdc7d3434486887d36250a7976` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:55:44.379072 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4824fd6e-b04d-4e33-abdc-7fbf54f6a742 admin admin] RBAC: Authorizing `identity:delete_user(user_id=9c442cfdc7d3434486887d36250a7976)` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 10:55:44.380973 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4824fd6e-b04d-4e33-abdc-7fbf54f6a742 admin admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 10:55:44.394035 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=94,keystone:INSERT=5,keystone:DELETE=1 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 10:55:44.406093 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 10:55:44.443101 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d69c797-50d2-4382-a31b-5d2347219bfe None None] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 10:55:44.449558 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1db5ecd3-99cf-4bd7-a579-d0c0b8da4d2c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88dbd6dfd1164994a3e16da553561910', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6de1f3cac02c4bcdb05dd2337176033d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 10:55:44.450275 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1db5ecd3-99cf-4bd7-a579-d0c0b8da4d2c service cinder] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:55:44.450628 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1db5ecd3-99cf-4bd7-a579-d0c0b8da4d2c service cinder] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:55:44.451044 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1db5ecd3-99cf-4bd7-a579-d0c0b8da4d2c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:55:44.454103 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.notifications [None req-4824fd6e-b04d-4e33-abdc-7fbf54f6a742 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9c442cfdc7d3434486887d36250a7976'} {{(pid=80144) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 10:55:44.461972 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.notifications [None req-4824fd6e-b04d-4e33-abdc-7fbf54f6a742 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9c442cfdc7d3434486887d36250a7976'} {{(pid=80144) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 10:55:44.472589 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.notifications [None req-4824fd6e-b04d-4e33-abdc-7fbf54f6a742 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9c442cfdc7d3434486887d36250a7976'} {{(pid=80144) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 10:55:44.473380 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.notifications [None req-4824fd6e-b04d-4e33-abdc-7fbf54f6a742 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9c442cfdc7d3434486887d36250a7976'} {{(pid=80144) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 10:55:44.474062 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.notifications [None req-4824fd6e-b04d-4e33-abdc-7fbf54f6a742 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9c442cfdc7d3434486887d36250a7976'} {{(pid=80144) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 10:55:44.499276 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 1042/2057] 10.222.0.25 () {58 vars in 1224 bytes} [Sun Aug 7 10:55:44 2022] DELETE /identity/v3/users/9c442cfdc7d3434486887d36250a7976 => generated 0 bytes in 219 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 07 10:55:44.504358 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d592ac7-2fa9-4c85-98dc-c1f96b919739 None None] REQUEST_METHOD: `POST` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:55:44.504835 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d592ac7-2fa9-4c85-98dc-c1f96b919739 None None] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:55:44.505208 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d592ac7-2fa9-4c85-98dc-c1f96b919739 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:55:44.523373 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1db5ecd3-99cf-4bd7-a579-d0c0b8da4d2c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 10:55:44.524823 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1db5ecd3-99cf-4bd7-a579-d0c0b8da4d2c service cinder] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 10:55:44.532741 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.auth.core [None req-2d592ac7-2fa9-4c85-98dc-c1f96b919739 None None] MFA Rules not processed for user `bac4e6b76fe64217ac8fa44e71cffe6a`. Rule list: `[]` (Enabled: `True`). {{(pid=80144) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 10:55:44.534330 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.fernet_utils [None req-2d592ac7-2fa9-4c85-98dc-c1f96b919739 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80144) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 10:55:44.535373 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1016/2058] 10.222.0.25 () {60 vars in 1365 bytes} [Sun Aug 7 10:55:44 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 169 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 07 10:55:44.656534 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 1043/2059] 10.222.0.25 () {58 vars in 958 bytes} [Sun Aug 7 10:55:44 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 154 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 07 10:55:44.667214 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1db5ecd3-99cf-4bd7-a579-d0c0b8da4d2c service cinder] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 10:55:44.668724 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-1db5ecd3-99cf-4bd7-a579-d0c0b8da4d2c service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 10:55:44.838853 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1db5ecd3-99cf-4bd7-a579-d0c0b8da4d2c service cinder] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 10:55:44.946308 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11c33b30-499a-4758-aeed-a9edf2415081 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be1283c9bf47424c9aa3ad27cab03027', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6de1f3cac02c4bcdb05dd2337176033d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 10:55:44.946972 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11c33b30-499a-4758-aeed-a9edf2415081 service neutron] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:55:44.947261 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11c33b30-499a-4758-aeed-a9edf2415081 service neutron] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:55:44.947538 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11c33b30-499a-4758-aeed-a9edf2415081 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:55:44.976432 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11c33b30-499a-4758-aeed-a9edf2415081 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 10:55:44.978167 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11c33b30-499a-4758-aeed-a9edf2415081 service neutron] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 10:55:44.991233 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1017/2060] 10.222.0.25 () {60 vars in 1365 bytes} [Sun Aug 7 10:55:44 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 324 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 07 10:55:45.344649 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d592ac7-2fa9-4c85-98dc-c1f96b919739 None None] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 10:55:45.346153 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.fernet_utils [None req-2d592ac7-2fa9-4c85-98dc-c1f96b919739 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80144) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 10:55:45.368047 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d592ac7-2fa9-4c85-98dc-c1f96b919739 None None] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 10:55:45.373164 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10febcf3-bbbb-44e7-9767-e1d5b63e54d8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5ac47e3b09f43d19a23a0f457fe9e23', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 10:55:45.373845 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10febcf3-bbbb-44e7-9767-e1d5b63e54d8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:55:45.374171 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10febcf3-bbbb-44e7-9767-e1d5b63e54d8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:55:45.374515 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10febcf3-bbbb-44e7-9767-e1d5b63e54d8 admin admin] PATH_INFO: `/v3/projects/3e02268d7a254dc08ae29fc6b691bb9a` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:55:45.376391 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10febcf3-bbbb-44e7-9767-e1d5b63e54d8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=3e02268d7a254dc08ae29fc6b691bb9a)` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 10:55:45.379487 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10febcf3-bbbb-44e7-9767-e1d5b63e54d8 admin admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 10:55:45.434201 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.notifications [None req-10febcf3-bbbb-44e7-9767-e1d5b63e54d8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3e02268d7a254dc08ae29fc6b691bb9a', 'request_id': 'req-10febcf3-bbbb-44e7-9767-e1d5b63e54d8'} {{(pid=80144) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 10:55:45.440706 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.notifications [None req-10febcf3-bbbb-44e7-9767-e1d5b63e54d8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3e02268d7a254dc08ae29fc6b691bb9a', 'request_id': 'req-10febcf3-bbbb-44e7-9767-e1d5b63e54d8'} {{(pid=80144) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 10:55:45.448040 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.notifications [None req-10febcf3-bbbb-44e7-9767-e1d5b63e54d8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3e02268d7a254dc08ae29fc6b691bb9a', 'request_id': 'req-10febcf3-bbbb-44e7-9767-e1d5b63e54d8'} {{(pid=80144) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 10:55:45.460339 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.notifications [None req-10febcf3-bbbb-44e7-9767-e1d5b63e54d8 admin admin] The token cache is being invalidate because project 3e02268d7a254dc08ae29fc6b691bb9a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80144) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 07 10:55:45.460680 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.notifications [None req-10febcf3-bbbb-44e7-9767-e1d5b63e54d8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80144) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 10:55:45.461329 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.notifications [None req-10febcf3-bbbb-44e7-9767-e1d5b63e54d8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80144) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 10:55:45.463050 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 1044/2061] 10.222.0.25 () {58 vars in 1233 bytes} [Sun Aug 7 10:55:45 2022] DELETE /identity/v3/projects/3e02268d7a254dc08ae29fc6b691bb9a => generated 0 bytes in 119 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 07 10:55:45.467161 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11c33b30-499a-4758-aeed-a9edf2415081 service neutron] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 10:55:45.476861 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11c33b30-499a-4758-aeed-a9edf2415081 service neutron] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 10:55:45.518427 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=100 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 10:55:45.542689 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0daf6d28-0f40-4848-ac12-b4d3b886b527 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5ac47e3b09f43d19a23a0f457fe9e23', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 10:55:45.543417 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0daf6d28-0f40-4848-ac12-b4d3b886b527 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:55:45.543669 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0daf6d28-0f40-4848-ac12-b4d3b886b527 admin admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:55:45.543943 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0daf6d28-0f40-4848-ac12-b4d3b886b527 admin admin] PATH_INFO: `/v3/users/8fad464767a9417ca3116a1a1eb82d70` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:55:45.545755 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0daf6d28-0f40-4848-ac12-b4d3b886b527 admin admin] RBAC: Authorizing `identity:delete_user(user_id=8fad464767a9417ca3116a1a1eb82d70)` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 10:55:45.547774 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0daf6d28-0f40-4848-ac12-b4d3b886b527 admin admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 10:55:45.604505 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-0daf6d28-0f40-4848-ac12-b4d3b886b527 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8fad464767a9417ca3116a1a1eb82d70'} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 10:55:45.614250 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-0daf6d28-0f40-4848-ac12-b4d3b886b527 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8fad464767a9417ca3116a1a1eb82d70'} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 10:55:45.628030 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-0daf6d28-0f40-4848-ac12-b4d3b886b527 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8fad464767a9417ca3116a1a1eb82d70'} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 10:55:45.628924 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-0daf6d28-0f40-4848-ac12-b4d3b886b527 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8fad464767a9417ca3116a1a1eb82d70'} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 10:55:45.637438 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-0daf6d28-0f40-4848-ac12-b4d3b886b527 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8fad464767a9417ca3116a1a1eb82d70'} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 10:55:45.644833 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1018/2062] 10.222.0.25 () {58 vars in 1224 bytes} [Sun Aug 7 10:55:45 2022] DELETE /identity/v3/users/8fad464767a9417ca3116a1a1eb82d70 => generated 0 bytes in 178 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 07 10:55:46.088651 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10febcf3-bbbb-44e7-9767-e1d5b63e54d8 admin admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 10:55:46.090196 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.fernet_utils [None req-10febcf3-bbbb-44e7-9767-e1d5b63e54d8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80144) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 10:55:46.117212 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10febcf3-bbbb-44e7-9767-e1d5b63e54d8 admin admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 10:55:46.151777 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=82,keystone:DELETE=16,keystone:INSERT=2 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 10:55:46.231873 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d493f3ba-2879-4392-a131-2fd77130c8e6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5ac47e3b09f43d19a23a0f457fe9e23', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 10:55:46.232806 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d493f3ba-2879-4392-a131-2fd77130c8e6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:55:46.234222 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d493f3ba-2879-4392-a131-2fd77130c8e6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:55:46.234830 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d493f3ba-2879-4392-a131-2fd77130c8e6 admin admin] PATH_INFO: `/v3/projects/c733617fe0ff49e28c5641f347b61228` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:55:46.236719 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d493f3ba-2879-4392-a131-2fd77130c8e6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c733617fe0ff49e28c5641f347b61228)` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 10:55:46.238993 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d493f3ba-2879-4392-a131-2fd77130c8e6 admin admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 10:55:46.313135 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.notifications [None req-d493f3ba-2879-4392-a131-2fd77130c8e6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c733617fe0ff49e28c5641f347b61228', 'request_id': 'req-d493f3ba-2879-4392-a131-2fd77130c8e6'} {{(pid=80144) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 10:55:46.320507 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.notifications [None req-d493f3ba-2879-4392-a131-2fd77130c8e6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c733617fe0ff49e28c5641f347b61228', 'request_id': 'req-d493f3ba-2879-4392-a131-2fd77130c8e6'} {{(pid=80144) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 10:55:46.335238 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.notifications [None req-d493f3ba-2879-4392-a131-2fd77130c8e6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c733617fe0ff49e28c5641f347b61228', 'request_id': 'req-d493f3ba-2879-4392-a131-2fd77130c8e6'} {{(pid=80144) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 10:55:46.349191 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.notifications [None req-d493f3ba-2879-4392-a131-2fd77130c8e6 admin admin] The token cache is being invalidate because project c733617fe0ff49e28c5641f347b61228 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80144) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 07 10:55:46.349919 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.notifications [None req-d493f3ba-2879-4392-a131-2fd77130c8e6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80144) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 10:55:46.350206 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.notifications [None req-d493f3ba-2879-4392-a131-2fd77130c8e6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80144) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 10:55:46.351743 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 1045/2063] 10.222.0.25 () {58 vars in 1233 bytes} [Sun Aug 7 10:55:46 2022] DELETE /identity/v3/projects/c733617fe0ff49e28c5641f347b61228 => generated 0 bytes in 264 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 07 10:55:46.356611 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72b710e2-32a3-4f54-b4a3-253b1ece09e9 None None] REQUEST_METHOD: `POST` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:55:46.356901 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72b710e2-32a3-4f54-b4a3-253b1ece09e9 None None] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:55:46.357178 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72b710e2-32a3-4f54-b4a3-253b1ece09e9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:55:46.378351 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.auth.core [None req-72b710e2-32a3-4f54-b4a3-253b1ece09e9 None None] MFA Rules not processed for user `bac4e6b76fe64217ac8fa44e71cffe6a`. Rule list: `[]` (Enabled: `True`). {{(pid=80143) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 10:55:46.379605 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-72b710e2-32a3-4f54-b4a3-253b1ece09e9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 10:55:46.444476 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1019/2064] 10.222.0.25 () {58 vars in 958 bytes} [Sun Aug 7 10:55:46 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 07 10:55:46.453314 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3490afbc-640e-40eb-acc5-9c79bd37e080 None None] REQUEST_METHOD: `POST` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:55:46.453656 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3490afbc-640e-40eb-acc5-9c79bd37e080 None None] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:55:46.453990 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3490afbc-640e-40eb-acc5-9c79bd37e080 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:55:46.488381 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.auth.core [None req-3490afbc-640e-40eb-acc5-9c79bd37e080 None None] MFA Rules not processed for user `bac4e6b76fe64217ac8fa44e71cffe6a`. Rule list: `[]` (Enabled: `True`). {{(pid=80144) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 10:55:46.489753 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.fernet_utils [None req-3490afbc-640e-40eb-acc5-9c79bd37e080 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80144) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 10:55:46.498695 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 1046/2065] 10.222.0.25 () {58 vars in 958 bytes} [Sun Aug 7 10:55:46 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 47 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 07 10:55:46.503179 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72b710e2-32a3-4f54-b4a3-253b1ece09e9 None None] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 10:55:46.513162 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72b710e2-32a3-4f54-b4a3-253b1ece09e9 None None] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 10:55:46.518304 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-420b7f5a-e8cc-4880-8f48-020ccdd558c2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5ac47e3b09f43d19a23a0f457fe9e23', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 10:55:46.518900 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-420b7f5a-e8cc-4880-8f48-020ccdd558c2 admin admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:55:46.519458 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-420b7f5a-e8cc-4880-8f48-020ccdd558c2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:55:46.519823 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-420b7f5a-e8cc-4880-8f48-020ccdd558c2 admin admin] PATH_INFO: `/v3/domains` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:55:46.520545 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-420b7f5a-e8cc-4880-8f48-020ccdd558c2 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80143) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 10:55:46.520854 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-420b7f5a-e8cc-4880-8f48-020ccdd558c2 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 10:55:46.523264 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-420b7f5a-e8cc-4880-8f48-020ccdd558c2 admin admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 10:55:46.543052 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1020/2066] 10.222.0.25 () {58 vars in 1153 bytes} [Sun Aug 7 10:55:46 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 10:55:46.545283 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3490afbc-640e-40eb-acc5-9c79bd37e080 None None] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 10:55:46.556692 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3490afbc-640e-40eb-acc5-9c79bd37e080 None None] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 10:55:46.562753 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a4ecd05-1678-4938-a7bf-fbd8591dbb3e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5ac47e3b09f43d19a23a0f457fe9e23', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 10:55:46.563384 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a4ecd05-1678-4938-a7bf-fbd8591dbb3e admin admin] REQUEST_METHOD: `POST` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:55:46.563667 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a4ecd05-1678-4938-a7bf-fbd8591dbb3e admin admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:55:46.563938 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a4ecd05-1678-4938-a7bf-fbd8591dbb3e admin admin] PATH_INFO: `/v3/projects` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:55:46.564623 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a4ecd05-1678-4938-a7bf-fbd8591dbb3e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 10:55:46.566532 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a4ecd05-1678-4938-a7bf-fbd8591dbb3e admin admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 10:55:46.595442 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 1047/2067] 10.222.0.25 () {60 vars in 1153 bytes} [Sun Aug 7 10:55:46 2022] POST /identity/v3/projects => generated 387 bytes in 50 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 07 10:55:46.600143 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-420b7f5a-e8cc-4880-8f48-020ccdd558c2 admin admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 10:55:46.620152 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-420b7f5a-e8cc-4880-8f48-020ccdd558c2 admin admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 10:55:46.677388 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 10:55:46.693582 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f20cde0-ad73-46cb-a2c7-9eefc0b5cd02 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5ac47e3b09f43d19a23a0f457fe9e23', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 10:55:46.694203 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f20cde0-ad73-46cb-a2c7-9eefc0b5cd02 admin admin] REQUEST_METHOD: `POST` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:55:46.694622 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f20cde0-ad73-46cb-a2c7-9eefc0b5cd02 admin admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:55:46.694846 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f20cde0-ad73-46cb-a2c7-9eefc0b5cd02 admin admin] PATH_INFO: `/v3/users` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:55:46.695513 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f20cde0-ad73-46cb-a2c7-9eefc0b5cd02 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 10:55:46.697441 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f20cde0-ad73-46cb-a2c7-9eefc0b5cd02 admin admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 10:55:46.719397 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1021/2068] 10.222.0.25 () {60 vars in 1144 bytes} [Sun Aug 7 10:55:46 2022] POST /identity/v3/users => generated 294 bytes in 120 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 07 10:55:46.723709 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a4ecd05-1678-4938-a7bf-fbd8591dbb3e admin admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 10:55:46.735529 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a4ecd05-1678-4938-a7bf-fbd8591dbb3e admin admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 10:55:46.740474 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54840dcd-7c31-4c7b-a1d1-28f5b8d1476e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5ac47e3b09f43d19a23a0f457fe9e23', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 10:55:46.741088 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54840dcd-7c31-4c7b-a1d1-28f5b8d1476e admin admin] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:55:46.741366 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54840dcd-7c31-4c7b-a1d1-28f5b8d1476e admin admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:55:46.741638 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54840dcd-7c31-4c7b-a1d1-28f5b8d1476e admin admin] PATH_INFO: `/v3/roles` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:55:46.742280 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54840dcd-7c31-4c7b-a1d1-28f5b8d1476e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 10:55:46.743916 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54840dcd-7c31-4c7b-a1d1-28f5b8d1476e admin admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 10:55:46.757513 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 1048/2069] 10.222.0.25 () {58 vars in 1122 bytes} [Sun Aug 7 10:55:46 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 07 10:55:46.759136 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f20cde0-ad73-46cb-a2c7-9eefc0b5cd02 admin admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 10:55:46.771909 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f20cde0-ad73-46cb-a2c7-9eefc0b5cd02 admin admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 10:55:46.776782 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ed3a2b2-3f6e-4756-bb39-c6ce7ed74422 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5ac47e3b09f43d19a23a0f457fe9e23', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 10:55:46.777418 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ed3a2b2-3f6e-4756-bb39-c6ce7ed74422 admin admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:55:46.777695 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ed3a2b2-3f6e-4756-bb39-c6ce7ed74422 admin admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:55:46.777963 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ed3a2b2-3f6e-4756-bb39-c6ce7ed74422 admin admin] PATH_INFO: `/v3/roles` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:55:46.778614 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ed3a2b2-3f6e-4756-bb39-c6ce7ed74422 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 10:55:46.781046 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ed3a2b2-3f6e-4756-bb39-c6ce7ed74422 admin admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 10:55:46.796602 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1022/2070] 10.222.0.25 () {58 vars in 1122 bytes} [Sun Aug 7 10:55:46 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 07 10:55:46.800981 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54840dcd-7c31-4c7b-a1d1-28f5b8d1476e admin admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 10:55:46.811330 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54840dcd-7c31-4c7b-a1d1-28f5b8d1476e admin admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 10:55:46.816367 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a248a0f-0449-4033-8dfe-d889fc089ad4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5ac47e3b09f43d19a23a0f457fe9e23', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 10:55:46.817075 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a248a0f-0449-4033-8dfe-d889fc089ad4 admin admin] REQUEST_METHOD: `PUT` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:55:46.817365 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a248a0f-0449-4033-8dfe-d889fc089ad4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:55:46.817643 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a248a0f-0449-4033-8dfe-d889fc089ad4 admin admin] PATH_INFO: `/v3/projects/82389a28a28d4694bdaab21524f172b8/users/f8b91e679e1d4f45846801a0bccae3e3/roles/b98dfb3b9fee4644af2887a3ac3fd2e6` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:55:46.848687 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a248a0f-0449-4033-8dfe-d889fc089ad4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=82389a28a28d4694bdaab21524f172b8, user_id=f8b91e679e1d4f45846801a0bccae3e3, role_id=b98dfb3b9fee4644af2887a3ac3fd2e6)` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 10:55:46.848938 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a248a0f-0449-4033-8dfe-d889fc089ad4 admin admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 10:55:46.874302 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 1049/2071] 10.222.0.25 () {60 vars in 1483 bytes} [Sun Aug 7 10:55:46 2022] PUT /identity/v3/projects/82389a28a28d4694bdaab21524f172b8/users/f8b91e679e1d4f45846801a0bccae3e3/roles/b98dfb3b9fee4644af2887a3ac3fd2e6 => generated 0 bytes in 65 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 07 10:55:46.877992 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c88d379-7c13-49f2-a86a-cbe851094e25 None None] REQUEST_METHOD: `POST` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:55:46.878333 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c88d379-7c13-49f2-a86a-cbe851094e25 None None] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:55:46.878678 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c88d379-7c13-49f2-a86a-cbe851094e25 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:55:46.931139 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.auth.core [None req-1c88d379-7c13-49f2-a86a-cbe851094e25 None None] MFA Rules not processed for user `f8b91e679e1d4f45846801a0bccae3e3`. Rule list: `[]` (Enabled: `True`). {{(pid=80143) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 10:55:46.932370 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-1c88d379-7c13-49f2-a86a-cbe851094e25 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 10:55:47.003627 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1023/2072] 10.222.0.25 () {58 vars in 958 bytes} [Sun Aug 7 10:55:46 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 07 10:55:47.008977 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a248a0f-0449-4033-8dfe-d889fc089ad4 admin admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 10:55:47.060471 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 10:55:47.082534 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a248a0f-0449-4033-8dfe-d889fc089ad4 admin admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 10:55:47.087607 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13203f67-2218-4b4e-a503-57e99281b1b6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5ac47e3b09f43d19a23a0f457fe9e23', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 10:55:47.088220 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13203f67-2218-4b4e-a503-57e99281b1b6 admin admin] REQUEST_METHOD: `POST` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:55:47.088626 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13203f67-2218-4b4e-a503-57e99281b1b6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:55:47.088908 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13203f67-2218-4b4e-a503-57e99281b1b6 admin admin] PATH_INFO: `/v3/projects` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:55:47.089550 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13203f67-2218-4b4e-a503-57e99281b1b6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 10:55:47.091484 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13203f67-2218-4b4e-a503-57e99281b1b6 admin admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 10:55:47.121482 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 1050/2073] 10.222.0.25 () {60 vars in 1153 bytes} [Sun Aug 7 10:55:47 2022] POST /identity/v3/projects => generated 387 bytes in 113 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 07 10:55:47.126456 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c88d379-7c13-49f2-a86a-cbe851094e25 None None] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 10:55:47.138320 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c88d379-7c13-49f2-a86a-cbe851094e25 None None] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 10:55:47.244705 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18e1b8eb-372d-43fd-94aa-8123152421cd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5ac47e3b09f43d19a23a0f457fe9e23', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 10:55:47.245881 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18e1b8eb-372d-43fd-94aa-8123152421cd admin admin] REQUEST_METHOD: `POST` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:55:47.246438 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18e1b8eb-372d-43fd-94aa-8123152421cd admin admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:55:47.247025 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18e1b8eb-372d-43fd-94aa-8123152421cd admin admin] PATH_INFO: `/v3/users` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:55:47.249308 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18e1b8eb-372d-43fd-94aa-8123152421cd admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 10:55:47.253189 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18e1b8eb-372d-43fd-94aa-8123152421cd admin admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 10:55:47.293531 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1024/2074] 10.222.0.25 () {60 vars in 1144 bytes} [Sun Aug 7 10:55:47 2022] POST /identity/v3/users => generated 300 bytes in 168 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 07 10:55:47.300890 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13203f67-2218-4b4e-a503-57e99281b1b6 admin admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 10:55:47.328538 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13203f67-2218-4b4e-a503-57e99281b1b6 admin admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 10:55:47.336446 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42c7f45e-96cc-43bd-936f-c6ba88d1be3c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5ac47e3b09f43d19a23a0f457fe9e23', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 10:55:47.337062 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42c7f45e-96cc-43bd-936f-c6ba88d1be3c admin admin] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:55:47.337340 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42c7f45e-96cc-43bd-936f-c6ba88d1be3c admin admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:55:47.337654 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42c7f45e-96cc-43bd-936f-c6ba88d1be3c admin admin] PATH_INFO: `/v3/roles` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:55:47.338340 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42c7f45e-96cc-43bd-936f-c6ba88d1be3c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 10:55:47.340063 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42c7f45e-96cc-43bd-936f-c6ba88d1be3c admin admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 10:55:47.358229 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 1051/2075] 10.222.0.25 () {58 vars in 1122 bytes} [Sun Aug 7 10:55:47 2022] GET /identity/v3/roles => generated 1420 bytes in 59 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 07 10:55:47.364645 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18e1b8eb-372d-43fd-94aa-8123152421cd admin admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 10:55:47.376598 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18e1b8eb-372d-43fd-94aa-8123152421cd admin admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 10:55:47.381315 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16d2112d-4770-4e37-836a-79358a0b8757 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5ac47e3b09f43d19a23a0f457fe9e23', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 10:55:47.381982 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16d2112d-4770-4e37-836a-79358a0b8757 admin admin] REQUEST_METHOD: `PUT` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:55:47.382258 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16d2112d-4770-4e37-836a-79358a0b8757 admin admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:55:47.382527 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16d2112d-4770-4e37-836a-79358a0b8757 admin admin] PATH_INFO: `/v3/projects/3e0ffc2e341c4b20a568b649b3294ac1/users/44ea8851cb11486fb73bd4bef0d113aa/roles/81b1f1bcee2a473088fbbd327c1c5b23` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:55:47.453635 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16d2112d-4770-4e37-836a-79358a0b8757 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3e0ffc2e341c4b20a568b649b3294ac1, user_id=44ea8851cb11486fb73bd4bef0d113aa, role_id=81b1f1bcee2a473088fbbd327c1c5b23)` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 10:55:47.457077 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16d2112d-4770-4e37-836a-79358a0b8757 admin admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 10:55:47.487831 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42c7f45e-96cc-43bd-936f-c6ba88d1be3c admin admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 10:55:47.500417 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42c7f45e-96cc-43bd-936f-c6ba88d1be3c admin admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 10:55:47.501116 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1025/2076] 10.222.0.25 () {60 vars in 1483 bytes} [Sun Aug 7 10:55:47 2022] PUT /identity/v3/projects/3e0ffc2e341c4b20a568b649b3294ac1/users/44ea8851cb11486fb73bd4bef0d113aa/roles/81b1f1bcee2a473088fbbd327c1c5b23 => generated 0 bytes in 118 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 07 10:55:47.540831 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16d2112d-4770-4e37-836a-79358a0b8757 admin admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 10:55:47.543032 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-16d2112d-4770-4e37-836a-79358a0b8757 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 10:55:47.579643 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 10:55:47.611699 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a261fc1a-cc27-4c4a-ab42-d8a5a677f7f0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5ac47e3b09f43d19a23a0f457fe9e23', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 10:55:47.613809 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a261fc1a-cc27-4c4a-ab42-d8a5a677f7f0 admin admin] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:55:47.614127 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a261fc1a-cc27-4c4a-ab42-d8a5a677f7f0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:55:47.614586 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a261fc1a-cc27-4c4a-ab42-d8a5a677f7f0 admin admin] PATH_INFO: `/v3/roles` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:55:47.615326 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a261fc1a-cc27-4c4a-ab42-d8a5a677f7f0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 10:55:47.616976 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a261fc1a-cc27-4c4a-ab42-d8a5a677f7f0 admin admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 10:55:47.635680 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 1052/2077] 10.222.0.25 () {58 vars in 1122 bytes} [Sun Aug 7 10:55:47 2022] GET /identity/v3/roles => generated 1420 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 07 10:55:47.646001 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a261fc1a-cc27-4c4a-ab42-d8a5a677f7f0 admin admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 10:55:47.657510 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a261fc1a-cc27-4c4a-ab42-d8a5a677f7f0 admin admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 10:55:47.662611 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88ca8d08-a2bb-4d58-ad0d-be83b033a654 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5ac47e3b09f43d19a23a0f457fe9e23', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 10:55:47.663442 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88ca8d08-a2bb-4d58-ad0d-be83b033a654 admin admin] REQUEST_METHOD: `PUT` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:55:47.663916 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88ca8d08-a2bb-4d58-ad0d-be83b033a654 admin admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:55:47.664193 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88ca8d08-a2bb-4d58-ad0d-be83b033a654 admin admin] PATH_INFO: `/v3/projects/3e0ffc2e341c4b20a568b649b3294ac1/users/44ea8851cb11486fb73bd4bef0d113aa/roles/81b1f1bcee2a473088fbbd327c1c5b23` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:55:47.666752 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88ca8d08-a2bb-4d58-ad0d-be83b033a654 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3e0ffc2e341c4b20a568b649b3294ac1, user_id=44ea8851cb11486fb73bd4bef0d113aa, role_id=81b1f1bcee2a473088fbbd327c1c5b23)` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 10:55:47.669458 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88ca8d08-a2bb-4d58-ad0d-be83b033a654 admin admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 10:55:47.674113 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16d2112d-4770-4e37-836a-79358a0b8757 admin admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 10:55:47.682403 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 1053/2078] 10.222.0.25 () {60 vars in 1483 bytes} [Sun Aug 7 10:55:47 2022] PUT /identity/v3/projects/3e0ffc2e341c4b20a568b649b3294ac1/users/44ea8851cb11486fb73bd4bef0d113aa/roles/81b1f1bcee2a473088fbbd327c1c5b23 => generated 0 bytes in 38 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 07 10:55:47.703827 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fc254be-6882-46c6-993f-9ff2a6cb8bf0 None None] REQUEST_METHOD: `POST` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:55:47.704386 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fc254be-6882-46c6-993f-9ff2a6cb8bf0 None None] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:55:47.704668 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fc254be-6882-46c6-993f-9ff2a6cb8bf0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:55:47.735786 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.auth.core [None req-4fc254be-6882-46c6-993f-9ff2a6cb8bf0 None None] MFA Rules not processed for user `44ea8851cb11486fb73bd4bef0d113aa`. Rule list: `[]` (Enabled: `True`). {{(pid=80144) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 10:55:47.737150 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.fernet_utils [None req-4fc254be-6882-46c6-993f-9ff2a6cb8bf0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80144) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 10:55:47.765337 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea9e022b-a0d9-43ec-843f-d22bb27a046c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88dbd6dfd1164994a3e16da553561910', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6de1f3cac02c4bcdb05dd2337176033d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 10:55:47.765950 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea9e022b-a0d9-43ec-843f-d22bb27a046c service cinder] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:55:47.766228 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea9e022b-a0d9-43ec-843f-d22bb27a046c service cinder] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:55:47.766501 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea9e022b-a0d9-43ec-843f-d22bb27a046c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:55:47.768364 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-ea9e022b-a0d9-43ec-843f-d22bb27a046c service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 10:55:47.821968 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 1054/2079] 10.222.0.25 () {58 vars in 958 bytes} [Sun Aug 7 10:55:47 2022] POST /identity/v3/auth/tokens => generated 3331 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 07 10:55:47.835376 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fc254be-6882-46c6-993f-9ff2a6cb8bf0 None None] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 10:55:47.837409 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.fernet_utils [None req-4fc254be-6882-46c6-993f-9ff2a6cb8bf0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80144) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 10:55:47.857279 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fc254be-6882-46c6-993f-9ff2a6cb8bf0 None None] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 10:55:47.863015 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a82c1e8c-007c-4b23-9300-008b1a704211 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88dbd6dfd1164994a3e16da553561910', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6de1f3cac02c4bcdb05dd2337176033d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 10:55:47.863423 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea9e022b-a0d9-43ec-843f-d22bb27a046c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 10:55:47.863654 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a82c1e8c-007c-4b23-9300-008b1a704211 service cinder] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:55:47.863937 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a82c1e8c-007c-4b23-9300-008b1a704211 service cinder] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:55:47.864212 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a82c1e8c-007c-4b23-9300-008b1a704211 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:55:47.865565 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea9e022b-a0d9-43ec-843f-d22bb27a046c service cinder] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 10:55:47.875405 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a82c1e8c-007c-4b23-9300-008b1a704211 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 10:55:47.877058 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a82c1e8c-007c-4b23-9300-008b1a704211 service cinder] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 10:55:47.882435 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1026/2080] 10.222.0.25 () {60 vars in 1365 bytes} [Sun Aug 7 10:55:47 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 342 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 07 10:55:47.890948 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 1055/2081] 10.222.0.25 () {60 vars in 1365 bytes} [Sun Aug 7 10:55:47 2022] GET /identity/v3/auth/tokens => generated 3331 bytes in 56 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 07 10:55:48.029714 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea9e022b-a0d9-43ec-843f-d22bb27a046c service cinder] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 10:55:48.155204 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea9e022b-a0d9-43ec-843f-d22bb27a046c service cinder] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 10:55:48.161308 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-854dd9b7-1f72-4a45-92a7-abf9d9e0962d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5ac47e3b09f43d19a23a0f457fe9e23', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 10:55:48.162274 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-854dd9b7-1f72-4a45-92a7-abf9d9e0962d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:55:48.162794 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-854dd9b7-1f72-4a45-92a7-abf9d9e0962d admin admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:55:48.163337 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-854dd9b7-1f72-4a45-92a7-abf9d9e0962d admin admin] PATH_INFO: `/v3/users/f8b91e679e1d4f45846801a0bccae3e3` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:55:48.164903 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-854dd9b7-1f72-4a45-92a7-abf9d9e0962d admin admin] RBAC: Authorizing `identity:delete_user(user_id=f8b91e679e1d4f45846801a0bccae3e3)` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 10:55:48.167604 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-854dd9b7-1f72-4a45-92a7-abf9d9e0962d admin admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 10:55:48.232359 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=95,keystone:DELETE=5 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 10:55:48.259038 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-854dd9b7-1f72-4a45-92a7-abf9d9e0962d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f8b91e679e1d4f45846801a0bccae3e3'} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 10:55:48.268488 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-854dd9b7-1f72-4a45-92a7-abf9d9e0962d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f8b91e679e1d4f45846801a0bccae3e3'} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 10:55:48.285752 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-854dd9b7-1f72-4a45-92a7-abf9d9e0962d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f8b91e679e1d4f45846801a0bccae3e3'} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 10:55:48.286663 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-854dd9b7-1f72-4a45-92a7-abf9d9e0962d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f8b91e679e1d4f45846801a0bccae3e3'} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 10:55:48.303721 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-854dd9b7-1f72-4a45-92a7-abf9d9e0962d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f8b91e679e1d4f45846801a0bccae3e3'} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 10:55:48.311250 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1027/2082] 10.222.0.25 () {58 vars in 1224 bytes} [Sun Aug 7 10:55:48 2022] DELETE /identity/v3/users/f8b91e679e1d4f45846801a0bccae3e3 => generated 0 bytes in 281 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 07 10:55:48.315630 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-419a2892-fa1d-4835-b7c6-634747b861ad None None] REQUEST_METHOD: `POST` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:55:48.315950 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-419a2892-fa1d-4835-b7c6-634747b861ad None None] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:55:48.316331 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-419a2892-fa1d-4835-b7c6-634747b861ad None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:55:48.338129 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.auth.core [None req-419a2892-fa1d-4835-b7c6-634747b861ad None None] MFA Rules not processed for user `bac4e6b76fe64217ac8fa44e71cffe6a`. Rule list: `[]` (Enabled: `True`). {{(pid=80144) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 10:55:48.339432 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.fernet_utils [None req-419a2892-fa1d-4835-b7c6-634747b861ad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80144) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 10:55:48.344199 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 10:55:48.421410 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 1056/2083] 10.222.0.25 () {58 vars in 958 bytes} [Sun Aug 7 10:55:48 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 07 10:55:48.428540 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-854dd9b7-1f72-4a45-92a7-abf9d9e0962d admin admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 10:55:48.429954 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-854dd9b7-1f72-4a45-92a7-abf9d9e0962d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 10:55:48.499895 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-854dd9b7-1f72-4a45-92a7-abf9d9e0962d admin admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 10:55:48.504623 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f17d48a3-a233-4dbb-88b1-45d94e6178e8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be1283c9bf47424c9aa3ad27cab03027', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6de1f3cac02c4bcdb05dd2337176033d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 10:55:48.505429 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f17d48a3-a233-4dbb-88b1-45d94e6178e8 service neutron] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:55:48.505936 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f17d48a3-a233-4dbb-88b1-45d94e6178e8 service neutron] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:55:48.506536 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f17d48a3-a233-4dbb-88b1-45d94e6178e8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:55:48.518462 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f17d48a3-a233-4dbb-88b1-45d94e6178e8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 10:55:48.520397 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f17d48a3-a233-4dbb-88b1-45d94e6178e8 service neutron] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 10:55:48.532536 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1028/2084] 10.222.0.25 () {60 vars in 1365 bytes} [Sun Aug 7 10:55:48 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 104 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 07 10:55:48.691070 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-419a2892-fa1d-4835-b7c6-634747b861ad None None] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 10:55:48.692705 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.fernet_utils [None req-419a2892-fa1d-4835-b7c6-634747b861ad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80144) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 10:55:48.768475 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-419a2892-fa1d-4835-b7c6-634747b861ad None None] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 10:55:48.772587 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d33af56-9fd6-4f6e-8761-e20db0e734d0 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88dbd6dfd1164994a3e16da553561910', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6de1f3cac02c4bcdb05dd2337176033d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 10:55:48.773294 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d33af56-9fd6-4f6e-8761-e20db0e734d0 service cinder] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:55:48.773631 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d33af56-9fd6-4f6e-8761-e20db0e734d0 service cinder] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:55:48.773973 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d33af56-9fd6-4f6e-8761-e20db0e734d0 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:55:48.776050 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.fernet_utils [None req-4d33af56-9fd6-4f6e-8761-e20db0e734d0 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80144) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 10:55:48.869250 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d33af56-9fd6-4f6e-8761-e20db0e734d0 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 10:55:48.870708 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d33af56-9fd6-4f6e-8761-e20db0e734d0 service cinder] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 10:55:48.882332 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 1057/2085] 10.222.0.25 () {60 vars in 1365 bytes} [Sun Aug 7 10:55:48 2022] GET /identity/v3/auth/tokens => generated 3329 bytes in 192 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 07 10:55:49.208806 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f17d48a3-a233-4dbb-88b1-45d94e6178e8 service neutron] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 10:55:49.210455 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-f17d48a3-a233-4dbb-88b1-45d94e6178e8 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 10:55:49.230658 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f17d48a3-a233-4dbb-88b1-45d94e6178e8 service neutron] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 10:55:49.251863 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ff8a471-2812-4dd6-86d9-747af02e2b49 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5ac47e3b09f43d19a23a0f457fe9e23', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 10:55:49.252686 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ff8a471-2812-4dd6-86d9-747af02e2b49 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:55:49.252988 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ff8a471-2812-4dd6-86d9-747af02e2b49 admin admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:55:49.253246 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ff8a471-2812-4dd6-86d9-747af02e2b49 admin admin] PATH_INFO: `/v3/projects/82389a28a28d4694bdaab21524f172b8` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:55:49.254419 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ff8a471-2812-4dd6-86d9-747af02e2b49 admin admin] RBAC: Authorizing `identity:delete_project(project_id=82389a28a28d4694bdaab21524f172b8)` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 10:55:49.256392 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ff8a471-2812-4dd6-86d9-747af02e2b49 admin admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 10:55:49.311483 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-4ff8a471-2812-4dd6-86d9-747af02e2b49 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '82389a28a28d4694bdaab21524f172b8', 'request_id': 'req-4ff8a471-2812-4dd6-86d9-747af02e2b49'} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 10:55:49.318630 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-4ff8a471-2812-4dd6-86d9-747af02e2b49 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '82389a28a28d4694bdaab21524f172b8', 'request_id': 'req-4ff8a471-2812-4dd6-86d9-747af02e2b49'} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 10:55:49.326843 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-4ff8a471-2812-4dd6-86d9-747af02e2b49 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '82389a28a28d4694bdaab21524f172b8', 'request_id': 'req-4ff8a471-2812-4dd6-86d9-747af02e2b49'} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 10:55:49.340934 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-4ff8a471-2812-4dd6-86d9-747af02e2b49 admin admin] The token cache is being invalidate because project 82389a28a28d4694bdaab21524f172b8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80143) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 07 10:55:49.341495 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-4ff8a471-2812-4dd6-86d9-747af02e2b49 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 10:55:49.342330 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-4ff8a471-2812-4dd6-86d9-747af02e2b49 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 10:55:49.343948 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1029/2086] 10.222.0.25 () {58 vars in 1233 bytes} [Sun Aug 7 10:55:49 2022] DELETE /identity/v3/projects/82389a28a28d4694bdaab21524f172b8 => generated 0 bytes in 136 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 07 10:55:49.348050 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d33af56-9fd6-4f6e-8761-e20db0e734d0 service cinder] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 10:55:49.359966 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d33af56-9fd6-4f6e-8761-e20db0e734d0 service cinder] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 10:55:49.436808 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5cbd535-d116-42a0-9316-1ca4871a2901 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5ac47e3b09f43d19a23a0f457fe9e23', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 10:55:49.437539 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5cbd535-d116-42a0-9316-1ca4871a2901 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:55:49.437829 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5cbd535-d116-42a0-9316-1ca4871a2901 admin admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:55:49.438019 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5cbd535-d116-42a0-9316-1ca4871a2901 admin admin] PATH_INFO: `/v3/users/44ea8851cb11486fb73bd4bef0d113aa` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:55:49.439154 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5cbd535-d116-42a0-9316-1ca4871a2901 admin admin] RBAC: Authorizing `identity:delete_user(user_id=44ea8851cb11486fb73bd4bef0d113aa)` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 10:55:49.441070 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5cbd535-d116-42a0-9316-1ca4871a2901 admin admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 10:55:49.442870 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=100 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 10:55:49.506574 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.notifications [None req-b5cbd535-d116-42a0-9316-1ca4871a2901 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '44ea8851cb11486fb73bd4bef0d113aa'} {{(pid=80144) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 10:55:49.514762 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.notifications [None req-b5cbd535-d116-42a0-9316-1ca4871a2901 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '44ea8851cb11486fb73bd4bef0d113aa'} {{(pid=80144) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 10:55:49.528186 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.notifications [None req-b5cbd535-d116-42a0-9316-1ca4871a2901 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '44ea8851cb11486fb73bd4bef0d113aa'} {{(pid=80144) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 10:55:49.529088 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.notifications [None req-b5cbd535-d116-42a0-9316-1ca4871a2901 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '44ea8851cb11486fb73bd4bef0d113aa'} {{(pid=80144) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 10:55:49.531101 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.notifications [None req-b5cbd535-d116-42a0-9316-1ca4871a2901 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '44ea8851cb11486fb73bd4bef0d113aa'} {{(pid=80144) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 10:55:49.552155 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 1058/2087] 10.222.0.25 () {58 vars in 1224 bytes} [Sun Aug 7 10:55:49 2022] DELETE /identity/v3/users/44ea8851cb11486fb73bd4bef0d113aa => generated 0 bytes in 205 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 07 10:55:49.868758 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ff8a471-2812-4dd6-86d9-747af02e2b49 admin admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 10:55:49.870261 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-4ff8a471-2812-4dd6-86d9-747af02e2b49 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 10:55:49.883935 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ff8a471-2812-4dd6-86d9-747af02e2b49 admin admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 10:55:49.916868 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=86,keystone:DELETE=12,keystone:INSERT=2 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 10:55:49.948822 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b76eafce-345f-49a6-8be4-f0a834224bed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5ac47e3b09f43d19a23a0f457fe9e23', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 10:55:49.950459 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b76eafce-345f-49a6-8be4-f0a834224bed admin admin] REQUEST_METHOD: `DELETE` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:55:49.950740 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b76eafce-345f-49a6-8be4-f0a834224bed admin admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:55:49.952420 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b76eafce-345f-49a6-8be4-f0a834224bed admin admin] PATH_INFO: `/v3/projects/3e0ffc2e341c4b20a568b649b3294ac1` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:55:49.952420 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b76eafce-345f-49a6-8be4-f0a834224bed admin admin] RBAC: Authorizing `identity:delete_project(project_id=3e0ffc2e341c4b20a568b649b3294ac1)` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 10:55:49.954292 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b76eafce-345f-49a6-8be4-f0a834224bed admin admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 10:55:50.031108 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-b76eafce-345f-49a6-8be4-f0a834224bed admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3e0ffc2e341c4b20a568b649b3294ac1', 'request_id': 'req-b76eafce-345f-49a6-8be4-f0a834224bed'} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 10:55:50.051137 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-b76eafce-345f-49a6-8be4-f0a834224bed admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3e0ffc2e341c4b20a568b649b3294ac1', 'request_id': 'req-b76eafce-345f-49a6-8be4-f0a834224bed'} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 10:55:50.058587 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-b76eafce-345f-49a6-8be4-f0a834224bed admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3e0ffc2e341c4b20a568b649b3294ac1', 'request_id': 'req-b76eafce-345f-49a6-8be4-f0a834224bed'} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 10:55:50.087440 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-b76eafce-345f-49a6-8be4-f0a834224bed admin admin] The token cache is being invalidate because project 3e0ffc2e341c4b20a568b649b3294ac1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80143) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 07 10:55:50.087837 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-b76eafce-345f-49a6-8be4-f0a834224bed admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 10:55:50.088836 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-b76eafce-345f-49a6-8be4-f0a834224bed admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 10:55:50.090884 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1030/2088] 10.222.0.25 () {58 vars in 1233 bytes} [Sun Aug 7 10:55:49 2022] DELETE /identity/v3/projects/3e0ffc2e341c4b20a568b649b3294ac1 => generated 0 bytes in 223 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 07 10:55:50.095944 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1e16355-caa1-411d-98ac-6219abfea9b6 None None] REQUEST_METHOD: `POST` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:55:50.096306 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1e16355-caa1-411d-98ac-6219abfea9b6 None None] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:55:50.096565 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1e16355-caa1-411d-98ac-6219abfea9b6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:55:50.124526 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.auth.core [None req-f1e16355-caa1-411d-98ac-6219abfea9b6 None None] MFA Rules not processed for user `bac4e6b76fe64217ac8fa44e71cffe6a`. Rule list: `[]` (Enabled: `True`). {{(pid=80144) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 10:55:50.125857 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.fernet_utils [None req-f1e16355-caa1-411d-98ac-6219abfea9b6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80144) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 10:55:50.273975 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 1059/2089] 10.222.0.25 () {58 vars in 958 bytes} [Sun Aug 7 10:55:50 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 179 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 07 10:55:50.293813 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f91a7eee-4f87-452e-b966-b18a9e8e5531 None None] REQUEST_METHOD: `POST` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:55:50.295386 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f91a7eee-4f87-452e-b966-b18a9e8e5531 None None] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:55:50.296443 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f91a7eee-4f87-452e-b966-b18a9e8e5531 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:55:50.330197 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.auth.core [None req-f91a7eee-4f87-452e-b966-b18a9e8e5531 None None] MFA Rules not processed for user `bac4e6b76fe64217ac8fa44e71cffe6a`. Rule list: `[]` (Enabled: `True`). {{(pid=80143) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 10:55:50.337695 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-f91a7eee-4f87-452e-b966-b18a9e8e5531 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 10:55:50.343564 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1031/2090] 10.222.0.25 () {58 vars in 958 bytes} [Sun Aug 7 10:55:50 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 62 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 07 10:55:50.348377 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1e16355-caa1-411d-98ac-6219abfea9b6 None None] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 10:55:50.372040 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1e16355-caa1-411d-98ac-6219abfea9b6 None None] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 10:55:50.378342 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bc9b6ff-3152-437b-9715-30d88ed7ebe1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5ac47e3b09f43d19a23a0f457fe9e23', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 10:55:50.383658 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bc9b6ff-3152-437b-9715-30d88ed7ebe1 admin admin] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:55:50.383977 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bc9b6ff-3152-437b-9715-30d88ed7ebe1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:55:50.384267 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bc9b6ff-3152-437b-9715-30d88ed7ebe1 admin admin] PATH_INFO: `/v3/domains` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:55:50.384990 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bc9b6ff-3152-437b-9715-30d88ed7ebe1 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80144) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 10:55:50.385277 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bc9b6ff-3152-437b-9715-30d88ed7ebe1 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 10:55:50.387041 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bc9b6ff-3152-437b-9715-30d88ed7ebe1 admin admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 10:55:50.400056 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 1060/2091] 10.222.0.25 () {58 vars in 1153 bytes} [Sun Aug 7 10:55:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 52 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 10:55:50.407367 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f91a7eee-4f87-452e-b966-b18a9e8e5531 None None] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 10:55:50.435194 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f91a7eee-4f87-452e-b966-b18a9e8e5531 None None] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 10:55:50.442907 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-284e6eba-3292-4757-ab36-e164ed1c4cba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5ac47e3b09f43d19a23a0f457fe9e23', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 10:55:50.443686 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-284e6eba-3292-4757-ab36-e164ed1c4cba admin admin] REQUEST_METHOD: `POST` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:55:50.443973 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-284e6eba-3292-4757-ab36-e164ed1c4cba admin admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:55:50.444253 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-284e6eba-3292-4757-ab36-e164ed1c4cba admin admin] PATH_INFO: `/v3/projects` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:55:50.444908 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-284e6eba-3292-4757-ab36-e164ed1c4cba admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 10:55:50.446899 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-284e6eba-3292-4757-ab36-e164ed1c4cba admin admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 10:55:50.492650 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1032/2092] 10.222.0.25 () {60 vars in 1153 bytes} [Sun Aug 7 10:55:50 2022] POST /identity/v3/projects => generated 401 bytes in 86 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 07 10:55:50.499188 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bc9b6ff-3152-437b-9715-30d88ed7ebe1 admin admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 10:55:50.513747 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bc9b6ff-3152-437b-9715-30d88ed7ebe1 admin admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 10:55:50.625061 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a5884fc-ccbe-48ac-b090-df543aa20c01 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5ac47e3b09f43d19a23a0f457fe9e23', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 10:55:50.626893 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a5884fc-ccbe-48ac-b090-df543aa20c01 admin admin] REQUEST_METHOD: `POST` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:55:50.627659 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a5884fc-ccbe-48ac-b090-df543aa20c01 admin admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:55:50.628035 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a5884fc-ccbe-48ac-b090-df543aa20c01 admin admin] PATH_INFO: `/v3/users` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:55:50.630237 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a5884fc-ccbe-48ac-b090-df543aa20c01 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 10:55:50.635108 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a5884fc-ccbe-48ac-b090-df543aa20c01 admin admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 10:55:50.649686 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=86,keystone:DELETE=10,keystone:INSERT=4 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 10:55:50.663740 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 1061/2093] 10.222.0.25 () {60 vars in 1144 bytes} [Sun Aug 7 10:55:50 2022] POST /identity/v3/users => generated 301 bytes in 165 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 07 10:55:50.668483 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-284e6eba-3292-4757-ab36-e164ed1c4cba admin admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 10:55:50.683347 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-284e6eba-3292-4757-ab36-e164ed1c4cba admin admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 10:55:50.689187 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1758aee5-db05-4801-bc7b-b93bd28a7757 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5ac47e3b09f43d19a23a0f457fe9e23', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 10:55:50.689795 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1758aee5-db05-4801-bc7b-b93bd28a7757 admin admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:55:50.690073 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1758aee5-db05-4801-bc7b-b93bd28a7757 admin admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:55:50.690375 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1758aee5-db05-4801-bc7b-b93bd28a7757 admin admin] PATH_INFO: `/v3/roles` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:55:50.691058 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1758aee5-db05-4801-bc7b-b93bd28a7757 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 10:55:50.692692 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1758aee5-db05-4801-bc7b-b93bd28a7757 admin admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 10:55:50.704035 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1033/2094] 10.222.0.25 () {58 vars in 1122 bytes} [Sun Aug 7 10:55:50 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 07 10:55:50.708240 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a5884fc-ccbe-48ac-b090-df543aa20c01 admin admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 10:55:50.719784 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a5884fc-ccbe-48ac-b090-df543aa20c01 admin admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 10:55:50.724663 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df39f772-3767-4011-a824-35cd585d2dac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5ac47e3b09f43d19a23a0f457fe9e23', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 10:55:50.725578 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df39f772-3767-4011-a824-35cd585d2dac admin admin] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:55:50.725941 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df39f772-3767-4011-a824-35cd585d2dac admin admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:55:50.726284 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df39f772-3767-4011-a824-35cd585d2dac admin admin] PATH_INFO: `/v3/roles` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:55:50.727254 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df39f772-3767-4011-a824-35cd585d2dac admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 10:55:50.729263 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df39f772-3767-4011-a824-35cd585d2dac admin admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 10:55:50.742974 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 1062/2095] 10.222.0.25 () {58 vars in 1122 bytes} [Sun Aug 7 10:55:50 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 07 10:55:50.745606 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1758aee5-db05-4801-bc7b-b93bd28a7757 admin admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 10:55:50.755713 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1758aee5-db05-4801-bc7b-b93bd28a7757 admin admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 10:55:50.760682 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf308510-a1f4-4844-a5ce-b51aa1c1b760 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5ac47e3b09f43d19a23a0f457fe9e23', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 10:55:50.761332 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf308510-a1f4-4844-a5ce-b51aa1c1b760 admin admin] REQUEST_METHOD: `PUT` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:55:50.761615 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf308510-a1f4-4844-a5ce-b51aa1c1b760 admin admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:55:50.761893 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf308510-a1f4-4844-a5ce-b51aa1c1b760 admin admin] PATH_INFO: `/v3/projects/4c6cac93794d4adfa32a7e90db7d8658/users/9f26c60fbcca483ebeb9f8ff41a3361d/roles/b98dfb3b9fee4644af2887a3ac3fd2e6` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:55:50.776541 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf308510-a1f4-4844-a5ce-b51aa1c1b760 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4c6cac93794d4adfa32a7e90db7d8658, user_id=9f26c60fbcca483ebeb9f8ff41a3361d, role_id=b98dfb3b9fee4644af2887a3ac3fd2e6)` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 10:55:50.779796 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf308510-a1f4-4844-a5ce-b51aa1c1b760 admin admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 10:55:50.793889 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1034/2096] 10.222.0.25 () {60 vars in 1483 bytes} [Sun Aug 7 10:55:50 2022] PUT /identity/v3/projects/4c6cac93794d4adfa32a7e90db7d8658/users/9f26c60fbcca483ebeb9f8ff41a3361d/roles/b98dfb3b9fee4644af2887a3ac3fd2e6 => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 07 10:55:50.804614 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dff43fd1-4053-42d5-af8b-812a46ee3228 None None] REQUEST_METHOD: `POST` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:55:50.804905 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dff43fd1-4053-42d5-af8b-812a46ee3228 None None] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:55:50.805183 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dff43fd1-4053-42d5-af8b-812a46ee3228 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:55:50.838311 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.auth.core [None req-dff43fd1-4053-42d5-af8b-812a46ee3228 None None] MFA Rules not processed for user `9f26c60fbcca483ebeb9f8ff41a3361d`. Rule list: `[]` (Enabled: `True`). {{(pid=80144) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 10:55:50.839569 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.fernet_utils [None req-dff43fd1-4053-42d5-af8b-812a46ee3228 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80144) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 10:55:50.917960 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 1063/2097] 10.222.0.25 () {58 vars in 958 bytes} [Sun Aug 7 10:55:50 2022] POST /identity/v3/auth/tokens => generated 3278 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 07 10:55:50.923187 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf308510-a1f4-4844-a5ce-b51aa1c1b760 admin admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 10:55:50.992854 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf308510-a1f4-4844-a5ce-b51aa1c1b760 admin admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 10:55:50.997509 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e030818-497b-4acd-a66d-052312cf1d1b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5ac47e3b09f43d19a23a0f457fe9e23', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 10:55:50.998666 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e030818-497b-4acd-a66d-052312cf1d1b admin admin] REQUEST_METHOD: `POST` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:55:50.998970 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e030818-497b-4acd-a66d-052312cf1d1b admin admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:55:50.999244 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e030818-497b-4acd-a66d-052312cf1d1b admin admin] PATH_INFO: `/v3/projects` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:55:50.999875 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e030818-497b-4acd-a66d-052312cf1d1b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 10:55:51.001758 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e030818-497b-4acd-a66d-052312cf1d1b admin admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 10:55:51.007106 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 10:55:51.054782 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1035/2098] 10.222.0.25 () {60 vars in 1153 bytes} [Sun Aug 7 10:55:50 2022] POST /identity/v3/projects => generated 401 bytes in 132 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 07 10:55:51.058905 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dff43fd1-4053-42d5-af8b-812a46ee3228 None None] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 10:55:51.073557 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dff43fd1-4053-42d5-af8b-812a46ee3228 None None] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 10:55:51.136882 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5712888-3e5d-46b2-ba84-ca151a61a1cf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5ac47e3b09f43d19a23a0f457fe9e23', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 10:55:51.137812 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5712888-3e5d-46b2-ba84-ca151a61a1cf admin admin] REQUEST_METHOD: `POST` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:55:51.138314 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5712888-3e5d-46b2-ba84-ca151a61a1cf admin admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:55:51.139240 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5712888-3e5d-46b2-ba84-ca151a61a1cf admin admin] PATH_INFO: `/v3/users` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:55:51.140185 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5712888-3e5d-46b2-ba84-ca151a61a1cf admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 10:55:51.142423 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5712888-3e5d-46b2-ba84-ca151a61a1cf admin admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 10:55:51.166750 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 1064/2099] 10.222.0.25 () {60 vars in 1144 bytes} [Sun Aug 7 10:55:51 2022] POST /identity/v3/users => generated 307 bytes in 108 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 07 10:55:51.171926 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e030818-497b-4acd-a66d-052312cf1d1b admin admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 10:55:51.181988 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e030818-497b-4acd-a66d-052312cf1d1b admin admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 10:55:51.186892 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-feb3f7b8-7a0f-4050-957b-b35ca43f15ca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5ac47e3b09f43d19a23a0f457fe9e23', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 10:55:51.187632 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feb3f7b8-7a0f-4050-957b-b35ca43f15ca admin admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:55:51.187923 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feb3f7b8-7a0f-4050-957b-b35ca43f15ca admin admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:55:51.188196 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feb3f7b8-7a0f-4050-957b-b35ca43f15ca admin admin] PATH_INFO: `/v3/roles` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:55:51.188832 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-feb3f7b8-7a0f-4050-957b-b35ca43f15ca admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 10:55:51.190473 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-feb3f7b8-7a0f-4050-957b-b35ca43f15ca admin admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 10:55:51.201506 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1036/2100] 10.222.0.25 () {58 vars in 1122 bytes} [Sun Aug 7 10:55:51 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 07 10:55:51.205494 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5712888-3e5d-46b2-ba84-ca151a61a1cf admin admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 10:55:51.218426 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5712888-3e5d-46b2-ba84-ca151a61a1cf admin admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 10:55:51.223705 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fe0d273-949c-4eff-a8d3-7c4cd8a92164 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5ac47e3b09f43d19a23a0f457fe9e23', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 10:55:51.224371 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fe0d273-949c-4eff-a8d3-7c4cd8a92164 admin admin] REQUEST_METHOD: `PUT` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:55:51.224658 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fe0d273-949c-4eff-a8d3-7c4cd8a92164 admin admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:55:51.224934 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fe0d273-949c-4eff-a8d3-7c4cd8a92164 admin admin] PATH_INFO: `/v3/projects/987cc28bfd3545d3bc320a116b8271bc/users/2e539e8834a149e2a214a5bd595d8b97/roles/81b1f1bcee2a473088fbbd327c1c5b23` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:55:51.242660 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fe0d273-949c-4eff-a8d3-7c4cd8a92164 admin admin] RBAC: Authorizing `identity:create_grant(project_id=987cc28bfd3545d3bc320a116b8271bc, user_id=2e539e8834a149e2a214a5bd595d8b97, role_id=81b1f1bcee2a473088fbbd327c1c5b23)` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 10:55:51.245447 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fe0d273-949c-4eff-a8d3-7c4cd8a92164 admin admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 10:55:51.268371 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 1065/2101] 10.222.0.25 () {60 vars in 1483 bytes} [Sun Aug 7 10:55:51 2022] PUT /identity/v3/projects/987cc28bfd3545d3bc320a116b8271bc/users/2e539e8834a149e2a214a5bd595d8b97/roles/81b1f1bcee2a473088fbbd327c1c5b23 => generated 0 bytes in 63 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 07 10:55:51.272122 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-feb3f7b8-7a0f-4050-957b-b35ca43f15ca admin admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 10:55:51.283481 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-feb3f7b8-7a0f-4050-957b-b35ca43f15ca admin admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 10:55:51.353495 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23516a82-e237-48fd-a8a2-3c43b5a08ee8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5ac47e3b09f43d19a23a0f457fe9e23', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 10:55:51.354154 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23516a82-e237-48fd-a8a2-3c43b5a08ee8 admin admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:55:51.354396 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23516a82-e237-48fd-a8a2-3c43b5a08ee8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:55:51.354673 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23516a82-e237-48fd-a8a2-3c43b5a08ee8 admin admin] PATH_INFO: `/v3/roles` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:55:51.355357 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23516a82-e237-48fd-a8a2-3c43b5a08ee8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 10:55:51.356941 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23516a82-e237-48fd-a8a2-3c43b5a08ee8 admin admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 10:55:51.371725 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1037/2102] 10.222.0.25 () {58 vars in 1122 bytes} [Sun Aug 7 10:55:51 2022] GET /identity/v3/roles => generated 1420 bytes in 97 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 07 10:55:51.372920 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fe0d273-949c-4eff-a8d3-7c4cd8a92164 admin admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 10:55:51.383586 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fe0d273-949c-4eff-a8d3-7c4cd8a92164 admin admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 10:55:51.388307 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61c57a95-5ad7-40f4-9653-8ca5c86d9a8a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5ac47e3b09f43d19a23a0f457fe9e23', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 10:55:51.389679 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61c57a95-5ad7-40f4-9653-8ca5c86d9a8a admin admin] REQUEST_METHOD: `PUT` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:55:51.390414 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61c57a95-5ad7-40f4-9653-8ca5c86d9a8a admin admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:55:51.390742 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61c57a95-5ad7-40f4-9653-8ca5c86d9a8a admin admin] PATH_INFO: `/v3/projects/987cc28bfd3545d3bc320a116b8271bc/users/2e539e8834a149e2a214a5bd595d8b97/roles/81b1f1bcee2a473088fbbd327c1c5b23` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:55:51.393753 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61c57a95-5ad7-40f4-9653-8ca5c86d9a8a admin admin] RBAC: Authorizing `identity:create_grant(project_id=987cc28bfd3545d3bc320a116b8271bc, user_id=2e539e8834a149e2a214a5bd595d8b97, role_id=81b1f1bcee2a473088fbbd327c1c5b23)` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 10:55:51.399886 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61c57a95-5ad7-40f4-9653-8ca5c86d9a8a admin admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 10:55:51.437634 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 1066/2103] 10.222.0.25 () {60 vars in 1483 bytes} [Sun Aug 7 10:55:51 2022] PUT /identity/v3/projects/987cc28bfd3545d3bc320a116b8271bc/users/2e539e8834a149e2a214a5bd595d8b97/roles/81b1f1bcee2a473088fbbd327c1c5b23 => generated 0 bytes in 65 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 07 10:55:51.448730 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7893a56a-1907-4930-95ae-591f544d0b49 None None] REQUEST_METHOD: `POST` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:55:51.449666 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7893a56a-1907-4930-95ae-591f544d0b49 None None] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:55:51.449957 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7893a56a-1907-4930-95ae-591f544d0b49 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:55:51.502325 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.auth.core [None req-7893a56a-1907-4930-95ae-591f544d0b49 None None] MFA Rules not processed for user `2e539e8834a149e2a214a5bd595d8b97`. Rule list: `[]` (Enabled: `True`). {{(pid=80143) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 10:55:51.504243 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-7893a56a-1907-4930-95ae-591f544d0b49 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 10:55:51.598350 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1038/2104] 10.222.0.25 () {58 vars in 958 bytes} [Sun Aug 7 10:55:51 2022] POST /identity/v3/auth/tokens => generated 3345 bytes in 145 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 07 10:55:51.615419 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61c57a95-5ad7-40f4-9653-8ca5c86d9a8a admin admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 10:55:51.617336 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.fernet_utils [None req-61c57a95-5ad7-40f4-9653-8ca5c86d9a8a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80144) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 10:55:51.692262 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 10:55:51.724493 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61c57a95-5ad7-40f4-9653-8ca5c86d9a8a admin admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 10:55:51.728253 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd4b6a70-b7fc-40c4-9379-4b91ffd89eb7 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88dbd6dfd1164994a3e16da553561910', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6de1f3cac02c4bcdb05dd2337176033d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 10:55:51.728855 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd4b6a70-b7fc-40c4-9379-4b91ffd89eb7 service cinder] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:55:51.729139 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd4b6a70-b7fc-40c4-9379-4b91ffd89eb7 service cinder] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:55:51.729415 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd4b6a70-b7fc-40c4-9379-4b91ffd89eb7 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:55:51.739352 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd4b6a70-b7fc-40c4-9379-4b91ffd89eb7 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 10:55:51.740775 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd4b6a70-b7fc-40c4-9379-4b91ffd89eb7 service cinder] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 10:55:51.751289 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 1067/2105] 10.222.0.25 () {60 vars in 1365 bytes} [Sun Aug 7 10:55:51 2022] GET /identity/v3/auth/tokens => generated 3345 bytes in 136 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 07 10:55:51.831406 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7893a56a-1907-4930-95ae-591f544d0b49 None None] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 10:55:51.841366 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7893a56a-1907-4930-95ae-591f544d0b49 None None] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 10:55:51.846303 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a49bf209-5131-4218-9309-7884f7634fd0 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88dbd6dfd1164994a3e16da553561910', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6de1f3cac02c4bcdb05dd2337176033d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 10:55:51.846948 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a49bf209-5131-4218-9309-7884f7634fd0 service cinder] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:55:51.847221 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a49bf209-5131-4218-9309-7884f7634fd0 service cinder] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:55:51.847492 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a49bf209-5131-4218-9309-7884f7634fd0 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:55:51.908865 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 10:55:51.911579 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a49bf209-5131-4218-9309-7884f7634fd0 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 10:55:51.913220 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a49bf209-5131-4218-9309-7884f7634fd0 service cinder] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 10:55:51.924149 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1039/2106] 10.222.0.25 () {60 vars in 1365 bytes} [Sun Aug 7 10:55:51 2022] GET /identity/v3/auth/tokens => generated 3278 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 07 10:55:54.300951 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd4b6a70-b7fc-40c4-9379-4b91ffd89eb7 service cinder] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 10:55:54.302458 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.fernet_utils [None req-fd4b6a70-b7fc-40c4-9379-4b91ffd89eb7 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80144) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 10:55:54.378518 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd4b6a70-b7fc-40c4-9379-4b91ffd89eb7 service cinder] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 10:55:54.384560 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb7c666c-6046-4695-88e5-aaf8d9f7abde admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5ac47e3b09f43d19a23a0f457fe9e23', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 10:55:54.385804 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb7c666c-6046-4695-88e5-aaf8d9f7abde admin admin] REQUEST_METHOD: `DELETE` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:55:54.386082 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb7c666c-6046-4695-88e5-aaf8d9f7abde admin admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:55:54.386395 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb7c666c-6046-4695-88e5-aaf8d9f7abde admin admin] PATH_INFO: `/v3/users/79778b8885304daa940b3d03f7c3a862` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:55:54.388004 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb7c666c-6046-4695-88e5-aaf8d9f7abde admin admin] RBAC: Authorizing `identity:delete_user(user_id=79778b8885304daa940b3d03f7c3a862)` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 10:55:54.389946 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb7c666c-6046-4695-88e5-aaf8d9f7abde admin admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 10:55:54.458320 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.notifications [None req-eb7c666c-6046-4695-88e5-aaf8d9f7abde admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '79778b8885304daa940b3d03f7c3a862'} {{(pid=80144) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 10:55:54.471760 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.notifications [None req-eb7c666c-6046-4695-88e5-aaf8d9f7abde admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '79778b8885304daa940b3d03f7c3a862'} {{(pid=80144) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 10:55:54.483942 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.notifications [None req-eb7c666c-6046-4695-88e5-aaf8d9f7abde admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '79778b8885304daa940b3d03f7c3a862'} {{(pid=80144) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 10:55:54.484621 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.notifications [None req-eb7c666c-6046-4695-88e5-aaf8d9f7abde admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '79778b8885304daa940b3d03f7c3a862'} {{(pid=80144) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 10:55:54.485242 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.notifications [None req-eb7c666c-6046-4695-88e5-aaf8d9f7abde admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '79778b8885304daa940b3d03f7c3a862'} {{(pid=80144) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 10:55:54.534634 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cc4c442-2ada-4b9a-9e04-a60a3ed81c2f None None] REQUEST_METHOD: `POST` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:55:54.534634 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cc4c442-2ada-4b9a-9e04-a60a3ed81c2f None None] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:55:54.534634 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cc4c442-2ada-4b9a-9e04-a60a3ed81c2f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:55:54.535270 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 1068/2107] 10.222.0.25 () {58 vars in 1224 bytes} [Sun Aug 7 10:55:54 2022] DELETE /identity/v3/users/79778b8885304daa940b3d03f7c3a862 => generated 0 bytes in 222 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 07 10:55:54.567753 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.auth.core [None req-0cc4c442-2ada-4b9a-9e04-a60a3ed81c2f None None] MFA Rules not processed for user `bac4e6b76fe64217ac8fa44e71cffe6a`. Rule list: `[]` (Enabled: `True`). {{(pid=80143) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 10:55:54.569910 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-0cc4c442-2ada-4b9a-9e04-a60a3ed81c2f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 10:55:54.689001 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1040/2108] 10.222.0.25 () {58 vars in 958 bytes} [Sun Aug 7 10:55:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 07 10:55:54.698996 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb7c666c-6046-4695-88e5-aaf8d9f7abde admin admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 10:55:54.700455 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.fernet_utils [None req-eb7c666c-6046-4695-88e5-aaf8d9f7abde admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80144) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 10:55:54.783862 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 10:55:54.788535 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb7c666c-6046-4695-88e5-aaf8d9f7abde admin admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 10:55:54.800963 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff67fc25-4e72-4eec-8cb8-dd6262a37a16 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be1283c9bf47424c9aa3ad27cab03027', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6de1f3cac02c4bcdb05dd2337176033d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 10:55:54.802148 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff67fc25-4e72-4eec-8cb8-dd6262a37a16 service neutron] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:55:54.802771 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff67fc25-4e72-4eec-8cb8-dd6262a37a16 service neutron] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:55:54.803375 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff67fc25-4e72-4eec-8cb8-dd6262a37a16 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:55:54.817045 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff67fc25-4e72-4eec-8cb8-dd6262a37a16 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 10:55:54.818732 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff67fc25-4e72-4eec-8cb8-dd6262a37a16 service neutron] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 10:55:54.830632 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 1069/2109] 10.222.0.25 () {60 vars in 1365 bytes} [Sun Aug 7 10:55:54 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 132 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 07 10:55:55.145959 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cc4c442-2ada-4b9a-9e04-a60a3ed81c2f None None] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 10:55:55.147950 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-0cc4c442-2ada-4b9a-9e04-a60a3ed81c2f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 10:55:55.161017 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cc4c442-2ada-4b9a-9e04-a60a3ed81c2f None None] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 10:55:55.168719 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9510f32a-0c6a-4eae-affe-899ed369bc3e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5ac47e3b09f43d19a23a0f457fe9e23', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 10:55:55.168719 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9510f32a-0c6a-4eae-affe-899ed369bc3e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:55:55.168719 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9510f32a-0c6a-4eae-affe-899ed369bc3e admin admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:55:55.168719 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9510f32a-0c6a-4eae-affe-899ed369bc3e admin admin] PATH_INFO: `/v3/projects/23533bbdae174017bf68eb3072591809` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:55:55.169414 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9510f32a-0c6a-4eae-affe-899ed369bc3e admin admin] RBAC: Authorizing `identity:delete_project(project_id=23533bbdae174017bf68eb3072591809)` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 10:55:55.172401 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9510f32a-0c6a-4eae-affe-899ed369bc3e admin admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 10:55:55.243519 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-9510f32a-0c6a-4eae-affe-899ed369bc3e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '23533bbdae174017bf68eb3072591809', 'request_id': 'req-9510f32a-0c6a-4eae-affe-899ed369bc3e'} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 10:55:55.251280 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-9510f32a-0c6a-4eae-affe-899ed369bc3e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '23533bbdae174017bf68eb3072591809', 'request_id': 'req-9510f32a-0c6a-4eae-affe-899ed369bc3e'} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 10:55:55.260402 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-9510f32a-0c6a-4eae-affe-899ed369bc3e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '23533bbdae174017bf68eb3072591809', 'request_id': 'req-9510f32a-0c6a-4eae-affe-899ed369bc3e'} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 10:55:55.278759 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-9510f32a-0c6a-4eae-affe-899ed369bc3e admin admin] The token cache is being invalidate because project 23533bbdae174017bf68eb3072591809 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80143) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 07 10:55:55.279127 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-9510f32a-0c6a-4eae-affe-899ed369bc3e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 10:55:55.279955 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-9510f32a-0c6a-4eae-affe-899ed369bc3e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 10:55:55.284586 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1041/2110] 10.222.0.25 () {58 vars in 1233 bytes} [Sun Aug 7 10:55:55 2022] DELETE /identity/v3/projects/23533bbdae174017bf68eb3072591809 => generated 0 bytes in 136 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 07 10:55:55.285710 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff67fc25-4e72-4eec-8cb8-dd6262a37a16 service neutron] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 10:55:55.316156 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff67fc25-4e72-4eec-8cb8-dd6262a37a16 service neutron] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 10:55:55.403906 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c3c86eb-6ee6-4dd8-b983-60e9d9546e46 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5ac47e3b09f43d19a23a0f457fe9e23', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 10:55:55.408709 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c3c86eb-6ee6-4dd8-b983-60e9d9546e46 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:55:55.409114 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c3c86eb-6ee6-4dd8-b983-60e9d9546e46 admin admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:55:55.409286 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c3c86eb-6ee6-4dd8-b983-60e9d9546e46 admin admin] PATH_INFO: `/v3/users/de0b670616264a1aa6b1269240cdf67a` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:55:55.410766 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c3c86eb-6ee6-4dd8-b983-60e9d9546e46 admin admin] RBAC: Authorizing `identity:delete_user(user_id=de0b670616264a1aa6b1269240cdf67a)` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 10:55:55.416763 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c3c86eb-6ee6-4dd8-b983-60e9d9546e46 admin admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 10:55:55.482714 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.notifications [None req-5c3c86eb-6ee6-4dd8-b983-60e9d9546e46 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'de0b670616264a1aa6b1269240cdf67a'} {{(pid=80144) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 10:55:55.490528 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.notifications [None req-5c3c86eb-6ee6-4dd8-b983-60e9d9546e46 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'de0b670616264a1aa6b1269240cdf67a'} {{(pid=80144) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 10:55:55.499392 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.notifications [None req-5c3c86eb-6ee6-4dd8-b983-60e9d9546e46 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'de0b670616264a1aa6b1269240cdf67a'} {{(pid=80144) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 10:55:55.500110 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.notifications [None req-5c3c86eb-6ee6-4dd8-b983-60e9d9546e46 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'de0b670616264a1aa6b1269240cdf67a'} {{(pid=80144) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 10:55:55.500788 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.notifications [None req-5c3c86eb-6ee6-4dd8-b983-60e9d9546e46 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'de0b670616264a1aa6b1269240cdf67a'} {{(pid=80144) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 10:55:55.525727 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 1070/2111] 10.222.0.25 () {58 vars in 1224 bytes} [Sun Aug 7 10:55:55 2022] DELETE /identity/v3/users/de0b670616264a1aa6b1269240cdf67a => generated 0 bytes in 237 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 07 10:55:55.912083 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9510f32a-0c6a-4eae-affe-899ed369bc3e admin admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 10:55:55.916349 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-9510f32a-0c6a-4eae-affe-899ed369bc3e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 10:55:55.946713 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9510f32a-0c6a-4eae-affe-899ed369bc3e admin admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 10:55:56.019882 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e11f835d-8d48-4b3e-801a-036ac455aee1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5ac47e3b09f43d19a23a0f457fe9e23', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 10:55:56.020642 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e11f835d-8d48-4b3e-801a-036ac455aee1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:55:56.020928 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e11f835d-8d48-4b3e-801a-036ac455aee1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:55:56.021202 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e11f835d-8d48-4b3e-801a-036ac455aee1 admin admin] PATH_INFO: `/v3/projects/c3c499c3f7d642f0a4fb5f98489c8750` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:55:56.022323 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e11f835d-8d48-4b3e-801a-036ac455aee1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c3c499c3f7d642f0a4fb5f98489c8750)` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 10:55:56.024321 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e11f835d-8d48-4b3e-801a-036ac455aee1 admin admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 10:55:56.058211 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 10:55:56.089711 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-e11f835d-8d48-4b3e-801a-036ac455aee1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c3c499c3f7d642f0a4fb5f98489c8750', 'request_id': 'req-e11f835d-8d48-4b3e-801a-036ac455aee1'} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 10:55:56.095897 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-e11f835d-8d48-4b3e-801a-036ac455aee1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c3c499c3f7d642f0a4fb5f98489c8750', 'request_id': 'req-e11f835d-8d48-4b3e-801a-036ac455aee1'} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 10:55:56.102261 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-e11f835d-8d48-4b3e-801a-036ac455aee1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c3c499c3f7d642f0a4fb5f98489c8750', 'request_id': 'req-e11f835d-8d48-4b3e-801a-036ac455aee1'} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 10:55:56.121878 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-e11f835d-8d48-4b3e-801a-036ac455aee1 admin admin] The token cache is being invalidate because project c3c499c3f7d642f0a4fb5f98489c8750 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80143) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 07 10:55:56.121878 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-e11f835d-8d48-4b3e-801a-036ac455aee1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 10:55:56.122636 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-e11f835d-8d48-4b3e-801a-036ac455aee1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 10:55:56.124610 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1042/2112] 10.222.0.25 () {58 vars in 1233 bytes} [Sun Aug 7 10:55:55 2022] DELETE /identity/v3/projects/c3c499c3f7d642f0a4fb5f98489c8750 => generated 0 bytes in 213 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 07 10:55:56.132024 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c3c86eb-6ee6-4dd8-b983-60e9d9546e46 admin admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 10:55:56.133720 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.fernet_utils [None req-5c3c86eb-6ee6-4dd8-b983-60e9d9546e46 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80144) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 10:55:56.157821 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c3c86eb-6ee6-4dd8-b983-60e9d9546e46 admin admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 10:55:56.255445 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aacf81e4-82ec-4227-a9af-37017d60babb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5ac47e3b09f43d19a23a0f457fe9e23', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 10:55:56.257368 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aacf81e4-82ec-4227-a9af-37017d60babb admin admin] REQUEST_METHOD: `DELETE` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:55:56.257692 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aacf81e4-82ec-4227-a9af-37017d60babb admin admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:55:56.258136 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aacf81e4-82ec-4227-a9af-37017d60babb admin admin] PATH_INFO: `/v3/users/becc5e7ed3d0486b8723495b6fd94cc3` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:55:56.259443 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aacf81e4-82ec-4227-a9af-37017d60babb admin admin] RBAC: Authorizing `identity:delete_user(user_id=becc5e7ed3d0486b8723495b6fd94cc3)` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 10:55:56.261682 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aacf81e4-82ec-4227-a9af-37017d60babb admin admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 10:55:56.276560 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 10:55:56.338381 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.notifications [None req-aacf81e4-82ec-4227-a9af-37017d60babb admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'becc5e7ed3d0486b8723495b6fd94cc3'} {{(pid=80144) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 10:55:56.350618 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.notifications [None req-aacf81e4-82ec-4227-a9af-37017d60babb admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'becc5e7ed3d0486b8723495b6fd94cc3'} {{(pid=80144) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 10:55:56.359513 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.notifications [None req-aacf81e4-82ec-4227-a9af-37017d60babb admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'becc5e7ed3d0486b8723495b6fd94cc3'} {{(pid=80144) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 10:55:56.360195 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.notifications [None req-aacf81e4-82ec-4227-a9af-37017d60babb admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'becc5e7ed3d0486b8723495b6fd94cc3'} {{(pid=80144) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 10:55:56.360964 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.notifications [None req-aacf81e4-82ec-4227-a9af-37017d60babb admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'becc5e7ed3d0486b8723495b6fd94cc3'} {{(pid=80144) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 10:55:56.385173 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 1071/2113] 10.222.0.25 () {58 vars in 1224 bytes} [Sun Aug 7 10:55:56 2022] DELETE /identity/v3/users/becc5e7ed3d0486b8723495b6fd94cc3 => generated 0 bytes in 251 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 07 10:55:56.883862 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e11f835d-8d48-4b3e-801a-036ac455aee1 admin admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 10:55:56.885282 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-e11f835d-8d48-4b3e-801a-036ac455aee1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 10:55:56.900809 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e11f835d-8d48-4b3e-801a-036ac455aee1 admin admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 10:55:56.971566 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0860d457-2887-406a-8720-0a8a42591d67 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5ac47e3b09f43d19a23a0f457fe9e23', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 10:55:56.972804 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0860d457-2887-406a-8720-0a8a42591d67 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:55:56.973097 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0860d457-2887-406a-8720-0a8a42591d67 admin admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:55:56.973374 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0860d457-2887-406a-8720-0a8a42591d67 admin admin] PATH_INFO: `/v3/projects/a010c7af94fe44ac91756ad7878560df` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:55:56.974504 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0860d457-2887-406a-8720-0a8a42591d67 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a010c7af94fe44ac91756ad7878560df)` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 10:55:56.976411 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0860d457-2887-406a-8720-0a8a42591d67 admin admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 10:55:57.023102 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-0860d457-2887-406a-8720-0a8a42591d67 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a010c7af94fe44ac91756ad7878560df', 'request_id': 'req-0860d457-2887-406a-8720-0a8a42591d67'} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 10:55:57.028553 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-0860d457-2887-406a-8720-0a8a42591d67 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a010c7af94fe44ac91756ad7878560df', 'request_id': 'req-0860d457-2887-406a-8720-0a8a42591d67'} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 10:55:57.035697 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-0860d457-2887-406a-8720-0a8a42591d67 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a010c7af94fe44ac91756ad7878560df', 'request_id': 'req-0860d457-2887-406a-8720-0a8a42591d67'} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 10:55:57.048825 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-0860d457-2887-406a-8720-0a8a42591d67 admin admin] The token cache is being invalidate because project a010c7af94fe44ac91756ad7878560df was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80143) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 07 10:55:57.049191 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-0860d457-2887-406a-8720-0a8a42591d67 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 10:55:57.049873 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-0860d457-2887-406a-8720-0a8a42591d67 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 10:55:57.051610 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1043/2114] 10.222.0.25 () {58 vars in 1233 bytes} [Sun Aug 7 10:55:56 2022] DELETE /identity/v3/projects/a010c7af94fe44ac91756ad7878560df => generated 0 bytes in 168 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 07 10:55:57.056724 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a8835ce-4b67-4050-a851-80e1163bae4c None None] REQUEST_METHOD: `POST` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:55:57.057015 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a8835ce-4b67-4050-a851-80e1163bae4c None None] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:55:57.057297 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a8835ce-4b67-4050-a851-80e1163bae4c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:55:57.077552 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.auth.core [None req-4a8835ce-4b67-4050-a851-80e1163bae4c None None] MFA Rules not processed for user `bac4e6b76fe64217ac8fa44e71cffe6a`. Rule list: `[]` (Enabled: `True`). {{(pid=80144) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 10:55:57.078749 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.fernet_utils [None req-4a8835ce-4b67-4050-a851-80e1163bae4c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80144) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 10:55:57.140509 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 1072/2115] 10.222.0.25 () {58 vars in 958 bytes} [Sun Aug 7 10:55:57 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 07 10:55:57.148523 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dda2eaa3-9224-43d0-a05f-c6f73899afd5 None None] REQUEST_METHOD: `POST` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:55:57.148859 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dda2eaa3-9224-43d0-a05f-c6f73899afd5 None None] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:55:57.149177 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dda2eaa3-9224-43d0-a05f-c6f73899afd5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:55:57.172418 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.auth.core [None req-dda2eaa3-9224-43d0-a05f-c6f73899afd5 None None] MFA Rules not processed for user `bac4e6b76fe64217ac8fa44e71cffe6a`. Rule list: `[]` (Enabled: `True`). {{(pid=80143) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 10:55:57.173614 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-dda2eaa3-9224-43d0-a05f-c6f73899afd5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 10:55:57.181624 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1044/2116] 10.222.0.25 () {58 vars in 958 bytes} [Sun Aug 7 10:55:57 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 07 10:55:57.186707 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a8835ce-4b67-4050-a851-80e1163bae4c None None] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 10:55:57.196626 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a8835ce-4b67-4050-a851-80e1163bae4c None None] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 10:55:57.201173 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a32a048a-52e9-48f0-89fd-345b13b17974 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5ac47e3b09f43d19a23a0f457fe9e23', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 10:55:57.201914 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a32a048a-52e9-48f0-89fd-345b13b17974 admin admin] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:55:57.202192 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a32a048a-52e9-48f0-89fd-345b13b17974 admin admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:55:57.202490 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a32a048a-52e9-48f0-89fd-345b13b17974 admin admin] PATH_INFO: `/v3/domains` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:55:57.203354 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a32a048a-52e9-48f0-89fd-345b13b17974 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80144) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 10:55:57.203687 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a32a048a-52e9-48f0-89fd-345b13b17974 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 10:55:57.205504 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a32a048a-52e9-48f0-89fd-345b13b17974 admin admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 10:55:57.218570 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 1073/2117] 10.222.0.25 () {58 vars in 1153 bytes} [Sun Aug 7 10:55:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 10:55:57.223032 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dda2eaa3-9224-43d0-a05f-c6f73899afd5 None None] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 10:55:57.233693 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dda2eaa3-9224-43d0-a05f-c6f73899afd5 None None] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 10:55:57.238437 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52cf2b18-94fe-4f4c-b8e5-744df1663932 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5ac47e3b09f43d19a23a0f457fe9e23', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 10:55:57.239098 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52cf2b18-94fe-4f4c-b8e5-744df1663932 admin admin] REQUEST_METHOD: `POST` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:55:57.239384 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52cf2b18-94fe-4f4c-b8e5-744df1663932 admin admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:55:57.239661 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52cf2b18-94fe-4f4c-b8e5-744df1663932 admin admin] PATH_INFO: `/v3/projects` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:55:57.240294 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52cf2b18-94fe-4f4c-b8e5-744df1663932 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 10:55:57.242235 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52cf2b18-94fe-4f4c-b8e5-744df1663932 admin admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 10:55:57.273873 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1045/2118] 10.222.0.25 () {60 vars in 1153 bytes} [Sun Aug 7 10:55:57 2022] POST /identity/v3/projects => generated 383 bytes in 47 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 07 10:55:57.275378 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a32a048a-52e9-48f0-89fd-345b13b17974 admin admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 10:55:57.286830 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a32a048a-52e9-48f0-89fd-345b13b17974 admin admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 10:55:57.350592 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41f07241-fd81-4600-a315-8a5a1203110e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5ac47e3b09f43d19a23a0f457fe9e23', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 10:55:57.351399 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41f07241-fd81-4600-a315-8a5a1203110e admin admin] REQUEST_METHOD: `POST` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:55:57.351522 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41f07241-fd81-4600-a315-8a5a1203110e admin admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:55:57.351798 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41f07241-fd81-4600-a315-8a5a1203110e admin admin] PATH_INFO: `/v3/users` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:55:57.352473 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41f07241-fd81-4600-a315-8a5a1203110e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 10:55:57.356104 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41f07241-fd81-4600-a315-8a5a1203110e admin admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 10:55:57.384268 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 1074/2119] 10.222.0.25 () {60 vars in 1144 bytes} [Sun Aug 7 10:55:57 2022] POST /identity/v3/users => generated 292 bytes in 109 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 07 10:55:57.388622 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52cf2b18-94fe-4f4c-b8e5-744df1663932 admin admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 10:55:57.399822 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52cf2b18-94fe-4f4c-b8e5-744df1663932 admin admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 10:55:57.405642 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68f2cd44-2570-4fb7-856c-051d48f6d635 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5ac47e3b09f43d19a23a0f457fe9e23', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 10:55:57.406319 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68f2cd44-2570-4fb7-856c-051d48f6d635 admin admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:55:57.406657 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68f2cd44-2570-4fb7-856c-051d48f6d635 admin admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:55:57.406963 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68f2cd44-2570-4fb7-856c-051d48f6d635 admin admin] PATH_INFO: `/v3/roles` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:55:57.408413 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68f2cd44-2570-4fb7-856c-051d48f6d635 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 10:55:57.410081 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68f2cd44-2570-4fb7-856c-051d48f6d635 admin admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 10:55:57.418589 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:DELETE=14,keystone:SELECT=83,keystone:INSERT=3 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 10:55:57.436000 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1046/2120] 10.222.0.25 () {58 vars in 1122 bytes} [Sun Aug 7 10:55:57 2022] GET /identity/v3/roles => generated 1420 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 07 10:55:57.438265 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41f07241-fd81-4600-a315-8a5a1203110e admin admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 10:55:57.449702 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41f07241-fd81-4600-a315-8a5a1203110e admin admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 10:55:57.455558 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1256fd2e-d1f0-4769-9a04-37ef42791f48 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5ac47e3b09f43d19a23a0f457fe9e23', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 10:55:57.456195 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1256fd2e-d1f0-4769-9a04-37ef42791f48 admin admin] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:55:57.456474 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1256fd2e-d1f0-4769-9a04-37ef42791f48 admin admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:55:57.456746 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1256fd2e-d1f0-4769-9a04-37ef42791f48 admin admin] PATH_INFO: `/v3/roles` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:55:57.457406 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1256fd2e-d1f0-4769-9a04-37ef42791f48 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 10:55:57.459099 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1256fd2e-d1f0-4769-9a04-37ef42791f48 admin admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 10:55:57.461022 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=87,keystone:DELETE=9,keystone:INSERT=4 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 10:55:57.474368 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 1075/2121] 10.222.0.25 () {58 vars in 1122 bytes} [Sun Aug 7 10:55:57 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 07 10:55:57.478464 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68f2cd44-2570-4fb7-856c-051d48f6d635 admin admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 10:55:57.490617 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68f2cd44-2570-4fb7-856c-051d48f6d635 admin admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 10:55:57.498756 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e720c27-f86c-4b39-84fb-bf719cc1f997 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5ac47e3b09f43d19a23a0f457fe9e23', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 10:55:57.499478 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e720c27-f86c-4b39-84fb-bf719cc1f997 admin admin] REQUEST_METHOD: `PUT` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:55:57.499758 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e720c27-f86c-4b39-84fb-bf719cc1f997 admin admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:55:57.500038 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e720c27-f86c-4b39-84fb-bf719cc1f997 admin admin] PATH_INFO: `/v3/projects/3e4147dc53b246f78e50703eb33acce6/users/ae9ef7de6a554145a7f2a651dc3d9a42/roles/b98dfb3b9fee4644af2887a3ac3fd2e6` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:55:57.517160 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e720c27-f86c-4b39-84fb-bf719cc1f997 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3e4147dc53b246f78e50703eb33acce6, user_id=ae9ef7de6a554145a7f2a651dc3d9a42, role_id=b98dfb3b9fee4644af2887a3ac3fd2e6)` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 10:55:57.519915 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e720c27-f86c-4b39-84fb-bf719cc1f997 admin admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 10:55:57.535146 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1047/2122] 10.222.0.25 () {60 vars in 1483 bytes} [Sun Aug 7 10:55:57 2022] PUT /identity/v3/projects/3e4147dc53b246f78e50703eb33acce6/users/ae9ef7de6a554145a7f2a651dc3d9a42/roles/b98dfb3b9fee4644af2887a3ac3fd2e6 => generated 0 bytes in 57 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 07 10:55:57.548281 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb6e5b5c-eb3f-417c-97f3-8a1088331150 None None] REQUEST_METHOD: `POST` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:55:57.548941 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb6e5b5c-eb3f-417c-97f3-8a1088331150 None None] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:55:57.548941 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb6e5b5c-eb3f-417c-97f3-8a1088331150 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:55:57.580354 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.auth.core [None req-cb6e5b5c-eb3f-417c-97f3-8a1088331150 None None] MFA Rules not processed for user `ae9ef7de6a554145a7f2a651dc3d9a42`. Rule list: `[]` (Enabled: `True`). {{(pid=80144) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 10:55:57.581617 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.fernet_utils [None req-cb6e5b5c-eb3f-417c-97f3-8a1088331150 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80144) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 10:55:57.589005 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e720c27-f86c-4b39-84fb-bf719cc1f997 admin admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 10:55:57.590661 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-1e720c27-f86c-4b39-84fb-bf719cc1f997 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 10:55:57.658469 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 1076/2123] 10.222.0.25 () {58 vars in 958 bytes} [Sun Aug 7 10:55:57 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 07 10:55:57.669107 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e720c27-f86c-4b39-84fb-bf719cc1f997 admin admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 10:55:57.672542 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb6e5b5c-eb3f-417c-97f3-8a1088331150 None None] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 10:55:57.674487 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a298586d-78d9-4f4f-a97f-5fe3aeaf40cf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5ac47e3b09f43d19a23a0f457fe9e23', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 10:55:57.674807 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.fernet_utils [None req-cb6e5b5c-eb3f-417c-97f3-8a1088331150 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80144) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 10:55:57.675225 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a298586d-78d9-4f4f-a97f-5fe3aeaf40cf admin admin] REQUEST_METHOD: `DELETE` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:55:57.675523 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a298586d-78d9-4f4f-a97f-5fe3aeaf40cf admin admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:55:57.675780 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a298586d-78d9-4f4f-a97f-5fe3aeaf40cf admin admin] PATH_INFO: `/v3/users/9f26c60fbcca483ebeb9f8ff41a3361d` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:55:57.677023 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a298586d-78d9-4f4f-a97f-5fe3aeaf40cf admin admin] RBAC: Authorizing `identity:delete_user(user_id=9f26c60fbcca483ebeb9f8ff41a3361d)` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 10:55:57.679627 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a298586d-78d9-4f4f-a97f-5fe3aeaf40cf admin admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 10:55:57.739348 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-a298586d-78d9-4f4f-a97f-5fe3aeaf40cf admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9f26c60fbcca483ebeb9f8ff41a3361d'} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 10:55:57.744253 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb6e5b5c-eb3f-417c-97f3-8a1088331150 None None] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 10:55:57.748425 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-a298586d-78d9-4f4f-a97f-5fe3aeaf40cf admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9f26c60fbcca483ebeb9f8ff41a3361d'} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 10:55:57.749053 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b77cdb0-67f0-4aac-9dc9-9089223b98b9 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88dbd6dfd1164994a3e16da553561910', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6de1f3cac02c4bcdb05dd2337176033d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 10:55:57.749672 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b77cdb0-67f0-4aac-9dc9-9089223b98b9 service cinder] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:55:57.749959 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b77cdb0-67f0-4aac-9dc9-9089223b98b9 service cinder] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:55:57.750229 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b77cdb0-67f0-4aac-9dc9-9089223b98b9 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:55:57.759501 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b77cdb0-67f0-4aac-9dc9-9089223b98b9 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 10:55:57.760784 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-a298586d-78d9-4f4f-a97f-5fe3aeaf40cf admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9f26c60fbcca483ebeb9f8ff41a3361d'} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 10:55:57.760918 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b77cdb0-67f0-4aac-9dc9-9089223b98b9 service cinder] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 10:55:57.761501 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-a298586d-78d9-4f4f-a97f-5fe3aeaf40cf admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9f26c60fbcca483ebeb9f8ff41a3361d'} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 10:55:57.781656 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 1077/2124] 10.222.0.25 () {60 vars in 1365 bytes} [Sun Aug 7 10:55:57 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 111 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 07 10:55:57.782201 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-a298586d-78d9-4f4f-a97f-5fe3aeaf40cf admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9f26c60fbcca483ebeb9f8ff41a3361d'} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 10:55:57.788933 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1048/2125] 10.222.0.25 () {58 vars in 1224 bytes} [Sun Aug 7 10:55:57 2022] DELETE /identity/v3/users/9f26c60fbcca483ebeb9f8ff41a3361d => generated 0 bytes in 200 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 07 10:55:57.793435 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce7e98aa-f351-4acc-8125-2f8ab5689808 None None] REQUEST_METHOD: `POST` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:55:57.793883 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce7e98aa-f351-4acc-8125-2f8ab5689808 None None] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:55:57.794175 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce7e98aa-f351-4acc-8125-2f8ab5689808 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:55:57.815929 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.auth.core [None req-ce7e98aa-f351-4acc-8125-2f8ab5689808 None None] MFA Rules not processed for user `bac4e6b76fe64217ac8fa44e71cffe6a`. Rule list: `[]` (Enabled: `True`). {{(pid=80144) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 10:55:57.817293 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.fernet_utils [None req-ce7e98aa-f351-4acc-8125-2f8ab5689808 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80144) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 10:55:57.898250 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 1078/2126] 10.222.0.25 () {58 vars in 958 bytes} [Sun Aug 7 10:55:57 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 07 10:55:57.908041 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a298586d-78d9-4f4f-a97f-5fe3aeaf40cf admin admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 10:55:57.909808 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-a298586d-78d9-4f4f-a97f-5fe3aeaf40cf admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 10:55:58.001516 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a298586d-78d9-4f4f-a97f-5fe3aeaf40cf admin admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 10:55:58.005863 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31e1cb16-5732-4daa-a76b-194a5015257c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be1283c9bf47424c9aa3ad27cab03027', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6de1f3cac02c4bcdb05dd2337176033d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 10:55:58.006996 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31e1cb16-5732-4daa-a76b-194a5015257c service neutron] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:55:58.006996 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31e1cb16-5732-4daa-a76b-194a5015257c service neutron] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:55:58.007224 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31e1cb16-5732-4daa-a76b-194a5015257c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:55:58.020059 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31e1cb16-5732-4daa-a76b-194a5015257c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 10:55:58.021636 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31e1cb16-5732-4daa-a76b-194a5015257c service neutron] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 10:55:58.026762 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=88,keystone:INSERT=2,keystone:DELETE=10 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 10:55:58.051339 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1049/2127] 10.222.0.25 () {60 vars in 1365 bytes} [Sun Aug 7 10:55:57 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 144 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 07 10:55:58.363614 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce7e98aa-f351-4acc-8125-2f8ab5689808 None None] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 10:55:58.366502 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.fernet_utils [None req-ce7e98aa-f351-4acc-8125-2f8ab5689808 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80144) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 10:55:58.387497 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce7e98aa-f351-4acc-8125-2f8ab5689808 None None] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 10:55:58.393262 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84cf218d-a561-4a61-9991-54f8785bcf67 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5ac47e3b09f43d19a23a0f457fe9e23', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 10:55:58.393841 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84cf218d-a561-4a61-9991-54f8785bcf67 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:55:58.394120 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84cf218d-a561-4a61-9991-54f8785bcf67 admin admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:55:58.394392 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84cf218d-a561-4a61-9991-54f8785bcf67 admin admin] PATH_INFO: `/v3/projects/4c6cac93794d4adfa32a7e90db7d8658` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:55:58.395585 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84cf218d-a561-4a61-9991-54f8785bcf67 admin admin] RBAC: Authorizing `identity:delete_project(project_id=4c6cac93794d4adfa32a7e90db7d8658)` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 10:55:58.397508 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84cf218d-a561-4a61-9991-54f8785bcf67 admin admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 10:55:58.424684 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=99,keystone:DELETE=1 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 10:55:58.448461 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.notifications [None req-84cf218d-a561-4a61-9991-54f8785bcf67 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4c6cac93794d4adfa32a7e90db7d8658', 'request_id': 'req-84cf218d-a561-4a61-9991-54f8785bcf67'} {{(pid=80144) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 10:55:58.454500 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.notifications [None req-84cf218d-a561-4a61-9991-54f8785bcf67 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4c6cac93794d4adfa32a7e90db7d8658', 'request_id': 'req-84cf218d-a561-4a61-9991-54f8785bcf67'} {{(pid=80144) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 10:55:58.461538 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.notifications [None req-84cf218d-a561-4a61-9991-54f8785bcf67 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4c6cac93794d4adfa32a7e90db7d8658', 'request_id': 'req-84cf218d-a561-4a61-9991-54f8785bcf67'} {{(pid=80144) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 10:55:58.473000 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.notifications [None req-84cf218d-a561-4a61-9991-54f8785bcf67 admin admin] The token cache is being invalidate because project 4c6cac93794d4adfa32a7e90db7d8658 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80144) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 07 10:55:58.473292 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.notifications [None req-84cf218d-a561-4a61-9991-54f8785bcf67 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80144) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 10:55:58.473926 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.notifications [None req-84cf218d-a561-4a61-9991-54f8785bcf67 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80144) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 10:55:58.475446 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 1079/2128] 10.222.0.25 () {58 vars in 1233 bytes} [Sun Aug 7 10:55:58 2022] DELETE /identity/v3/projects/4c6cac93794d4adfa32a7e90db7d8658 => generated 0 bytes in 112 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 07 10:55:58.479445 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31e1cb16-5732-4daa-a76b-194a5015257c service neutron] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 10:55:58.490120 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31e1cb16-5732-4daa-a76b-194a5015257c service neutron] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 10:55:58.546800 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f72ac07e-d88c-4aed-848d-ba5ee1a59671 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5ac47e3b09f43d19a23a0f457fe9e23', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 10:55:58.547453 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f72ac07e-d88c-4aed-848d-ba5ee1a59671 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:55:58.547729 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f72ac07e-d88c-4aed-848d-ba5ee1a59671 admin admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:55:58.547997 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f72ac07e-d88c-4aed-848d-ba5ee1a59671 admin admin] PATH_INFO: `/v3/users/2e539e8834a149e2a214a5bd595d8b97` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:55:58.549120 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f72ac07e-d88c-4aed-848d-ba5ee1a59671 admin admin] RBAC: Authorizing `identity:delete_user(user_id=2e539e8834a149e2a214a5bd595d8b97)` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 10:55:58.551046 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f72ac07e-d88c-4aed-848d-ba5ee1a59671 admin admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 10:55:58.596939 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-f72ac07e-d88c-4aed-848d-ba5ee1a59671 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2e539e8834a149e2a214a5bd595d8b97'} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 10:55:58.603964 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-f72ac07e-d88c-4aed-848d-ba5ee1a59671 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2e539e8834a149e2a214a5bd595d8b97'} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 10:55:58.617409 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-f72ac07e-d88c-4aed-848d-ba5ee1a59671 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2e539e8834a149e2a214a5bd595d8b97'} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 10:55:58.618226 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-f72ac07e-d88c-4aed-848d-ba5ee1a59671 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2e539e8834a149e2a214a5bd595d8b97'} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 10:55:58.625895 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-f72ac07e-d88c-4aed-848d-ba5ee1a59671 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2e539e8834a149e2a214a5bd595d8b97'} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 10:55:58.632427 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1050/2129] 10.222.0.25 () {58 vars in 1224 bytes} [Sun Aug 7 10:55:58 2022] DELETE /identity/v3/users/2e539e8834a149e2a214a5bd595d8b97 => generated 0 bytes in 153 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 07 10:55:58.911070 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84cf218d-a561-4a61-9991-54f8785bcf67 admin admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 10:55:58.912528 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.fernet_utils [None req-84cf218d-a561-4a61-9991-54f8785bcf67 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80144) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 10:55:58.926889 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84cf218d-a561-4a61-9991-54f8785bcf67 admin admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 10:55:58.986117 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d198524-26fa-4357-9572-e036f36a09c3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5ac47e3b09f43d19a23a0f457fe9e23', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 10:55:58.986679 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d198524-26fa-4357-9572-e036f36a09c3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:55:58.986995 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d198524-26fa-4357-9572-e036f36a09c3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:55:58.987277 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d198524-26fa-4357-9572-e036f36a09c3 admin admin] PATH_INFO: `/v3/projects/987cc28bfd3545d3bc320a116b8271bc` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:55:58.988393 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d198524-26fa-4357-9572-e036f36a09c3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=987cc28bfd3545d3bc320a116b8271bc)` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 10:55:58.990298 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d198524-26fa-4357-9572-e036f36a09c3 admin admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 10:55:59.036021 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.notifications [None req-6d198524-26fa-4357-9572-e036f36a09c3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '987cc28bfd3545d3bc320a116b8271bc', 'request_id': 'req-6d198524-26fa-4357-9572-e036f36a09c3'} {{(pid=80144) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 10:55:59.041362 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.notifications [None req-6d198524-26fa-4357-9572-e036f36a09c3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '987cc28bfd3545d3bc320a116b8271bc', 'request_id': 'req-6d198524-26fa-4357-9572-e036f36a09c3'} {{(pid=80144) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 10:55:59.048758 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.notifications [None req-6d198524-26fa-4357-9572-e036f36a09c3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '987cc28bfd3545d3bc320a116b8271bc', 'request_id': 'req-6d198524-26fa-4357-9572-e036f36a09c3'} {{(pid=80144) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 10:55:59.059583 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.notifications [None req-6d198524-26fa-4357-9572-e036f36a09c3 admin admin] The token cache is being invalidate because project 987cc28bfd3545d3bc320a116b8271bc was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80144) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 07 10:55:59.059909 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.notifications [None req-6d198524-26fa-4357-9572-e036f36a09c3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80144) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 10:55:59.060573 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.notifications [None req-6d198524-26fa-4357-9572-e036f36a09c3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80144) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 10:55:59.062128 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 1080/2130] 10.222.0.25 () {58 vars in 1233 bytes} [Sun Aug 7 10:55:58 2022] DELETE /identity/v3/projects/987cc28bfd3545d3bc320a116b8271bc => generated 0 bytes in 152 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 07 10:55:59.068714 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebe8ae6a-4bfc-40fc-8b0d-f1af1940d817 None None] REQUEST_METHOD: `POST` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:55:59.069214 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebe8ae6a-4bfc-40fc-8b0d-f1af1940d817 None None] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:55:59.069316 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebe8ae6a-4bfc-40fc-8b0d-f1af1940d817 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:55:59.089949 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.auth.core [None req-ebe8ae6a-4bfc-40fc-8b0d-f1af1940d817 None None] MFA Rules not processed for user `bac4e6b76fe64217ac8fa44e71cffe6a`. Rule list: `[]` (Enabled: `True`). {{(pid=80143) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 10:55:59.090660 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-ebe8ae6a-4bfc-40fc-8b0d-f1af1940d817 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 10:55:59.160329 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1051/2131] 10.222.0.25 () {58 vars in 958 bytes} [Sun Aug 7 10:55:59 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 07 10:55:59.169534 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50a11dc7-c5aa-4e6a-8fba-b689b2db1fe2 None None] REQUEST_METHOD: `POST` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:55:59.169534 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50a11dc7-c5aa-4e6a-8fba-b689b2db1fe2 None None] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:55:59.169958 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50a11dc7-c5aa-4e6a-8fba-b689b2db1fe2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:55:59.193457 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.auth.core [None req-50a11dc7-c5aa-4e6a-8fba-b689b2db1fe2 None None] MFA Rules not processed for user `bac4e6b76fe64217ac8fa44e71cffe6a`. Rule list: `[]` (Enabled: `True`). {{(pid=80144) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 10:55:59.194811 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.fernet_utils [None req-50a11dc7-c5aa-4e6a-8fba-b689b2db1fe2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80144) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 10:55:59.207399 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 1081/2132] 10.222.0.25 () {58 vars in 958 bytes} [Sun Aug 7 10:55:59 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 07 10:55:59.212047 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebe8ae6a-4bfc-40fc-8b0d-f1af1940d817 None None] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 10:55:59.227495 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebe8ae6a-4bfc-40fc-8b0d-f1af1940d817 None None] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 10:55:59.233472 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4e8bd7e-9014-47d6-b3eb-98f2aa3fba80 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5ac47e3b09f43d19a23a0f457fe9e23', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 10:55:59.234148 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4e8bd7e-9014-47d6-b3eb-98f2aa3fba80 admin admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:55:59.234426 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4e8bd7e-9014-47d6-b3eb-98f2aa3fba80 admin admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:55:59.234698 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4e8bd7e-9014-47d6-b3eb-98f2aa3fba80 admin admin] PATH_INFO: `/v3/domains` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:55:59.235559 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4e8bd7e-9014-47d6-b3eb-98f2aa3fba80 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80143) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 10:55:59.235925 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4e8bd7e-9014-47d6-b3eb-98f2aa3fba80 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 10:55:59.237531 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4e8bd7e-9014-47d6-b3eb-98f2aa3fba80 admin admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 10:55:59.250354 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1052/2133] 10.222.0.25 () {58 vars in 1153 bytes} [Sun Aug 7 10:55:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 07 10:55:59.254693 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50a11dc7-c5aa-4e6a-8fba-b689b2db1fe2 None None] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 10:55:59.264691 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50a11dc7-c5aa-4e6a-8fba-b689b2db1fe2 None None] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 10:55:59.269429 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e45df66d-f94e-49fe-9c3b-58a97535824a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5ac47e3b09f43d19a23a0f457fe9e23', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 10:55:59.270034 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e45df66d-f94e-49fe-9c3b-58a97535824a admin admin] REQUEST_METHOD: `POST` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:55:59.270310 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e45df66d-f94e-49fe-9c3b-58a97535824a admin admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:55:59.270606 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e45df66d-f94e-49fe-9c3b-58a97535824a admin admin] PATH_INFO: `/v3/projects` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:55:59.271279 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e45df66d-f94e-49fe-9c3b-58a97535824a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 10:55:59.273152 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e45df66d-f94e-49fe-9c3b-58a97535824a admin admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 10:55:59.302865 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 1082/2134] 10.222.0.25 () {60 vars in 1153 bytes} [Sun Aug 7 10:55:59 2022] POST /identity/v3/projects => generated 375 bytes in 49 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 07 10:55:59.307301 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4e8bd7e-9014-47d6-b3eb-98f2aa3fba80 admin admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 10:55:59.317293 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 10:55:59.321606 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4e8bd7e-9014-47d6-b3eb-98f2aa3fba80 admin admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 10:55:59.392485 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8d3ca1f-1ed2-46c7-9bcb-1644d0bb4b6c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5ac47e3b09f43d19a23a0f457fe9e23', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 10:55:59.393144 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8d3ca1f-1ed2-46c7-9bcb-1644d0bb4b6c admin admin] REQUEST_METHOD: `POST` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:55:59.393425 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8d3ca1f-1ed2-46c7-9bcb-1644d0bb4b6c admin admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:55:59.393719 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8d3ca1f-1ed2-46c7-9bcb-1644d0bb4b6c admin admin] PATH_INFO: `/v3/users` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:55:59.394350 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8d3ca1f-1ed2-46c7-9bcb-1644d0bb4b6c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 10:55:59.396485 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8d3ca1f-1ed2-46c7-9bcb-1644d0bb4b6c admin admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 10:55:59.421407 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1053/2135] 10.222.0.25 () {60 vars in 1144 bytes} [Sun Aug 7 10:55:59 2022] POST /identity/v3/users => generated 288 bytes in 111 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 07 10:55:59.423065 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e45df66d-f94e-49fe-9c3b-58a97535824a admin admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 10:55:59.433986 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e45df66d-f94e-49fe-9c3b-58a97535824a admin admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 10:55:59.439617 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9367d727-9cf0-4a94-bd2c-db5610f5e966 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5ac47e3b09f43d19a23a0f457fe9e23', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 10:55:59.440223 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9367d727-9cf0-4a94-bd2c-db5610f5e966 admin admin] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:55:59.440512 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9367d727-9cf0-4a94-bd2c-db5610f5e966 admin admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:55:59.440788 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9367d727-9cf0-4a94-bd2c-db5610f5e966 admin admin] PATH_INFO: `/v3/roles` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:55:59.441436 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9367d727-9cf0-4a94-bd2c-db5610f5e966 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 10:55:59.443052 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9367d727-9cf0-4a94-bd2c-db5610f5e966 admin admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 10:55:59.454002 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 1083/2136] 10.222.0.25 () {58 vars in 1122 bytes} [Sun Aug 7 10:55:59 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 07 10:55:59.458485 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8d3ca1f-1ed2-46c7-9bcb-1644d0bb4b6c admin admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 10:55:59.470261 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8d3ca1f-1ed2-46c7-9bcb-1644d0bb4b6c admin admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 10:55:59.475064 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a89974f0-6d86-48d5-b584-27105026d12d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5ac47e3b09f43d19a23a0f457fe9e23', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 10:55:59.475664 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a89974f0-6d86-48d5-b584-27105026d12d admin admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:55:59.475939 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a89974f0-6d86-48d5-b584-27105026d12d admin admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:55:59.476209 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a89974f0-6d86-48d5-b584-27105026d12d admin admin] PATH_INFO: `/v3/roles` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:55:59.476851 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a89974f0-6d86-48d5-b584-27105026d12d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 10:55:59.478436 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a89974f0-6d86-48d5-b584-27105026d12d admin admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 10:55:59.489800 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1054/2137] 10.222.0.25 () {58 vars in 1122 bytes} [Sun Aug 7 10:55:59 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 07 10:55:59.493916 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9367d727-9cf0-4a94-bd2c-db5610f5e966 admin admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 10:55:59.500183 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=84,keystone:DELETE=13,keystone:INSERT=3 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 10:55:59.506415 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9367d727-9cf0-4a94-bd2c-db5610f5e966 admin admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 10:55:59.511776 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8e4547b-bac6-4e15-96cc-8d5629bae9dd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5ac47e3b09f43d19a23a0f457fe9e23', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 10:55:59.512458 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8e4547b-bac6-4e15-96cc-8d5629bae9dd admin admin] REQUEST_METHOD: `PUT` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:55:59.512740 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8e4547b-bac6-4e15-96cc-8d5629bae9dd admin admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:55:59.513017 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8e4547b-bac6-4e15-96cc-8d5629bae9dd admin admin] PATH_INFO: `/v3/projects/2f306584e65048f98a4128cafc1b210c/users/aeb2182f9b614007b61823075835fe33/roles/b98dfb3b9fee4644af2887a3ac3fd2e6` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:55:59.528592 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8e4547b-bac6-4e15-96cc-8d5629bae9dd admin admin] RBAC: Authorizing `identity:create_grant(project_id=2f306584e65048f98a4128cafc1b210c, user_id=aeb2182f9b614007b61823075835fe33, role_id=b98dfb3b9fee4644af2887a3ac3fd2e6)` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 10:55:59.531770 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8e4547b-bac6-4e15-96cc-8d5629bae9dd admin admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 10:55:59.544250 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 1084/2138] 10.222.0.25 () {60 vars in 1483 bytes} [Sun Aug 7 10:55:59 2022] PUT /identity/v3/projects/2f306584e65048f98a4128cafc1b210c/users/aeb2182f9b614007b61823075835fe33/roles/b98dfb3b9fee4644af2887a3ac3fd2e6 => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 07 10:55:59.556508 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ad90598-f812-4bc3-8dc2-47082f2d0e55 None None] REQUEST_METHOD: `POST` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:55:59.556825 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ad90598-f812-4bc3-8dc2-47082f2d0e55 None None] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:55:59.557125 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ad90598-f812-4bc3-8dc2-47082f2d0e55 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:55:59.588334 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.auth.core [None req-8ad90598-f812-4bc3-8dc2-47082f2d0e55 None None] MFA Rules not processed for user `aeb2182f9b614007b61823075835fe33`. Rule list: `[]` (Enabled: `True`). {{(pid=80143) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 10:55:59.589852 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-8ad90598-f812-4bc3-8dc2-47082f2d0e55 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 10:55:59.655675 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1055/2139] 10.222.0.25 () {58 vars in 958 bytes} [Sun Aug 7 10:55:59 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 07 10:55:59.666503 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8e4547b-bac6-4e15-96cc-8d5629bae9dd admin admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 10:55:59.668383 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.fernet_utils [None req-e8e4547b-bac6-4e15-96cc-8d5629bae9dd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80144) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 10:55:59.729902 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8e4547b-bac6-4e15-96cc-8d5629bae9dd admin admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 10:55:59.735012 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e05834d6-1940-49b9-874e-2fb1ce81bdce service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88dbd6dfd1164994a3e16da553561910', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6de1f3cac02c4bcdb05dd2337176033d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 10:55:59.735619 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e05834d6-1940-49b9-874e-2fb1ce81bdce service cinder] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:55:59.735900 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e05834d6-1940-49b9-874e-2fb1ce81bdce service cinder] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:55:59.736173 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e05834d6-1940-49b9-874e-2fb1ce81bdce service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:55:59.744359 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e05834d6-1940-49b9-874e-2fb1ce81bdce service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 10:55:59.745767 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e05834d6-1940-49b9-874e-2fb1ce81bdce service cinder] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 10:55:59.755230 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 1085/2140] 10.222.0.25 () {60 vars in 1365 bytes} [Sun Aug 7 10:55:59 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 07 10:56:02.191058 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a33989e-0683-48a5-b0a7-1329d7181760 None None] REQUEST_METHOD: `POST` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:56:02.191557 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a33989e-0683-48a5-b0a7-1329d7181760 None None] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:56:02.191678 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a33989e-0683-48a5-b0a7-1329d7181760 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:56:02.210499 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.auth.core [None req-8a33989e-0683-48a5-b0a7-1329d7181760 None None] MFA Rules not processed for user `aeb2182f9b614007b61823075835fe33`. Rule list: `[]` (Enabled: `True`). {{(pid=80143) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 10:56:02.211661 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-8a33989e-0683-48a5-b0a7-1329d7181760 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 10:56:02.218764 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1056/2141] 10.222.0.25 () {58 vars in 958 bytes} [Sun Aug 7 10:56:02 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 07 10:56:02.553164 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e05834d6-1940-49b9-874e-2fb1ce81bdce service cinder] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 10:56:02.554711 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.fernet_utils [None req-e05834d6-1940-49b9-874e-2fb1ce81bdce service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80144) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 10:56:02.624285 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e05834d6-1940-49b9-874e-2fb1ce81bdce service cinder] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 10:56:02.629223 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e61a5edb-dea1-40d3-931e-512a9fb385b5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5ac47e3b09f43d19a23a0f457fe9e23', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 10:56:02.629779 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e61a5edb-dea1-40d3-931e-512a9fb385b5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:56:02.630088 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e61a5edb-dea1-40d3-931e-512a9fb385b5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:56:02.630355 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e61a5edb-dea1-40d3-931e-512a9fb385b5 admin admin] PATH_INFO: `/v3/users/ae9ef7de6a554145a7f2a651dc3d9a42` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:56:02.631961 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e61a5edb-dea1-40d3-931e-512a9fb385b5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ae9ef7de6a554145a7f2a651dc3d9a42)` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 10:56:02.633879 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e61a5edb-dea1-40d3-931e-512a9fb385b5 admin admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 10:56:02.694374 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.notifications [None req-e61a5edb-dea1-40d3-931e-512a9fb385b5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ae9ef7de6a554145a7f2a651dc3d9a42'} {{(pid=80144) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 10:56:02.702148 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.notifications [None req-e61a5edb-dea1-40d3-931e-512a9fb385b5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ae9ef7de6a554145a7f2a651dc3d9a42'} {{(pid=80144) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 10:56:02.710032 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.notifications [None req-e61a5edb-dea1-40d3-931e-512a9fb385b5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ae9ef7de6a554145a7f2a651dc3d9a42'} {{(pid=80144) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 10:56:02.710694 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.notifications [None req-e61a5edb-dea1-40d3-931e-512a9fb385b5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ae9ef7de6a554145a7f2a651dc3d9a42'} {{(pid=80144) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 10:56:02.711337 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.notifications [None req-e61a5edb-dea1-40d3-931e-512a9fb385b5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ae9ef7de6a554145a7f2a651dc3d9a42'} {{(pid=80144) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 10:56:02.728598 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=88,keystone:INSERT=2,keystone:DELETE=10 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 10:56:02.734483 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 1086/2142] 10.222.0.25 () {58 vars in 1224 bytes} [Sun Aug 7 10:56:02 2022] DELETE /identity/v3/users/ae9ef7de6a554145a7f2a651dc3d9a42 => generated 0 bytes in 182 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 07 10:56:02.739086 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10bd1aef-a7da-4dd4-a4b3-c3a84e32ed5e None None] REQUEST_METHOD: `POST` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:56:02.739493 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10bd1aef-a7da-4dd4-a4b3-c3a84e32ed5e None None] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:56:02.739908 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10bd1aef-a7da-4dd4-a4b3-c3a84e32ed5e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:56:02.759689 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.auth.core [None req-10bd1aef-a7da-4dd4-a4b3-c3a84e32ed5e None None] MFA Rules not processed for user `bac4e6b76fe64217ac8fa44e71cffe6a`. Rule list: `[]` (Enabled: `True`). {{(pid=80143) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 10:56:02.761210 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-10bd1aef-a7da-4dd4-a4b3-c3a84e32ed5e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 10:56:02.834860 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1057/2143] 10.222.0.25 () {58 vars in 958 bytes} [Sun Aug 7 10:56:02 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 07 10:56:02.846406 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e61a5edb-dea1-40d3-931e-512a9fb385b5 admin admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 10:56:02.847971 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.fernet_utils [None req-e61a5edb-dea1-40d3-931e-512a9fb385b5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80144) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 10:56:02.929360 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e61a5edb-dea1-40d3-931e-512a9fb385b5 admin admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 10:56:02.934245 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-045fe7c3-56c2-47a3-93aa-2d3eb47d01cc service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be1283c9bf47424c9aa3ad27cab03027', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6de1f3cac02c4bcdb05dd2337176033d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 10:56:02.934857 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-045fe7c3-56c2-47a3-93aa-2d3eb47d01cc service neutron] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:56:02.935508 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-045fe7c3-56c2-47a3-93aa-2d3eb47d01cc service neutron] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:56:02.935887 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-045fe7c3-56c2-47a3-93aa-2d3eb47d01cc service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:56:02.948109 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-045fe7c3-56c2-47a3-93aa-2d3eb47d01cc service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 10:56:02.949750 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-045fe7c3-56c2-47a3-93aa-2d3eb47d01cc service neutron] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 10:56:02.968729 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 1087/2144] 10.222.0.25 () {60 vars in 1365 bytes} [Sun Aug 7 10:56:02 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 123 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 07 10:56:03.291085 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10bd1aef-a7da-4dd4-a4b3-c3a84e32ed5e None None] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 10:56:03.292600 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-10bd1aef-a7da-4dd4-a4b3-c3a84e32ed5e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 10:56:03.306625 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10bd1aef-a7da-4dd4-a4b3-c3a84e32ed5e None None] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 10:56:03.312478 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c994527e-0f7d-4e7e-887f-b0735bd80c22 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5ac47e3b09f43d19a23a0f457fe9e23', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 10:56:03.313127 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c994527e-0f7d-4e7e-887f-b0735bd80c22 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:56:03.313410 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c994527e-0f7d-4e7e-887f-b0735bd80c22 admin admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:56:03.313687 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c994527e-0f7d-4e7e-887f-b0735bd80c22 admin admin] PATH_INFO: `/v3/projects/3e4147dc53b246f78e50703eb33acce6` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:56:03.314827 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c994527e-0f7d-4e7e-887f-b0735bd80c22 admin admin] RBAC: Authorizing `identity:delete_project(project_id=3e4147dc53b246f78e50703eb33acce6)` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 10:56:03.316762 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c994527e-0f7d-4e7e-887f-b0735bd80c22 admin admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 10:56:03.318503 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 10:56:03.391009 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-c994527e-0f7d-4e7e-887f-b0735bd80c22 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3e4147dc53b246f78e50703eb33acce6', 'request_id': 'req-c994527e-0f7d-4e7e-887f-b0735bd80c22'} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 10:56:03.401035 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-c994527e-0f7d-4e7e-887f-b0735bd80c22 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3e4147dc53b246f78e50703eb33acce6', 'request_id': 'req-c994527e-0f7d-4e7e-887f-b0735bd80c22'} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 10:56:03.409304 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-c994527e-0f7d-4e7e-887f-b0735bd80c22 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3e4147dc53b246f78e50703eb33acce6', 'request_id': 'req-c994527e-0f7d-4e7e-887f-b0735bd80c22'} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 10:56:03.427607 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-c994527e-0f7d-4e7e-887f-b0735bd80c22 admin admin] The token cache is being invalidate because project 3e4147dc53b246f78e50703eb33acce6 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80143) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 07 10:56:03.428252 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-c994527e-0f7d-4e7e-887f-b0735bd80c22 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 10:56:03.429215 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-c994527e-0f7d-4e7e-887f-b0735bd80c22 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 10:56:03.432209 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1058/2145] 10.222.0.25 () {58 vars in 1233 bytes} [Sun Aug 7 10:56:03 2022] DELETE /identity/v3/projects/3e4147dc53b246f78e50703eb33acce6 => generated 0 bytes in 142 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 07 10:56:03.438116 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16a7bef4-5560-470a-93fe-a6b50772fea9 None None] REQUEST_METHOD: `POST` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:56:03.438984 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16a7bef4-5560-470a-93fe-a6b50772fea9 None None] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:56:03.439189 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16a7bef4-5560-470a-93fe-a6b50772fea9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:56:03.459652 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.auth.core [None req-16a7bef4-5560-470a-93fe-a6b50772fea9 None None] MFA Rules not processed for user `bac4e6b76fe64217ac8fa44e71cffe6a`. Rule list: `[]` (Enabled: `True`). {{(pid=80144) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 10:56:03.460857 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.fernet_utils [None req-16a7bef4-5560-470a-93fe-a6b50772fea9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80144) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 10:56:03.533267 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 1088/2146] 10.222.0.25 () {58 vars in 958 bytes} [Sun Aug 7 10:56:03 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 07 10:56:03.541485 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cd67ec8-0682-495b-be6e-67f2f11a6383 None None] REQUEST_METHOD: `POST` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:56:03.541694 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cd67ec8-0682-495b-be6e-67f2f11a6383 None None] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:56:03.541979 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cd67ec8-0682-495b-be6e-67f2f11a6383 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:56:03.563229 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.auth.core [None req-6cd67ec8-0682-495b-be6e-67f2f11a6383 None None] MFA Rules not processed for user `bac4e6b76fe64217ac8fa44e71cffe6a`. Rule list: `[]` (Enabled: `True`). {{(pid=80143) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 10:56:03.564465 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-6cd67ec8-0682-495b-be6e-67f2f11a6383 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 10:56:03.572849 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1059/2147] 10.222.0.25 () {58 vars in 958 bytes} [Sun Aug 7 10:56:03 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 07 10:56:03.577946 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16a7bef4-5560-470a-93fe-a6b50772fea9 None None] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 10:56:03.588403 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16a7bef4-5560-470a-93fe-a6b50772fea9 None None] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 10:56:03.594455 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c8b1802-15d8-48e9-be1a-971184b0b307 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5ac47e3b09f43d19a23a0f457fe9e23', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 10:56:03.595179 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c8b1802-15d8-48e9-be1a-971184b0b307 admin admin] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:56:03.595463 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c8b1802-15d8-48e9-be1a-971184b0b307 admin admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:56:03.595742 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c8b1802-15d8-48e9-be1a-971184b0b307 admin admin] PATH_INFO: `/v3/domains` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:56:03.596410 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c8b1802-15d8-48e9-be1a-971184b0b307 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80144) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 10:56:03.596720 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c8b1802-15d8-48e9-be1a-971184b0b307 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 10:56:03.598437 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c8b1802-15d8-48e9-be1a-971184b0b307 admin admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 10:56:03.615862 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cd67ec8-0682-495b-be6e-67f2f11a6383 None None] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 10:56:03.619019 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 1089/2148] 10.222.0.25 () {58 vars in 1153 bytes} [Sun Aug 7 10:56:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 10:56:03.627220 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cd67ec8-0682-495b-be6e-67f2f11a6383 None None] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 10:56:03.632261 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cc44be3-b65e-4611-9d3f-5660101c2900 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5ac47e3b09f43d19a23a0f457fe9e23', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 10:56:03.632808 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cc44be3-b65e-4611-9d3f-5660101c2900 admin admin] REQUEST_METHOD: `POST` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:56:03.633106 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cc44be3-b65e-4611-9d3f-5660101c2900 admin admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:56:03.633419 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cc44be3-b65e-4611-9d3f-5660101c2900 admin admin] PATH_INFO: `/v3/projects` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:56:03.634114 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cc44be3-b65e-4611-9d3f-5660101c2900 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 10:56:03.636472 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cc44be3-b65e-4611-9d3f-5660101c2900 admin admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 10:56:03.667353 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1060/2149] 10.222.0.25 () {60 vars in 1153 bytes} [Sun Aug 7 10:56:03 2022] POST /identity/v3/projects => generated 395 bytes in 52 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 07 10:56:03.671570 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c8b1802-15d8-48e9-be1a-971184b0b307 admin admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 10:56:03.682095 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c8b1802-15d8-48e9-be1a-971184b0b307 admin admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 10:56:03.742317 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e072532a-e800-4d1c-9c81-6c1927a73081 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5ac47e3b09f43d19a23a0f457fe9e23', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 10:56:03.742979 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e072532a-e800-4d1c-9c81-6c1927a73081 admin admin] REQUEST_METHOD: `POST` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:56:03.743251 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e072532a-e800-4d1c-9c81-6c1927a73081 admin admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:56:03.743530 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e072532a-e800-4d1c-9c81-6c1927a73081 admin admin] PATH_INFO: `/v3/users` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:56:03.744175 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e072532a-e800-4d1c-9c81-6c1927a73081 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 10:56:03.746106 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e072532a-e800-4d1c-9c81-6c1927a73081 admin admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 10:56:03.780874 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cc44be3-b65e-4611-9d3f-5660101c2900 admin admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 10:56:03.797894 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 1090/2150] 10.222.0.25 () {60 vars in 1144 bytes} [Sun Aug 7 10:56:03 2022] POST /identity/v3/users => generated 298 bytes in 126 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 07 10:56:03.805359 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cc44be3-b65e-4611-9d3f-5660101c2900 admin admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 10:56:03.822363 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81719acb-26e7-483a-b62d-2ab6579f57aa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5ac47e3b09f43d19a23a0f457fe9e23', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 10:56:03.823117 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81719acb-26e7-483a-b62d-2ab6579f57aa admin admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:56:03.823452 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81719acb-26e7-483a-b62d-2ab6579f57aa admin admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:56:03.823770 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81719acb-26e7-483a-b62d-2ab6579f57aa admin admin] PATH_INFO: `/v3/roles` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:56:03.824523 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81719acb-26e7-483a-b62d-2ab6579f57aa admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 10:56:03.828201 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81719acb-26e7-483a-b62d-2ab6579f57aa admin admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 10:56:03.843522 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1061/2151] 10.222.0.25 () {58 vars in 1122 bytes} [Sun Aug 7 10:56:03 2022] GET /identity/v3/roles => generated 1420 bytes in 63 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 07 10:56:03.848358 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e072532a-e800-4d1c-9c81-6c1927a73081 admin admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 10:56:03.866351 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e072532a-e800-4d1c-9c81-6c1927a73081 admin admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 10:56:03.874310 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46561a54-d8cf-4b12-b870-c4e3010ddd87 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5ac47e3b09f43d19a23a0f457fe9e23', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 10:56:03.875541 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46561a54-d8cf-4b12-b870-c4e3010ddd87 admin admin] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:56:03.875966 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46561a54-d8cf-4b12-b870-c4e3010ddd87 admin admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:56:03.876340 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46561a54-d8cf-4b12-b870-c4e3010ddd87 admin admin] PATH_INFO: `/v3/roles` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:56:03.877139 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46561a54-d8cf-4b12-b870-c4e3010ddd87 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 10:56:03.879363 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46561a54-d8cf-4b12-b870-c4e3010ddd87 admin admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 10:56:03.896796 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 1091/2152] 10.222.0.25 () {58 vars in 1122 bytes} [Sun Aug 7 10:56:03 2022] GET /identity/v3/roles => generated 1420 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 07 10:56:03.898215 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81719acb-26e7-483a-b62d-2ab6579f57aa admin admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 10:56:03.910816 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81719acb-26e7-483a-b62d-2ab6579f57aa admin admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 10:56:03.916328 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f44013b7-57e5-4566-a2dc-9375270c3c19 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5ac47e3b09f43d19a23a0f457fe9e23', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 10:56:03.917111 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f44013b7-57e5-4566-a2dc-9375270c3c19 admin admin] REQUEST_METHOD: `PUT` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:56:03.917464 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f44013b7-57e5-4566-a2dc-9375270c3c19 admin admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:56:03.917805 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f44013b7-57e5-4566-a2dc-9375270c3c19 admin admin] PATH_INFO: `/v3/projects/658b7a570d7d40fc837a988ca549a811/users/792c13a5287541868276bb7ee538afcc/roles/b98dfb3b9fee4644af2887a3ac3fd2e6` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:56:03.938535 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f44013b7-57e5-4566-a2dc-9375270c3c19 admin admin] RBAC: Authorizing `identity:create_grant(project_id=658b7a570d7d40fc837a988ca549a811, user_id=792c13a5287541868276bb7ee538afcc, role_id=b98dfb3b9fee4644af2887a3ac3fd2e6)` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 10:56:03.941333 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f44013b7-57e5-4566-a2dc-9375270c3c19 admin admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 10:56:03.956292 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1062/2153] 10.222.0.25 () {60 vars in 1483 bytes} [Sun Aug 7 10:56:03 2022] PUT /identity/v3/projects/658b7a570d7d40fc837a988ca549a811/users/792c13a5287541868276bb7ee538afcc/roles/b98dfb3b9fee4644af2887a3ac3fd2e6 => generated 0 bytes in 58 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 07 10:56:03.966299 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6835cb8d-9c49-4d39-93b8-100298ea9b81 None None] REQUEST_METHOD: `POST` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:56:03.966612 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6835cb8d-9c49-4d39-93b8-100298ea9b81 None None] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:56:03.966879 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6835cb8d-9c49-4d39-93b8-100298ea9b81 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:56:03.986611 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 10:56:03.996304 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.auth.core [None req-6835cb8d-9c49-4d39-93b8-100298ea9b81 None None] MFA Rules not processed for user `792c13a5287541868276bb7ee538afcc`. Rule list: `[]` (Enabled: `True`). {{(pid=80144) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 10:56:03.998012 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.fernet_utils [None req-6835cb8d-9c49-4d39-93b8-100298ea9b81 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80144) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 10:56:04.061201 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 1092/2154] 10.222.0.25 () {58 vars in 958 bytes} [Sun Aug 7 10:56:03 2022] POST /identity/v3/auth/tokens => generated 3272 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 07 10:56:04.073657 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f44013b7-57e5-4566-a2dc-9375270c3c19 admin admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 10:56:04.075521 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-f44013b7-57e5-4566-a2dc-9375270c3c19 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 10:56:04.140512 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f44013b7-57e5-4566-a2dc-9375270c3c19 admin admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 10:56:04.144908 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df64a034-abbf-42cb-9085-823493886d96 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88dbd6dfd1164994a3e16da553561910', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6de1f3cac02c4bcdb05dd2337176033d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 10:56:04.145604 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df64a034-abbf-42cb-9085-823493886d96 service cinder] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:56:04.145807 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df64a034-abbf-42cb-9085-823493886d96 service cinder] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:56:04.146154 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df64a034-abbf-42cb-9085-823493886d96 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:56:04.161546 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df64a034-abbf-42cb-9085-823493886d96 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 10:56:04.163438 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df64a034-abbf-42cb-9085-823493886d96 service cinder] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 10:56:04.176252 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1063/2155] 10.222.0.25 () {60 vars in 1365 bytes} [Sun Aug 7 10:56:04 2022] GET /identity/v3/auth/tokens => generated 3272 bytes in 103 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 07 10:56:14.057725 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=28 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 10:56:14.169824 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=77,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 10:56:15.477991 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6835cb8d-9c49-4d39-93b8-100298ea9b81 None None] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 10:56:15.578460 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6835cb8d-9c49-4d39-93b8-100298ea9b81 None None] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 10:56:15.583516 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc873945-4c0b-4ce4-a09a-bb240207adce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5ac47e3b09f43d19a23a0f457fe9e23', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 10:56:15.584259 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc873945-4c0b-4ce4-a09a-bb240207adce admin admin] REQUEST_METHOD: `DELETE` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:56:15.584609 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc873945-4c0b-4ce4-a09a-bb240207adce admin admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:56:15.585054 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc873945-4c0b-4ce4-a09a-bb240207adce admin admin] PATH_INFO: `/v3/users/792c13a5287541868276bb7ee538afcc` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:56:15.586263 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc873945-4c0b-4ce4-a09a-bb240207adce admin admin] RBAC: Authorizing `identity:delete_user(user_id=792c13a5287541868276bb7ee538afcc)` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 10:56:15.588258 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc873945-4c0b-4ce4-a09a-bb240207adce admin admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 10:56:15.642522 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.notifications [None req-bc873945-4c0b-4ce4-a09a-bb240207adce admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '792c13a5287541868276bb7ee538afcc'} {{(pid=80144) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 10:56:15.649965 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.notifications [None req-bc873945-4c0b-4ce4-a09a-bb240207adce admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '792c13a5287541868276bb7ee538afcc'} {{(pid=80144) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 10:56:15.659562 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.notifications [None req-bc873945-4c0b-4ce4-a09a-bb240207adce admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '792c13a5287541868276bb7ee538afcc'} {{(pid=80144) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 10:56:15.660186 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.notifications [None req-bc873945-4c0b-4ce4-a09a-bb240207adce admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '792c13a5287541868276bb7ee538afcc'} {{(pid=80144) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 10:56:15.660816 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.notifications [None req-bc873945-4c0b-4ce4-a09a-bb240207adce admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '792c13a5287541868276bb7ee538afcc'} {{(pid=80144) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 10:56:15.678078 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 1093/2156] 10.222.0.25 () {58 vars in 1224 bytes} [Sun Aug 7 10:56:15 2022] DELETE /identity/v3/users/792c13a5287541868276bb7ee538afcc => generated 0 bytes in 201 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 07 10:56:15.682496 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3e89024-b0e1-4b66-957f-5187f591f13c None None] REQUEST_METHOD: `POST` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:56:15.682984 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3e89024-b0e1-4b66-957f-5187f591f13c None None] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:56:15.683094 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3e89024-b0e1-4b66-957f-5187f591f13c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:56:15.702547 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.auth.core [None req-d3e89024-b0e1-4b66-957f-5187f591f13c None None] MFA Rules not processed for user `bac4e6b76fe64217ac8fa44e71cffe6a`. Rule list: `[]` (Enabled: `True`). {{(pid=80143) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 10:56:15.703803 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-d3e89024-b0e1-4b66-957f-5187f591f13c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 10:56:15.761561 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1064/2157] 10.222.0.25 () {58 vars in 958 bytes} [Sun Aug 7 10:56:15 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 07 10:56:15.772034 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc873945-4c0b-4ce4-a09a-bb240207adce admin admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 10:56:15.773443 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.fernet_utils [None req-bc873945-4c0b-4ce4-a09a-bb240207adce admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80144) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 10:56:15.832318 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc873945-4c0b-4ce4-a09a-bb240207adce admin admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 10:56:15.836314 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2645f3f8-ed6c-4e1d-a1c3-65f707ac2926 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be1283c9bf47424c9aa3ad27cab03027', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6de1f3cac02c4bcdb05dd2337176033d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 10:56:15.850700 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2645f3f8-ed6c-4e1d-a1c3-65f707ac2926 service neutron] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:56:15.850700 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2645f3f8-ed6c-4e1d-a1c3-65f707ac2926 service neutron] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:56:15.850700 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2645f3f8-ed6c-4e1d-a1c3-65f707ac2926 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:56:15.850700 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2645f3f8-ed6c-4e1d-a1c3-65f707ac2926 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 10:56:15.850700 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2645f3f8-ed6c-4e1d-a1c3-65f707ac2926 service neutron] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 10:56:15.858641 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 1094/2158] 10.222.0.25 () {60 vars in 1365 bytes} [Sun Aug 7 10:56:15 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 07 10:56:16.189890 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3e89024-b0e1-4b66-957f-5187f591f13c None None] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 10:56:16.191787 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-d3e89024-b0e1-4b66-957f-5187f591f13c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 10:56:16.210003 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3e89024-b0e1-4b66-957f-5187f591f13c None None] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 10:56:16.215213 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bcc6f88-c4d2-4711-bad1-89eb80b311a7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5ac47e3b09f43d19a23a0f457fe9e23', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 10:56:16.216481 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bcc6f88-c4d2-4711-bad1-89eb80b311a7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:56:16.216807 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bcc6f88-c4d2-4711-bad1-89eb80b311a7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:56:16.217178 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bcc6f88-c4d2-4711-bad1-89eb80b311a7 admin admin] PATH_INFO: `/v3/projects/658b7a570d7d40fc837a988ca549a811` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:56:16.218469 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bcc6f88-c4d2-4711-bad1-89eb80b311a7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=658b7a570d7d40fc837a988ca549a811)` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 10:56:16.220560 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bcc6f88-c4d2-4711-bad1-89eb80b311a7 admin admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 10:56:16.273614 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-2bcc6f88-c4d2-4711-bad1-89eb80b311a7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '658b7a570d7d40fc837a988ca549a811', 'request_id': 'req-2bcc6f88-c4d2-4711-bad1-89eb80b311a7'} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 10:56:16.280545 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-2bcc6f88-c4d2-4711-bad1-89eb80b311a7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '658b7a570d7d40fc837a988ca549a811', 'request_id': 'req-2bcc6f88-c4d2-4711-bad1-89eb80b311a7'} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 10:56:16.287548 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-2bcc6f88-c4d2-4711-bad1-89eb80b311a7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '658b7a570d7d40fc837a988ca549a811', 'request_id': 'req-2bcc6f88-c4d2-4711-bad1-89eb80b311a7'} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 10:56:16.300829 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-2bcc6f88-c4d2-4711-bad1-89eb80b311a7 admin admin] The token cache is being invalidate because project 658b7a570d7d40fc837a988ca549a811 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80143) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 07 10:56:16.301127 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-2bcc6f88-c4d2-4711-bad1-89eb80b311a7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 10:56:16.301757 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-2bcc6f88-c4d2-4711-bad1-89eb80b311a7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 10:56:16.303496 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1065/2159] 10.222.0.25 () {58 vars in 1233 bytes} [Sun Aug 7 10:56:16 2022] DELETE /identity/v3/projects/658b7a570d7d40fc837a988ca549a811 => generated 0 bytes in 114 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 07 10:56:16.309468 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-580070df-4662-40fb-9c78-ebc2a3fd8c7d None None] REQUEST_METHOD: `POST` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:56:16.309771 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-580070df-4662-40fb-9c78-ebc2a3fd8c7d None None] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:56:16.310054 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-580070df-4662-40fb-9c78-ebc2a3fd8c7d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:56:16.330596 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.auth.core [None req-580070df-4662-40fb-9c78-ebc2a3fd8c7d None None] MFA Rules not processed for user `bac4e6b76fe64217ac8fa44e71cffe6a`. Rule list: `[]` (Enabled: `True`). {{(pid=80144) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 10:56:16.331835 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.fernet_utils [None req-580070df-4662-40fb-9c78-ebc2a3fd8c7d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80144) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 10:56:16.350476 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 10:56:16.403148 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 1095/2160] 10.222.0.25 () {58 vars in 958 bytes} [Sun Aug 7 10:56:16 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 07 10:56:16.412149 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f7a058e-328d-44ab-8a00-9a4330ed760f None None] REQUEST_METHOD: `POST` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:56:16.412442 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f7a058e-328d-44ab-8a00-9a4330ed760f None None] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:56:16.412721 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f7a058e-328d-44ab-8a00-9a4330ed760f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:56:16.437471 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.auth.core [None req-8f7a058e-328d-44ab-8a00-9a4330ed760f None None] MFA Rules not processed for user `bac4e6b76fe64217ac8fa44e71cffe6a`. Rule list: `[]` (Enabled: `True`). {{(pid=80143) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 10:56:16.438693 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-8f7a058e-328d-44ab-8a00-9a4330ed760f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 10:56:16.446846 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1066/2161] 10.222.0.25 () {58 vars in 958 bytes} [Sun Aug 7 10:56:16 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 07 10:56:16.452392 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-580070df-4662-40fb-9c78-ebc2a3fd8c7d None None] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 10:56:16.464800 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-580070df-4662-40fb-9c78-ebc2a3fd8c7d None None] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 10:56:16.470577 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a20b2f49-7262-4247-9ba5-a5c362a6bd5b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5ac47e3b09f43d19a23a0f457fe9e23', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 10:56:16.471226 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a20b2f49-7262-4247-9ba5-a5c362a6bd5b admin admin] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:56:16.471512 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a20b2f49-7262-4247-9ba5-a5c362a6bd5b admin admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:56:16.471790 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a20b2f49-7262-4247-9ba5-a5c362a6bd5b admin admin] PATH_INFO: `/v3/domains` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:56:16.472444 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a20b2f49-7262-4247-9ba5-a5c362a6bd5b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80144) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 10:56:16.472752 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a20b2f49-7262-4247-9ba5-a5c362a6bd5b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 10:56:16.474374 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a20b2f49-7262-4247-9ba5-a5c362a6bd5b admin admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 10:56:16.487071 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 1096/2162] 10.222.0.25 () {58 vars in 1153 bytes} [Sun Aug 7 10:56:16 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 10:56:16.491620 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f7a058e-328d-44ab-8a00-9a4330ed760f None None] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 10:56:16.501401 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f7a058e-328d-44ab-8a00-9a4330ed760f None None] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 10:56:16.508174 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1197fbd4-b10e-43d7-b65e-6b512ae1bf8f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5ac47e3b09f43d19a23a0f457fe9e23', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 10:56:16.508904 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1197fbd4-b10e-43d7-b65e-6b512ae1bf8f admin admin] REQUEST_METHOD: `POST` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:56:16.509290 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1197fbd4-b10e-43d7-b65e-6b512ae1bf8f admin admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:56:16.509663 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1197fbd4-b10e-43d7-b65e-6b512ae1bf8f admin admin] PATH_INFO: `/v3/projects` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:56:16.510434 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1197fbd4-b10e-43d7-b65e-6b512ae1bf8f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 10:56:16.513376 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1197fbd4-b10e-43d7-b65e-6b512ae1bf8f admin admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 10:56:16.555928 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1067/2163] 10.222.0.25 () {60 vars in 1153 bytes} [Sun Aug 7 10:56:16 2022] POST /identity/v3/projects => generated 371 bytes in 62 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 07 10:56:16.558167 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a20b2f49-7262-4247-9ba5-a5c362a6bd5b admin admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 10:56:16.590385 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a20b2f49-7262-4247-9ba5-a5c362a6bd5b admin admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 10:56:16.688388 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6686a736-c55f-49d1-952d-de849a697ce0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5ac47e3b09f43d19a23a0f457fe9e23', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 10:56:16.689122 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6686a736-c55f-49d1-952d-de849a697ce0 admin admin] REQUEST_METHOD: `POST` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:56:16.689473 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6686a736-c55f-49d1-952d-de849a697ce0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:56:16.689821 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6686a736-c55f-49d1-952d-de849a697ce0 admin admin] PATH_INFO: `/v3/users` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:56:16.690533 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6686a736-c55f-49d1-952d-de849a697ce0 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 10:56:16.692581 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6686a736-c55f-49d1-952d-de849a697ce0 admin admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 10:56:16.714528 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 1097/2164] 10.222.0.25 () {60 vars in 1144 bytes} [Sun Aug 7 10:56:16 2022] POST /identity/v3/users => generated 286 bytes in 157 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 07 10:56:16.719206 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1197fbd4-b10e-43d7-b65e-6b512ae1bf8f admin admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 10:56:16.732283 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1197fbd4-b10e-43d7-b65e-6b512ae1bf8f admin admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 10:56:16.737143 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-544de197-f1e1-4f40-97de-fbbaded1a9f3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5ac47e3b09f43d19a23a0f457fe9e23', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 10:56:16.737812 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-544de197-f1e1-4f40-97de-fbbaded1a9f3 admin admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:56:16.738103 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-544de197-f1e1-4f40-97de-fbbaded1a9f3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:56:16.738381 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-544de197-f1e1-4f40-97de-fbbaded1a9f3 admin admin] PATH_INFO: `/v3/roles` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:56:16.739163 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-544de197-f1e1-4f40-97de-fbbaded1a9f3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 10:56:16.740829 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-544de197-f1e1-4f40-97de-fbbaded1a9f3 admin admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 10:56:16.751891 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1068/2165] 10.222.0.25 () {58 vars in 1122 bytes} [Sun Aug 7 10:56:16 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 07 10:56:16.755639 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6686a736-c55f-49d1-952d-de849a697ce0 admin admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 10:56:16.767352 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6686a736-c55f-49d1-952d-de849a697ce0 admin admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 10:56:16.772092 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5341bd34-339f-49aa-8f9d-b72dad22ae5e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5ac47e3b09f43d19a23a0f457fe9e23', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 10:56:16.772696 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5341bd34-339f-49aa-8f9d-b72dad22ae5e admin admin] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:56:16.772980 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5341bd34-339f-49aa-8f9d-b72dad22ae5e admin admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:56:16.773255 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5341bd34-339f-49aa-8f9d-b72dad22ae5e admin admin] PATH_INFO: `/v3/roles` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:56:16.773899 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5341bd34-339f-49aa-8f9d-b72dad22ae5e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 10:56:16.775523 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5341bd34-339f-49aa-8f9d-b72dad22ae5e admin admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 10:56:16.786800 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 1098/2166] 10.222.0.25 () {58 vars in 1122 bytes} [Sun Aug 7 10:56:16 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 07 10:56:16.791223 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-544de197-f1e1-4f40-97de-fbbaded1a9f3 admin admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 10:56:16.802875 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-544de197-f1e1-4f40-97de-fbbaded1a9f3 admin admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 10:56:16.807972 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d514ae1-718f-4e0c-99be-e7d1d3970f34 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5ac47e3b09f43d19a23a0f457fe9e23', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 10:56:16.808660 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d514ae1-718f-4e0c-99be-e7d1d3970f34 admin admin] REQUEST_METHOD: `PUT` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:56:16.808945 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d514ae1-718f-4e0c-99be-e7d1d3970f34 admin admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:56:16.809255 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d514ae1-718f-4e0c-99be-e7d1d3970f34 admin admin] PATH_INFO: `/v3/projects/758665157bfd4f7fa6219f898a9c99a2/users/8cb8ad1bff7f4e439cefdf6f0ef9a2d0/roles/b98dfb3b9fee4644af2887a3ac3fd2e6` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:56:16.825777 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d514ae1-718f-4e0c-99be-e7d1d3970f34 admin admin] RBAC: Authorizing `identity:create_grant(project_id=758665157bfd4f7fa6219f898a9c99a2, user_id=8cb8ad1bff7f4e439cefdf6f0ef9a2d0, role_id=b98dfb3b9fee4644af2887a3ac3fd2e6)` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 10:56:16.828505 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d514ae1-718f-4e0c-99be-e7d1d3970f34 admin admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 10:56:16.844275 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1069/2167] 10.222.0.25 () {60 vars in 1483 bytes} [Sun Aug 7 10:56:16 2022] PUT /identity/v3/projects/758665157bfd4f7fa6219f898a9c99a2/users/8cb8ad1bff7f4e439cefdf6f0ef9a2d0/roles/b98dfb3b9fee4644af2887a3ac3fd2e6 => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 07 10:56:16.849770 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-987f35a7-b6b3-46a9-8b09-c77e2b537794 None None] REQUEST_METHOD: `POST` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:56:16.850140 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-987f35a7-b6b3-46a9-8b09-c77e2b537794 None None] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:56:16.850487 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-987f35a7-b6b3-46a9-8b09-c77e2b537794 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:56:16.874769 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.auth.core [None req-987f35a7-b6b3-46a9-8b09-c77e2b537794 None None] MFA Rules not processed for user `bac4e6b76fe64217ac8fa44e71cffe6a`. Rule list: `[]` (Enabled: `True`). {{(pid=80144) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 10:56:16.876152 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.fernet_utils [None req-987f35a7-b6b3-46a9-8b09-c77e2b537794 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80144) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 10:56:16.940388 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 1099/2168] 10.222.0.25 () {58 vars in 958 bytes} [Sun Aug 7 10:56:16 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 07 10:56:16.951064 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d514ae1-718f-4e0c-99be-e7d1d3970f34 admin admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 10:56:16.952805 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-2d514ae1-718f-4e0c-99be-e7d1d3970f34 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 10:56:16.976605 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 10:56:17.018363 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d514ae1-718f-4e0c-99be-e7d1d3970f34 admin admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 10:56:17.022332 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df59580a-71d4-4cb6-aba8-25a9c3fdd4b9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be1283c9bf47424c9aa3ad27cab03027', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6de1f3cac02c4bcdb05dd2337176033d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 10:56:17.022957 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df59580a-71d4-4cb6-aba8-25a9c3fdd4b9 service neutron] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:56:17.023249 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df59580a-71d4-4cb6-aba8-25a9c3fdd4b9 service neutron] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:56:17.023508 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df59580a-71d4-4cb6-aba8-25a9c3fdd4b9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:56:17.033357 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df59580a-71d4-4cb6-aba8-25a9c3fdd4b9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 10:56:17.034787 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df59580a-71d4-4cb6-aba8-25a9c3fdd4b9 service neutron] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 10:56:17.044811 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1070/2169] 10.222.0.25 () {60 vars in 1365 bytes} [Sun Aug 7 10:56:16 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 07 10:56:22.211975 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8717cd12-bf47-4f25-ab6e-69291a1facd1 None None] REQUEST_METHOD: `POST` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:56:22.212493 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8717cd12-bf47-4f25-ab6e-69291a1facd1 None None] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:56:22.212585 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8717cd12-bf47-4f25-ab6e-69291a1facd1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:56:22.245477 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.auth.core [None req-8717cd12-bf47-4f25-ab6e-69291a1facd1 None None] MFA Rules not processed for user `8cb8ad1bff7f4e439cefdf6f0ef9a2d0`. Rule list: `[]` (Enabled: `True`). {{(pid=80144) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 10:56:22.246758 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.fernet_utils [None req-8717cd12-bf47-4f25-ab6e-69291a1facd1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80144) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 10:56:22.265030 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 10:56:22.317519 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 1100/2170] 10.222.0.25 () {58 vars in 958 bytes} [Sun Aug 7 10:56:22 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 07 10:56:22.322017 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df59580a-71d4-4cb6-aba8-25a9c3fdd4b9 service neutron] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 10:56:22.332191 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df59580a-71d4-4cb6-aba8-25a9c3fdd4b9 service neutron] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 10:56:22.336819 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11100246-6386-45d9-9f28-96da27ebd7a6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5ac47e3b09f43d19a23a0f457fe9e23', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 10:56:22.337427 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11100246-6386-45d9-9f28-96da27ebd7a6 admin admin] REQUEST_METHOD: `POST` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:56:22.337704 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11100246-6386-45d9-9f28-96da27ebd7a6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:56:22.337972 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11100246-6386-45d9-9f28-96da27ebd7a6 admin admin] PATH_INFO: `/v3/projects` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:56:22.338595 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11100246-6386-45d9-9f28-96da27ebd7a6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 10:56:22.340866 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11100246-6386-45d9-9f28-96da27ebd7a6 admin admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 10:56:22.370989 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1071/2171] 10.222.0.25 () {60 vars in 1153 bytes} [Sun Aug 7 10:56:22 2022] POST /identity/v3/projects => generated 373 bytes in 49 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 07 10:56:22.375383 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8717cd12-bf47-4f25-ab6e-69291a1facd1 None None] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 10:56:22.386497 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8717cd12-bf47-4f25-ab6e-69291a1facd1 None None] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 10:56:22.451565 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d60aaa78-9881-479e-87e5-a3f31f3224e3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5ac47e3b09f43d19a23a0f457fe9e23', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 10:56:22.452190 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d60aaa78-9881-479e-87e5-a3f31f3224e3 admin admin] REQUEST_METHOD: `POST` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:56:22.452470 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d60aaa78-9881-479e-87e5-a3f31f3224e3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:56:22.452745 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d60aaa78-9881-479e-87e5-a3f31f3224e3 admin admin] PATH_INFO: `/v3/users` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:56:22.453389 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d60aaa78-9881-479e-87e5-a3f31f3224e3 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 10:56:22.455359 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d60aaa78-9881-479e-87e5-a3f31f3224e3 admin admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 10:56:22.482725 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 1101/2172] 10.222.0.25 () {60 vars in 1144 bytes} [Sun Aug 7 10:56:22 2022] POST /identity/v3/users => generated 293 bytes in 105 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 07 10:56:22.484362 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11100246-6386-45d9-9f28-96da27ebd7a6 admin admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 10:56:22.495452 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11100246-6386-45d9-9f28-96da27ebd7a6 admin admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 10:56:22.501002 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4df261b0-7484-4c6d-9071-ee5b878cb5d0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5ac47e3b09f43d19a23a0f457fe9e23', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 10:56:22.501620 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4df261b0-7484-4c6d-9071-ee5b878cb5d0 admin admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:56:22.501899 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4df261b0-7484-4c6d-9071-ee5b878cb5d0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:56:22.502175 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4df261b0-7484-4c6d-9071-ee5b878cb5d0 admin admin] PATH_INFO: `/v3/roles` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:56:22.502820 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4df261b0-7484-4c6d-9071-ee5b878cb5d0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 10:56:22.504445 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4df261b0-7484-4c6d-9071-ee5b878cb5d0 admin admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 10:56:22.518162 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1072/2173] 10.222.0.25 () {58 vars in 1122 bytes} [Sun Aug 7 10:56:22 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 07 10:56:22.520954 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d60aaa78-9881-479e-87e5-a3f31f3224e3 admin admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 10:56:22.532358 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d60aaa78-9881-479e-87e5-a3f31f3224e3 admin admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 10:56:22.537351 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e610e7d-fd40-419b-961e-35b162a094e3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5ac47e3b09f43d19a23a0f457fe9e23', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 10:56:22.538120 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e610e7d-fd40-419b-961e-35b162a094e3 admin admin] REQUEST_METHOD: `PUT` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:56:22.538403 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e610e7d-fd40-419b-961e-35b162a094e3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:56:22.538679 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e610e7d-fd40-419b-961e-35b162a094e3 admin admin] PATH_INFO: `/v3/projects/25a95f83a19142bf817615584a5c5257/users/fca1beb3c14b4c6f8085a09697736f51/roles/81b1f1bcee2a473088fbbd327c1c5b23` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:56:22.557517 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e610e7d-fd40-419b-961e-35b162a094e3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=25a95f83a19142bf817615584a5c5257, user_id=fca1beb3c14b4c6f8085a09697736f51, role_id=81b1f1bcee2a473088fbbd327c1c5b23)` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 10:56:22.560866 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e610e7d-fd40-419b-961e-35b162a094e3 admin admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 10:56:22.581969 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 1102/2174] 10.222.0.25 () {60 vars in 1483 bytes} [Sun Aug 7 10:56:22 2022] PUT /identity/v3/projects/25a95f83a19142bf817615584a5c5257/users/fca1beb3c14b4c6f8085a09697736f51/roles/81b1f1bcee2a473088fbbd327c1c5b23 => generated 0 bytes in 61 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 07 10:56:22.586056 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4df261b0-7484-4c6d-9071-ee5b878cb5d0 admin admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 10:56:22.602691 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4df261b0-7484-4c6d-9071-ee5b878cb5d0 admin admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 10:56:22.717738 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07cebaee-421b-4cf6-bae4-e43f8bb0b45e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5ac47e3b09f43d19a23a0f457fe9e23', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 10:56:22.718377 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07cebaee-421b-4cf6-bae4-e43f8bb0b45e admin admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:56:22.718657 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07cebaee-421b-4cf6-bae4-e43f8bb0b45e admin admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:56:22.718962 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07cebaee-421b-4cf6-bae4-e43f8bb0b45e admin admin] PATH_INFO: `/v3/roles` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:56:22.720251 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07cebaee-421b-4cf6-bae4-e43f8bb0b45e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 10:56:22.721295 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07cebaee-421b-4cf6-bae4-e43f8bb0b45e admin admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 10:56:22.735402 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1073/2175] 10.222.0.25 () {58 vars in 1122 bytes} [Sun Aug 7 10:56:22 2022] GET /identity/v3/roles => generated 1420 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 07 10:56:22.739101 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e610e7d-fd40-419b-961e-35b162a094e3 admin admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 10:56:22.750650 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e610e7d-fd40-419b-961e-35b162a094e3 admin admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 10:56:22.756708 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-821a74fa-193f-417f-970b-c17273b6cd89 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5ac47e3b09f43d19a23a0f457fe9e23', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 10:56:22.757436 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-821a74fa-193f-417f-970b-c17273b6cd89 admin admin] REQUEST_METHOD: `PUT` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:56:22.757793 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-821a74fa-193f-417f-970b-c17273b6cd89 admin admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:56:22.758143 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-821a74fa-193f-417f-970b-c17273b6cd89 admin admin] PATH_INFO: `/v3/projects/25a95f83a19142bf817615584a5c5257/users/fca1beb3c14b4c6f8085a09697736f51/roles/81b1f1bcee2a473088fbbd327c1c5b23` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:56:22.760471 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-821a74fa-193f-417f-970b-c17273b6cd89 admin admin] RBAC: Authorizing `identity:create_grant(project_id=25a95f83a19142bf817615584a5c5257, user_id=fca1beb3c14b4c6f8085a09697736f51, role_id=81b1f1bcee2a473088fbbd327c1c5b23)` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 10:56:22.763516 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-821a74fa-193f-417f-970b-c17273b6cd89 admin admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 10:56:22.779867 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 1103/2176] 10.222.0.25 () {60 vars in 1483 bytes} [Sun Aug 7 10:56:22 2022] PUT /identity/v3/projects/25a95f83a19142bf817615584a5c5257/users/fca1beb3c14b4c6f8085a09697736f51/roles/81b1f1bcee2a473088fbbd327c1c5b23 => generated 0 bytes in 41 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 07 10:56:28.425651 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed7575f8-2bff-4ef9-a8b4-0f395f24f165 None None] REQUEST_METHOD: `POST` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:56:28.426181 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed7575f8-2bff-4ef9-a8b4-0f395f24f165 None None] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:56:28.426181 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed7575f8-2bff-4ef9-a8b4-0f395f24f165 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:56:28.503526 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.auth.core [None req-ed7575f8-2bff-4ef9-a8b4-0f395f24f165 None None] MFA Rules not processed for user `fca1beb3c14b4c6f8085a09697736f51`. Rule list: `[]` (Enabled: `True`). {{(pid=80143) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 10:56:28.506597 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-ed7575f8-2bff-4ef9-a8b4-0f395f24f165 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 10:56:28.694629 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1074/2177] 10.222.0.25 () {58 vars in 958 bytes} [Sun Aug 7 10:56:28 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 270 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 07 10:56:28.787790 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-821a74fa-193f-417f-970b-c17273b6cd89 admin admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 10:56:28.790845 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.fernet_utils [None req-821a74fa-193f-417f-970b-c17273b6cd89 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80144) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 10:56:28.923652 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-821a74fa-193f-417f-970b-c17273b6cd89 admin admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 10:56:28.929576 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66a13b4b-519e-435a-92c9-4d550a095049 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31d16d1b96104b4eb690398432b9829e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6de1f3cac02c4bcdb05dd2337176033d', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'reader', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 10:56:28.930209 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66a13b4b-519e-435a-92c9-4d550a095049 service nova] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:56:28.930486 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66a13b4b-519e-435a-92c9-4d550a095049 service nova] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:56:28.931003 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66a13b4b-519e-435a-92c9-4d550a095049 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:56:28.936987 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 10:56:29.036522 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66a13b4b-519e-435a-92c9-4d550a095049 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 10:56:29.038285 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66a13b4b-519e-435a-92c9-4d550a095049 service nova] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 10:56:29.062414 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 1104/2178] 10.222.0.25 () {60 vars in 1365 bytes} [Sun Aug 7 10:56:28 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 274 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 07 10:56:30.558699 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cae8cbca-cb8a-4fdb-8000-bd048188589c None None] REQUEST_METHOD: `POST` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:56:30.559456 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cae8cbca-cb8a-4fdb-8000-bd048188589c None None] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:56:30.560342 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cae8cbca-cb8a-4fdb-8000-bd048188589c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:56:30.585677 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.auth.core [None req-cae8cbca-cb8a-4fdb-8000-bd048188589c None None] MFA Rules not processed for user `8cb8ad1bff7f4e439cefdf6f0ef9a2d0`. Rule list: `[]` (Enabled: `True`). {{(pid=80143) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 10:56:30.587351 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-cae8cbca-cb8a-4fdb-8000-bd048188589c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 10:56:30.596022 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1075/2179] 10.222.0.25 () {58 vars in 958 bytes} [Sun Aug 7 10:56:30 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 07 10:56:34.781480 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66a13b4b-519e-435a-92c9-4d550a095049 service nova] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 10:56:34.783105 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.fernet_utils [None req-66a13b4b-519e-435a-92c9-4d550a095049 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80144) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 10:56:34.883601 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66a13b4b-519e-435a-92c9-4d550a095049 service nova] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 10:56:34.896990 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d1ff85c-b325-448c-9ae3-9a9fd79f205c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5ac47e3b09f43d19a23a0f457fe9e23', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 10:56:34.896990 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d1ff85c-b325-448c-9ae3-9a9fd79f205c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:56:34.897509 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d1ff85c-b325-448c-9ae3-9a9fd79f205c admin admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:56:34.897940 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d1ff85c-b325-448c-9ae3-9a9fd79f205c admin admin] PATH_INFO: `/v3/users/616a497708664cca8bf7cfe93cbfd01f` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:56:34.899905 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d1ff85c-b325-448c-9ae3-9a9fd79f205c admin admin] RBAC: Authorizing `identity:delete_user(user_id=616a497708664cca8bf7cfe93cbfd01f)` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 10:56:34.906344 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d1ff85c-b325-448c-9ae3-9a9fd79f205c admin admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 10:56:34.994314 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.notifications [None req-7d1ff85c-b325-448c-9ae3-9a9fd79f205c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '616a497708664cca8bf7cfe93cbfd01f'} {{(pid=80144) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 10:56:35.038130 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.notifications [None req-7d1ff85c-b325-448c-9ae3-9a9fd79f205c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '616a497708664cca8bf7cfe93cbfd01f'} {{(pid=80144) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 10:56:35.057647 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.notifications [None req-7d1ff85c-b325-448c-9ae3-9a9fd79f205c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '616a497708664cca8bf7cfe93cbfd01f'} {{(pid=80144) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 10:56:35.068854 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.notifications [None req-7d1ff85c-b325-448c-9ae3-9a9fd79f205c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '616a497708664cca8bf7cfe93cbfd01f'} {{(pid=80144) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 10:56:35.069870 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.notifications [None req-7d1ff85c-b325-448c-9ae3-9a9fd79f205c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '616a497708664cca8bf7cfe93cbfd01f'} {{(pid=80144) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 10:56:35.104960 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 1105/2180] 10.222.0.25 () {58 vars in 1224 bytes} [Sun Aug 7 10:56:34 2022] DELETE /identity/v3/users/616a497708664cca8bf7cfe93cbfd01f => generated 0 bytes in 324 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 07 10:56:35.310140 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cae8cbca-cb8a-4fdb-8000-bd048188589c None None] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 10:56:35.318972 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 10:56:35.329649 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cae8cbca-cb8a-4fdb-8000-bd048188589c None None] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 10:56:35.405136 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39ffb474-1a8d-4328-809a-2be5e9025217 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5ac47e3b09f43d19a23a0f457fe9e23', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 10:56:35.405946 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39ffb474-1a8d-4328-809a-2be5e9025217 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:56:35.406299 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39ffb474-1a8d-4328-809a-2be5e9025217 admin admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:56:35.406651 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39ffb474-1a8d-4328-809a-2be5e9025217 admin admin] PATH_INFO: `/v3/projects/2d2c614e07df4a29b2cd84c260fdc93a` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:56:35.408011 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39ffb474-1a8d-4328-809a-2be5e9025217 admin admin] RBAC: Authorizing `identity:delete_project(project_id=2d2c614e07df4a29b2cd84c260fdc93a)` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 10:56:35.410359 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39ffb474-1a8d-4328-809a-2be5e9025217 admin admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 10:56:35.467257 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-39ffb474-1a8d-4328-809a-2be5e9025217 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2d2c614e07df4a29b2cd84c260fdc93a', 'request_id': 'req-39ffb474-1a8d-4328-809a-2be5e9025217'} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 10:56:35.477048 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-39ffb474-1a8d-4328-809a-2be5e9025217 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2d2c614e07df4a29b2cd84c260fdc93a', 'request_id': 'req-39ffb474-1a8d-4328-809a-2be5e9025217'} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 10:56:35.485491 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-39ffb474-1a8d-4328-809a-2be5e9025217 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2d2c614e07df4a29b2cd84c260fdc93a', 'request_id': 'req-39ffb474-1a8d-4328-809a-2be5e9025217'} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 10:56:35.505917 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-39ffb474-1a8d-4328-809a-2be5e9025217 admin admin] The token cache is being invalidate because project 2d2c614e07df4a29b2cd84c260fdc93a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80143) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 07 10:56:35.506229 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-39ffb474-1a8d-4328-809a-2be5e9025217 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 10:56:35.506980 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-39ffb474-1a8d-4328-809a-2be5e9025217 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 10:56:35.508610 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1076/2181] 10.222.0.25 () {58 vars in 1233 bytes} [Sun Aug 7 10:56:35 2022] DELETE /identity/v3/projects/2d2c614e07df4a29b2cd84c260fdc93a => generated 0 bytes in 200 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 07 10:56:35.515313 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8724be16-63ea-4628-8eda-e6f5ef7c9edf None None] REQUEST_METHOD: `POST` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:56:35.515607 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8724be16-63ea-4628-8eda-e6f5ef7c9edf None None] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:56:35.515885 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8724be16-63ea-4628-8eda-e6f5ef7c9edf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:56:35.541228 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.auth.core [None req-8724be16-63ea-4628-8eda-e6f5ef7c9edf None None] MFA Rules not processed for user `bac4e6b76fe64217ac8fa44e71cffe6a`. Rule list: `[]` (Enabled: `True`). {{(pid=80144) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 10:56:35.542640 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.fernet_utils [None req-8724be16-63ea-4628-8eda-e6f5ef7c9edf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80144) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 10:56:35.566398 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 10:56:35.618484 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 1106/2182] 10.222.0.25 () {58 vars in 958 bytes} [Sun Aug 7 10:56:35 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 07 10:56:35.627447 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d695aa7-34a4-4ebb-be5a-5506c5483220 None None] REQUEST_METHOD: `POST` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:56:35.627637 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d695aa7-34a4-4ebb-be5a-5506c5483220 None None] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:56:35.627902 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d695aa7-34a4-4ebb-be5a-5506c5483220 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:56:35.660351 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.auth.core [None req-0d695aa7-34a4-4ebb-be5a-5506c5483220 None None] MFA Rules not processed for user `bac4e6b76fe64217ac8fa44e71cffe6a`. Rule list: `[]` (Enabled: `True`). {{(pid=80143) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 10:56:35.661681 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-0d695aa7-34a4-4ebb-be5a-5506c5483220 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 10:56:35.671280 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1077/2183] 10.222.0.25 () {58 vars in 958 bytes} [Sun Aug 7 10:56:35 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 45 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 07 10:56:35.675635 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8724be16-63ea-4628-8eda-e6f5ef7c9edf None None] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 10:56:35.686023 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8724be16-63ea-4628-8eda-e6f5ef7c9edf None None] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 10:56:35.691229 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ade2d103-2e38-4bcc-8129-b6cc075a2d8f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5ac47e3b09f43d19a23a0f457fe9e23', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 10:56:35.692338 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ade2d103-2e38-4bcc-8129-b6cc075a2d8f admin admin] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:56:35.692639 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ade2d103-2e38-4bcc-8129-b6cc075a2d8f admin admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:56:35.693962 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ade2d103-2e38-4bcc-8129-b6cc075a2d8f admin admin] PATH_INFO: `/v3/domains` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:56:35.694620 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ade2d103-2e38-4bcc-8129-b6cc075a2d8f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80144) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 10:56:35.694939 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ade2d103-2e38-4bcc-8129-b6cc075a2d8f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 10:56:35.696604 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ade2d103-2e38-4bcc-8129-b6cc075a2d8f admin admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 10:56:35.711830 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 1107/2184] 10.222.0.25 () {58 vars in 1153 bytes} [Sun Aug 7 10:56:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 10:56:35.713137 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d695aa7-34a4-4ebb-be5a-5506c5483220 None None] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 10:56:35.726507 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d695aa7-34a4-4ebb-be5a-5506c5483220 None None] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 10:56:35.731811 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72dca467-8514-4241-b33e-34695dfb3928 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5ac47e3b09f43d19a23a0f457fe9e23', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 10:56:35.732449 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72dca467-8514-4241-b33e-34695dfb3928 admin admin] REQUEST_METHOD: `POST` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:56:35.732729 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72dca467-8514-4241-b33e-34695dfb3928 admin admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:56:35.733263 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72dca467-8514-4241-b33e-34695dfb3928 admin admin] PATH_INFO: `/v3/projects` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:56:35.733910 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72dca467-8514-4241-b33e-34695dfb3928 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 10:56:35.735853 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72dca467-8514-4241-b33e-34695dfb3928 admin admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 10:56:35.764960 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1078/2185] 10.222.0.25 () {60 vars in 1153 bytes} [Sun Aug 7 10:56:35 2022] POST /identity/v3/projects => generated 385 bytes in 52 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 07 10:56:35.769141 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ade2d103-2e38-4bcc-8129-b6cc075a2d8f admin admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 10:56:35.779326 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ade2d103-2e38-4bcc-8129-b6cc075a2d8f admin admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 10:56:35.838077 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73236d78-42b1-45be-bb86-9342b2f02d9e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5ac47e3b09f43d19a23a0f457fe9e23', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 10:56:35.838707 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73236d78-42b1-45be-bb86-9342b2f02d9e admin admin] REQUEST_METHOD: `POST` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:56:35.839018 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73236d78-42b1-45be-bb86-9342b2f02d9e admin admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:56:35.839300 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73236d78-42b1-45be-bb86-9342b2f02d9e admin admin] PATH_INFO: `/v3/users` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:56:35.839926 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73236d78-42b1-45be-bb86-9342b2f02d9e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 10:56:35.842118 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73236d78-42b1-45be-bb86-9342b2f02d9e admin admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 10:56:35.863524 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 1108/2186] 10.222.0.25 () {60 vars in 1144 bytes} [Sun Aug 7 10:56:35 2022] POST /identity/v3/users => generated 293 bytes in 95 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 07 10:56:35.867911 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72dca467-8514-4241-b33e-34695dfb3928 admin admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 10:56:35.879134 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72dca467-8514-4241-b33e-34695dfb3928 admin admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 10:56:35.883717 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d076f2f-a1f5-4fd3-82ea-e8803d77e530 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5ac47e3b09f43d19a23a0f457fe9e23', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 10:56:35.884314 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d076f2f-a1f5-4fd3-82ea-e8803d77e530 admin admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:56:35.884595 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d076f2f-a1f5-4fd3-82ea-e8803d77e530 admin admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:56:35.884869 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d076f2f-a1f5-4fd3-82ea-e8803d77e530 admin admin] PATH_INFO: `/v3/roles` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:56:35.885511 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d076f2f-a1f5-4fd3-82ea-e8803d77e530 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 10:56:35.887140 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d076f2f-a1f5-4fd3-82ea-e8803d77e530 admin admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 10:56:35.897251 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1079/2187] 10.222.0.25 () {58 vars in 1122 bytes} [Sun Aug 7 10:56:35 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 07 10:56:35.901324 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73236d78-42b1-45be-bb86-9342b2f02d9e admin admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 10:56:35.911476 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73236d78-42b1-45be-bb86-9342b2f02d9e admin admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 10:56:35.917165 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9001930f-eadf-4c68-bd2a-77e3342f8156 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5ac47e3b09f43d19a23a0f457fe9e23', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 10:56:35.917776 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9001930f-eadf-4c68-bd2a-77e3342f8156 admin admin] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:56:35.918602 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9001930f-eadf-4c68-bd2a-77e3342f8156 admin admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:56:35.922254 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9001930f-eadf-4c68-bd2a-77e3342f8156 admin admin] PATH_INFO: `/v3/roles` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:56:35.923027 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9001930f-eadf-4c68-bd2a-77e3342f8156 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 10:56:35.924639 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9001930f-eadf-4c68-bd2a-77e3342f8156 admin admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 10:56:35.939308 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 1109/2188] 10.222.0.25 () {58 vars in 1122 bytes} [Sun Aug 7 10:56:35 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 07 10:56:35.945290 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d076f2f-a1f5-4fd3-82ea-e8803d77e530 admin admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 10:56:35.956261 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d076f2f-a1f5-4fd3-82ea-e8803d77e530 admin admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 10:56:35.961014 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-786f1fbb-da73-4b11-a1a6-8a2cacc51205 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5ac47e3b09f43d19a23a0f457fe9e23', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 10:56:35.961793 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-786f1fbb-da73-4b11-a1a6-8a2cacc51205 admin admin] REQUEST_METHOD: `PUT` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:56:35.962083 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-786f1fbb-da73-4b11-a1a6-8a2cacc51205 admin admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:56:35.962364 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-786f1fbb-da73-4b11-a1a6-8a2cacc51205 admin admin] PATH_INFO: `/v3/projects/01328619d6b24d359c0352ef4119ea90/users/06231f178e0d494084915b9368fa3b9b/roles/b98dfb3b9fee4644af2887a3ac3fd2e6` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:56:35.977741 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-786f1fbb-da73-4b11-a1a6-8a2cacc51205 admin admin] RBAC: Authorizing `identity:create_grant(project_id=01328619d6b24d359c0352ef4119ea90, user_id=06231f178e0d494084915b9368fa3b9b, role_id=b98dfb3b9fee4644af2887a3ac3fd2e6)` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 10:56:35.981087 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-786f1fbb-da73-4b11-a1a6-8a2cacc51205 admin admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 10:56:35.995723 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1080/2189] 10.222.0.25 () {60 vars in 1483 bytes} [Sun Aug 7 10:56:35 2022] PUT /identity/v3/projects/01328619d6b24d359c0352ef4119ea90/users/06231f178e0d494084915b9368fa3b9b/roles/b98dfb3b9fee4644af2887a3ac3fd2e6 => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 07 10:56:36.001931 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27bd02c3-ba95-4178-afc9-ae9da45c2413 None None] REQUEST_METHOD: `POST` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:56:36.002211 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27bd02c3-ba95-4178-afc9-ae9da45c2413 None None] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:56:36.002672 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27bd02c3-ba95-4178-afc9-ae9da45c2413 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:56:36.022410 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.auth.core [None req-27bd02c3-ba95-4178-afc9-ae9da45c2413 None None] MFA Rules not processed for user `bac4e6b76fe64217ac8fa44e71cffe6a`. Rule list: `[]` (Enabled: `True`). {{(pid=80144) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 10:56:36.023637 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.fernet_utils [None req-27bd02c3-ba95-4178-afc9-ae9da45c2413 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80144) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 10:56:36.108128 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 1110/2190] 10.222.0.25 () {58 vars in 958 bytes} [Sun Aug 7 10:56:36 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 07 10:56:36.123190 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-786f1fbb-da73-4b11-a1a6-8a2cacc51205 admin admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 10:56:36.125963 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-786f1fbb-da73-4b11-a1a6-8a2cacc51205 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 10:56:36.182882 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 10:56:36.218592 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-786f1fbb-da73-4b11-a1a6-8a2cacc51205 admin admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 10:56:36.222718 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d95764b-2d23-4696-a58d-a8ad5a6ba7da service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be1283c9bf47424c9aa3ad27cab03027', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6de1f3cac02c4bcdb05dd2337176033d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 10:56:36.223455 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d95764b-2d23-4696-a58d-a8ad5a6ba7da service neutron] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:56:36.223749 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d95764b-2d23-4696-a58d-a8ad5a6ba7da service neutron] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:56:36.224017 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d95764b-2d23-4696-a58d-a8ad5a6ba7da service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:56:36.236903 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d95764b-2d23-4696-a58d-a8ad5a6ba7da service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 10:56:36.238347 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d95764b-2d23-4696-a58d-a8ad5a6ba7da service neutron] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 10:56:36.249451 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1081/2191] 10.222.0.25 () {60 vars in 1365 bytes} [Sun Aug 7 10:56:36 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 127 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 07 10:56:41.406419 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0a8134b-19d4-40d4-b1bf-b971758a2c28 None None] REQUEST_METHOD: `POST` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:56:41.413561 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0a8134b-19d4-40d4-b1bf-b971758a2c28 None None] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:56:41.413561 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0a8134b-19d4-40d4-b1bf-b971758a2c28 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:56:41.449708 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.auth.core [None req-b0a8134b-19d4-40d4-b1bf-b971758a2c28 None None] MFA Rules not processed for user `06231f178e0d494084915b9368fa3b9b`. Rule list: `[]` (Enabled: `True`). {{(pid=80144) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 10:56:41.451189 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.fernet_utils [None req-b0a8134b-19d4-40d4-b1bf-b971758a2c28 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80144) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 10:56:41.479324 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 10:56:41.577306 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 1111/2192] 10.222.0.25 () {58 vars in 958 bytes} [Sun Aug 7 10:56:41 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 07 10:56:41.592614 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d95764b-2d23-4696-a58d-a8ad5a6ba7da service neutron] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 10:56:41.594123 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-4d95764b-2d23-4696-a58d-a8ad5a6ba7da service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 10:56:41.679750 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d95764b-2d23-4696-a58d-a8ad5a6ba7da service neutron] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 10:56:41.688055 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c01c642-7962-4c86-ba1b-f700ea24efd9 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31d16d1b96104b4eb690398432b9829e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6de1f3cac02c4bcdb05dd2337176033d', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'reader', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 10:56:41.688700 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c01c642-7962-4c86-ba1b-f700ea24efd9 service nova] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:56:41.688990 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c01c642-7962-4c86-ba1b-f700ea24efd9 service nova] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:56:41.689268 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c01c642-7962-4c86-ba1b-f700ea24efd9 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:56:41.698001 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c01c642-7962-4c86-ba1b-f700ea24efd9 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 10:56:41.700251 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c01c642-7962-4c86-ba1b-f700ea24efd9 service nova] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 10:56:41.711000 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1082/2193] 10.222.0.25 () {60 vars in 1365 bytes} [Sun Aug 7 10:56:41 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 118 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 07 10:56:42.157503 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0a8134b-19d4-40d4-b1bf-b971758a2c28 None None] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 10:56:42.158947 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.fernet_utils [None req-b0a8134b-19d4-40d4-b1bf-b971758a2c28 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80144) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 10:56:42.176610 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0a8134b-19d4-40d4-b1bf-b971758a2c28 None None] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 10:56:42.188739 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-473b416f-8743-48ad-8489-9855809ef264 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5ac47e3b09f43d19a23a0f457fe9e23', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 10:56:42.192846 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-473b416f-8743-48ad-8489-9855809ef264 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:56:42.193353 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-473b416f-8743-48ad-8489-9855809ef264 admin admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:56:42.193711 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-473b416f-8743-48ad-8489-9855809ef264 admin admin] PATH_INFO: `/v3/users/9e1e727405d74f82bf192e6154241811` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:56:42.195699 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-473b416f-8743-48ad-8489-9855809ef264 admin admin] RBAC: Authorizing `identity:delete_user(user_id=9e1e727405d74f82bf192e6154241811)` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 10:56:42.197670 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-473b416f-8743-48ad-8489-9855809ef264 admin admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 10:56:42.262338 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.notifications [None req-473b416f-8743-48ad-8489-9855809ef264 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9e1e727405d74f82bf192e6154241811'} {{(pid=80144) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 10:56:42.281872 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.notifications [None req-473b416f-8743-48ad-8489-9855809ef264 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9e1e727405d74f82bf192e6154241811'} {{(pid=80144) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 10:56:42.291738 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.notifications [None req-473b416f-8743-48ad-8489-9855809ef264 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9e1e727405d74f82bf192e6154241811'} {{(pid=80144) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 10:56:42.292390 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.notifications [None req-473b416f-8743-48ad-8489-9855809ef264 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9e1e727405d74f82bf192e6154241811'} {{(pid=80144) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 10:56:42.293103 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.notifications [None req-473b416f-8743-48ad-8489-9855809ef264 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9e1e727405d74f82bf192e6154241811'} {{(pid=80144) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 10:56:42.319181 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 1112/2194] 10.222.0.25 () {58 vars in 1224 bytes} [Sun Aug 7 10:56:42 2022] DELETE /identity/v3/users/9e1e727405d74f82bf192e6154241811 => generated 0 bytes in 160 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 07 10:56:42.320948 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6a9fc4c-ff1c-4a38-a775-ffb13c9f8eeb None None] REQUEST_METHOD: `POST` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:56:42.321354 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6a9fc4c-ff1c-4a38-a775-ffb13c9f8eeb None None] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:56:42.321729 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6a9fc4c-ff1c-4a38-a775-ffb13c9f8eeb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:56:42.344095 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.auth.core [None req-f6a9fc4c-ff1c-4a38-a775-ffb13c9f8eeb None None] MFA Rules not processed for user `bac4e6b76fe64217ac8fa44e71cffe6a`. Rule list: `[]` (Enabled: `True`). {{(pid=80143) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 10:56:42.345450 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-f6a9fc4c-ff1c-4a38-a775-ffb13c9f8eeb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 10:56:42.442552 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1083/2195] 10.222.0.25 () {58 vars in 958 bytes} [Sun Aug 7 10:56:42 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 07 10:56:42.453860 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-473b416f-8743-48ad-8489-9855809ef264 admin admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 10:56:42.455482 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.fernet_utils [None req-473b416f-8743-48ad-8489-9855809ef264 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80144) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 10:56:42.536752 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-473b416f-8743-48ad-8489-9855809ef264 admin admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 10:56:42.541536 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-278d469b-20d8-4567-b748-b9443f920312 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be1283c9bf47424c9aa3ad27cab03027', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6de1f3cac02c4bcdb05dd2337176033d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 10:56:42.542154 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-278d469b-20d8-4567-b748-b9443f920312 service neutron] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:56:42.542595 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-278d469b-20d8-4567-b748-b9443f920312 service neutron] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:56:42.543993 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-278d469b-20d8-4567-b748-b9443f920312 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:56:42.565537 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-278d469b-20d8-4567-b748-b9443f920312 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 10:56:42.567393 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-278d469b-20d8-4567-b748-b9443f920312 service neutron] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 10:56:42.609032 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 1113/2196] 10.222.0.25 () {60 vars in 1365 bytes} [Sun Aug 7 10:56:42 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 156 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 07 10:56:43.271402 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6a9fc4c-ff1c-4a38-a775-ffb13c9f8eeb None None] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 10:56:43.272877 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-f6a9fc4c-ff1c-4a38-a775-ffb13c9f8eeb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 10:56:43.290060 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6a9fc4c-ff1c-4a38-a775-ffb13c9f8eeb None None] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 10:56:43.302804 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8969807-aceb-4c08-866b-15593c006b08 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5ac47e3b09f43d19a23a0f457fe9e23', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 10:56:43.303768 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8969807-aceb-4c08-866b-15593c006b08 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:56:43.304145 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8969807-aceb-4c08-866b-15593c006b08 admin admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:56:43.304577 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8969807-aceb-4c08-866b-15593c006b08 admin admin] PATH_INFO: `/v3/projects/384bd09b1fcb4b6d838381ab958f1de3` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:56:43.305794 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8969807-aceb-4c08-866b-15593c006b08 admin admin] RBAC: Authorizing `identity:delete_project(project_id=384bd09b1fcb4b6d838381ab958f1de3)` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 10:56:43.307881 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8969807-aceb-4c08-866b-15593c006b08 admin admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 10:56:43.366003 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-e8969807-aceb-4c08-866b-15593c006b08 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '384bd09b1fcb4b6d838381ab958f1de3', 'request_id': 'req-e8969807-aceb-4c08-866b-15593c006b08'} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 10:56:43.370167 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=95,keystone:DELETE=5 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 10:56:43.373927 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-e8969807-aceb-4c08-866b-15593c006b08 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '384bd09b1fcb4b6d838381ab958f1de3', 'request_id': 'req-e8969807-aceb-4c08-866b-15593c006b08'} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 10:56:43.382311 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-e8969807-aceb-4c08-866b-15593c006b08 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '384bd09b1fcb4b6d838381ab958f1de3', 'request_id': 'req-e8969807-aceb-4c08-866b-15593c006b08'} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 10:56:43.398916 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-e8969807-aceb-4c08-866b-15593c006b08 admin admin] The token cache is being invalidate because project 384bd09b1fcb4b6d838381ab958f1de3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80143) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 07 10:56:43.399275 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-e8969807-aceb-4c08-866b-15593c006b08 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 10:56:43.400108 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-e8969807-aceb-4c08-866b-15593c006b08 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 10:56:43.401867 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1084/2197] 10.222.0.25 () {58 vars in 1233 bytes} [Sun Aug 7 10:56:43 2022] DELETE /identity/v3/projects/384bd09b1fcb4b6d838381ab958f1de3 => generated 0 bytes in 131 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 07 10:56:43.407188 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d04150ee-3e69-40ca-ad45-d2b2a8f6ceea None None] REQUEST_METHOD: `POST` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:56:43.407842 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d04150ee-3e69-40ca-ad45-d2b2a8f6ceea None None] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:56:43.408450 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d04150ee-3e69-40ca-ad45-d2b2a8f6ceea None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:56:43.437708 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.auth.core [None req-d04150ee-3e69-40ca-ad45-d2b2a8f6ceea None None] MFA Rules not processed for user `bac4e6b76fe64217ac8fa44e71cffe6a`. Rule list: `[]` (Enabled: `True`). {{(pid=80144) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 10:56:43.439155 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.fernet_utils [None req-d04150ee-3e69-40ca-ad45-d2b2a8f6ceea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80144) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 10:56:43.486002 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 10:56:43.534070 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 1114/2198] 10.222.0.25 () {58 vars in 958 bytes} [Sun Aug 7 10:56:43 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 07 10:56:43.543941 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27b4ed54-88d5-41a7-8f48-3a64b3aaf7bc None None] REQUEST_METHOD: `POST` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:56:43.546469 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27b4ed54-88d5-41a7-8f48-3a64b3aaf7bc None None] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:56:43.547406 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27b4ed54-88d5-41a7-8f48-3a64b3aaf7bc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:56:43.576143 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.auth.core [None req-27b4ed54-88d5-41a7-8f48-3a64b3aaf7bc None None] MFA Rules not processed for user `bac4e6b76fe64217ac8fa44e71cffe6a`. Rule list: `[]` (Enabled: `True`). {{(pid=80143) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 10:56:43.577654 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-27b4ed54-88d5-41a7-8f48-3a64b3aaf7bc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 10:56:43.593063 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1085/2199] 10.222.0.25 () {58 vars in 958 bytes} [Sun Aug 7 10:56:43 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 49 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 07 10:56:43.594028 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d04150ee-3e69-40ca-ad45-d2b2a8f6ceea None None] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 10:56:43.606675 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d04150ee-3e69-40ca-ad45-d2b2a8f6ceea None None] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 10:56:43.618898 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03b7a580-1c9d-488e-b2d0-39729d08db28 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5ac47e3b09f43d19a23a0f457fe9e23', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 10:56:43.619777 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03b7a580-1c9d-488e-b2d0-39729d08db28 admin admin] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:56:43.620207 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03b7a580-1c9d-488e-b2d0-39729d08db28 admin admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:56:43.620650 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03b7a580-1c9d-488e-b2d0-39729d08db28 admin admin] PATH_INFO: `/v3/domains` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:56:43.622061 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03b7a580-1c9d-488e-b2d0-39729d08db28 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80144) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 10:56:43.622661 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03b7a580-1c9d-488e-b2d0-39729d08db28 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 10:56:43.624800 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03b7a580-1c9d-488e-b2d0-39729d08db28 admin admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 10:56:43.647641 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 1115/2200] 10.222.0.25 () {58 vars in 1153 bytes} [Sun Aug 7 10:56:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 51 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 10:56:43.648809 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27b4ed54-88d5-41a7-8f48-3a64b3aaf7bc None None] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 10:56:43.658977 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27b4ed54-88d5-41a7-8f48-3a64b3aaf7bc None None] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 10:56:43.663786 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-860c6db5-f31b-49a0-b170-5e861fd7bebc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5ac47e3b09f43d19a23a0f457fe9e23', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 10:56:43.664390 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-860c6db5-f31b-49a0-b170-5e861fd7bebc admin admin] REQUEST_METHOD: `POST` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:56:43.664667 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-860c6db5-f31b-49a0-b170-5e861fd7bebc admin admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:56:43.664939 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-860c6db5-f31b-49a0-b170-5e861fd7bebc admin admin] PATH_INFO: `/v3/projects` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:56:43.665583 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-860c6db5-f31b-49a0-b170-5e861fd7bebc admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 10:56:43.669095 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-860c6db5-f31b-49a0-b170-5e861fd7bebc admin admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 10:56:43.708772 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1086/2201] 10.222.0.25 () {60 vars in 1153 bytes} [Sun Aug 7 10:56:43 2022] POST /identity/v3/projects => generated 365 bytes in 57 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 07 10:56:43.709902 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03b7a580-1c9d-488e-b2d0-39729d08db28 admin admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 10:56:43.722490 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03b7a580-1c9d-488e-b2d0-39729d08db28 admin admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 10:56:43.793630 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d80f4d3-a6d4-4fb7-979a-26ac5b811559 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5ac47e3b09f43d19a23a0f457fe9e23', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 10:56:43.794366 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d80f4d3-a6d4-4fb7-979a-26ac5b811559 admin admin] REQUEST_METHOD: `POST` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:56:43.795049 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d80f4d3-a6d4-4fb7-979a-26ac5b811559 admin admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:56:43.795049 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d80f4d3-a6d4-4fb7-979a-26ac5b811559 admin admin] PATH_INFO: `/v3/users` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:56:43.795811 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d80f4d3-a6d4-4fb7-979a-26ac5b811559 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 10:56:43.797754 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d80f4d3-a6d4-4fb7-979a-26ac5b811559 admin admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 10:56:43.824121 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 1116/2202] 10.222.0.25 () {60 vars in 1144 bytes} [Sun Aug 7 10:56:43 2022] POST /identity/v3/users => generated 283 bytes in 115 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 07 10:56:43.828704 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-860c6db5-f31b-49a0-b170-5e861fd7bebc admin admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 10:56:43.842033 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-860c6db5-f31b-49a0-b170-5e861fd7bebc admin admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 10:56:43.853948 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1dd4cde-763d-4ef1-b550-b29cee6de3d6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5ac47e3b09f43d19a23a0f457fe9e23', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 10:56:43.853948 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1dd4cde-763d-4ef1-b550-b29cee6de3d6 admin admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:56:43.853948 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1dd4cde-763d-4ef1-b550-b29cee6de3d6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:56:43.853948 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1dd4cde-763d-4ef1-b550-b29cee6de3d6 admin admin] PATH_INFO: `/v3/roles` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:56:43.854320 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1dd4cde-763d-4ef1-b550-b29cee6de3d6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 10:56:43.854320 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1dd4cde-763d-4ef1-b550-b29cee6de3d6 admin admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 10:56:43.857032 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18b5261d-4b57-4c6b-a60d-d66c7ba503ab None None] REQUEST_METHOD: `POST` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:56:43.857318 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18b5261d-4b57-4c6b-a60d-d66c7ba503ab None None] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:56:43.857724 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18b5261d-4b57-4c6b-a60d-d66c7ba503ab None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:56:43.863236 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1087/2203] 10.222.0.25 () {58 vars in 1122 bytes} [Sun Aug 7 10:56:43 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 07 10:56:43.866278 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1dd4cde-763d-4ef1-b550-b29cee6de3d6 admin admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 10:56:43.879950 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1dd4cde-763d-4ef1-b550-b29cee6de3d6 admin admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 10:56:43.880179 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.auth.core [None req-18b5261d-4b57-4c6b-a60d-d66c7ba503ab None None] MFA Rules not processed for user `06231f178e0d494084915b9368fa3b9b`. Rule list: `[]` (Enabled: `True`). {{(pid=80144) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 10:56:43.881176 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.fernet_utils [None req-18b5261d-4b57-4c6b-a60d-d66c7ba503ab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80144) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 10:56:43.884947 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7418281-178d-448c-adec-17358bb71df7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5ac47e3b09f43d19a23a0f457fe9e23', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 10:56:43.885634 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7418281-178d-448c-adec-17358bb71df7 admin admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:56:43.886638 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7418281-178d-448c-adec-17358bb71df7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:56:43.886638 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7418281-178d-448c-adec-17358bb71df7 admin admin] PATH_INFO: `/v3/roles` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:56:43.891117 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7418281-178d-448c-adec-17358bb71df7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 10:56:43.894489 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7418281-178d-448c-adec-17358bb71df7 admin admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 10:56:43.924401 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1088/2204] 10.222.0.25 () {58 vars in 1122 bytes} [Sun Aug 7 10:56:43 2022] GET /identity/v3/roles => generated 1420 bytes in 58 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 07 10:56:43.925404 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7418281-178d-448c-adec-17358bb71df7 admin admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 10:56:43.936052 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7418281-178d-448c-adec-17358bb71df7 admin admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 10:56:43.941027 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50aff6a1-5307-41cb-9626-2f3d1a6de9c7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5ac47e3b09f43d19a23a0f457fe9e23', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 10:56:43.941708 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50aff6a1-5307-41cb-9626-2f3d1a6de9c7 admin admin] REQUEST_METHOD: `PUT` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:56:43.941992 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50aff6a1-5307-41cb-9626-2f3d1a6de9c7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:56:43.942584 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50aff6a1-5307-41cb-9626-2f3d1a6de9c7 admin admin] PATH_INFO: `/v3/projects/bfa4e257f9ba4b93b55e28effac87706/users/167a1e2adda64bdb8634edf4556405ba/roles/b98dfb3b9fee4644af2887a3ac3fd2e6` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:56:43.954889 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 1117/2205] 10.222.0.25 () {58 vars in 958 bytes} [Sun Aug 7 10:56:43 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 07 10:56:43.958362 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50aff6a1-5307-41cb-9626-2f3d1a6de9c7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=bfa4e257f9ba4b93b55e28effac87706, user_id=167a1e2adda64bdb8634edf4556405ba, role_id=b98dfb3b9fee4644af2887a3ac3fd2e6)` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 10:56:43.961592 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50aff6a1-5307-41cb-9626-2f3d1a6de9c7 admin admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 10:56:43.978474 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1089/2206] 10.222.0.25 () {60 vars in 1483 bytes} [Sun Aug 7 10:56:43 2022] PUT /identity/v3/projects/bfa4e257f9ba4b93b55e28effac87706/users/167a1e2adda64bdb8634edf4556405ba/roles/b98dfb3b9fee4644af2887a3ac3fd2e6 => generated 0 bytes in 54 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 07 10:56:43.991049 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20f0b4ff-0f80-48b0-b667-cb813b1bbcc2 None None] REQUEST_METHOD: `POST` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:56:43.991338 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20f0b4ff-0f80-48b0-b667-cb813b1bbcc2 None None] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:56:43.991609 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20f0b4ff-0f80-48b0-b667-cb813b1bbcc2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:56:44.032007 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.auth.core [None req-20f0b4ff-0f80-48b0-b667-cb813b1bbcc2 None None] MFA Rules not processed for user `167a1e2adda64bdb8634edf4556405ba`. Rule list: `[]` (Enabled: `True`). {{(pid=80144) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 10:56:44.032981 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.fernet_utils [None req-20f0b4ff-0f80-48b0-b667-cb813b1bbcc2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80144) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 10:56:44.089436 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 10:56:44.112965 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 1118/2207] 10.222.0.25 () {58 vars in 958 bytes} [Sun Aug 7 10:56:43 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 07 10:56:44.125611 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50aff6a1-5307-41cb-9626-2f3d1a6de9c7 admin admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 10:56:44.127463 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-50aff6a1-5307-41cb-9626-2f3d1a6de9c7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 10:56:44.241671 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50aff6a1-5307-41cb-9626-2f3d1a6de9c7 admin admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 10:56:44.252590 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b16cbb3a-387b-485a-a649-8b656a8c74e3 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88dbd6dfd1164994a3e16da553561910', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6de1f3cac02c4bcdb05dd2337176033d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 10:56:44.253128 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b16cbb3a-387b-485a-a649-8b656a8c74e3 service cinder] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:56:44.253413 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b16cbb3a-387b-485a-a649-8b656a8c74e3 service cinder] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:56:44.253687 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b16cbb3a-387b-485a-a649-8b656a8c74e3 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:56:44.264015 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b16cbb3a-387b-485a-a649-8b656a8c74e3 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 10:56:44.265298 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b16cbb3a-387b-485a-a649-8b656a8c74e3 service cinder] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 10:56:44.275621 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1090/2208] 10.222.0.25 () {60 vars in 1365 bytes} [Sun Aug 7 10:56:44 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 150 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 07 10:56:46.777291 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20f0b4ff-0f80-48b0-b667-cb813b1bbcc2 None None] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 10:56:46.779968 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.fernet_utils [None req-20f0b4ff-0f80-48b0-b667-cb813b1bbcc2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80144) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 10:56:46.868091 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20f0b4ff-0f80-48b0-b667-cb813b1bbcc2 None None] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 10:56:46.893216 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c22078f-c67a-4cd0-80ea-d50613e29558 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be1283c9bf47424c9aa3ad27cab03027', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6de1f3cac02c4bcdb05dd2337176033d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 10:56:46.893833 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c22078f-c67a-4cd0-80ea-d50613e29558 service neutron] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:56:46.894570 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c22078f-c67a-4cd0-80ea-d50613e29558 service neutron] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:56:46.894570 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c22078f-c67a-4cd0-80ea-d50613e29558 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:56:46.913081 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.fernet_utils [None req-9c22078f-c67a-4cd0-80ea-d50613e29558 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80144) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 10:56:47.049340 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c22078f-c67a-4cd0-80ea-d50613e29558 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 10:56:47.051559 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c22078f-c67a-4cd0-80ea-d50613e29558 service neutron] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 10:56:47.069190 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 1119/2209] 10.222.0.25 () {60 vars in 1365 bytes} [Sun Aug 7 10:56:46 2022] GET /identity/v3/auth/tokens => generated 3329 bytes in 292 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 07 10:56:52.593230 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30667d40-adb0-4c45-a6e8-0260c9e3456c None None] REQUEST_METHOD: `POST` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:56:52.593700 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30667d40-adb0-4c45-a6e8-0260c9e3456c None None] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:56:52.593779 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30667d40-adb0-4c45-a6e8-0260c9e3456c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:56:52.614483 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.auth.core [None req-30667d40-adb0-4c45-a6e8-0260c9e3456c None None] MFA Rules not processed for user `de1d6c24b1a14d10a20d82a4c65793d8`. Rule list: `[]` (Enabled: `True`). {{(pid=80143) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 10:56:52.615667 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-30667d40-adb0-4c45-a6e8-0260c9e3456c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 10:56:52.698331 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1091/2210] 10.222.0.25 () {58 vars in 1009 bytes} [Sun Aug 7 10:56:52 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 07 10:56:52.757653 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c22078f-c67a-4cd0-80ea-d50613e29558 service neutron] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 10:56:52.759307 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.fernet_utils [None req-9c22078f-c67a-4cd0-80ea-d50613e29558 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80144) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 10:56:52.858282 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c22078f-c67a-4cd0-80ea-d50613e29558 service neutron] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 10:56:52.866609 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da559414-8879-49d4-8cb5-a58e1ccc1f2f service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69490ea2a9a340bf91aca38bf5655b45', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6de1f3cac02c4bcdb05dd2337176033d', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 10:56:52.868065 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da559414-8879-49d4-8cb5-a58e1ccc1f2f service swift] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:56:52.868065 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da559414-8879-49d4-8cb5-a58e1ccc1f2f service swift] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:56:52.868065 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da559414-8879-49d4-8cb5-a58e1ccc1f2f service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:56:52.900296 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da559414-8879-49d4-8cb5-a58e1ccc1f2f service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 10:56:52.903058 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da559414-8879-49d4-8cb5-a58e1ccc1f2f service swift] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 10:56:52.914266 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 1120/2211] 10.222.0.25 () {60 vars in 1384 bytes} [Sun Aug 7 10:56:52 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 157 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 07 10:56:53.460117 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a1017fb-579b-440c-bb17-7c37cbc9fdd7 None None] REQUEST_METHOD: `POST` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:56:53.460502 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a1017fb-579b-440c-bb17-7c37cbc9fdd7 None None] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:56:53.460880 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a1017fb-579b-440c-bb17-7c37cbc9fdd7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:56:53.481636 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.auth.core [None req-0a1017fb-579b-440c-bb17-7c37cbc9fdd7 None None] MFA Rules not processed for user `de1d6c24b1a14d10a20d82a4c65793d8`. Rule list: `[]` (Enabled: `True`). {{(pid=80143) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 10:56:53.483233 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-0a1017fb-579b-440c-bb17-7c37cbc9fdd7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 10:56:53.490738 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1092/2212] 10.222.0.25 () {58 vars in 967 bytes} [Sun Aug 7 10:56:53 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 07 10:56:53.504662 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da559414-8879-49d4-8cb5-a58e1ccc1f2f service swift] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 10:56:53.515656 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da559414-8879-49d4-8cb5-a58e1ccc1f2f service swift] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 10:56:53.521314 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d65d8283-56b0-4656-a847-2b93998c72cb service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69490ea2a9a340bf91aca38bf5655b45', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6de1f3cac02c4bcdb05dd2337176033d', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 10:56:53.522129 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d65d8283-56b0-4656-a847-2b93998c72cb service swift] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:56:53.522458 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d65d8283-56b0-4656-a847-2b93998c72cb service swift] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:56:53.522740 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d65d8283-56b0-4656-a847-2b93998c72cb service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:56:53.532020 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d65d8283-56b0-4656-a847-2b93998c72cb service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 10:56:53.533602 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d65d8283-56b0-4656-a847-2b93998c72cb service swift] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 10:56:53.541883 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 1121/2213] 10.222.0.25 () {60 vars in 1384 bytes} [Sun Aug 7 10:56:53 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 07 10:56:53.782428 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10e2217d-6cfa-46c0-b798-cf065a43b9c9 None None] REQUEST_METHOD: `POST` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:56:53.782959 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10e2217d-6cfa-46c0-b798-cf065a43b9c9 None None] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:56:53.783242 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10e2217d-6cfa-46c0-b798-cf065a43b9c9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:56:53.803170 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.auth.core [None req-10e2217d-6cfa-46c0-b798-cf065a43b9c9 None None] MFA Rules not processed for user `de1d6c24b1a14d10a20d82a4c65793d8`. Rule list: `[]` (Enabled: `True`). {{(pid=80143) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 10:56:53.804460 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-10e2217d-6cfa-46c0-b798-cf065a43b9c9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 10:56:53.812071 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1093/2214] 10.222.0.25 () {58 vars in 1009 bytes} [Sun Aug 7 10:56:53 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 07 10:56:53.825293 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d65d8283-56b0-4656-a847-2b93998c72cb service swift] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 10:56:53.835677 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d65d8283-56b0-4656-a847-2b93998c72cb service swift] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 10:56:53.841820 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-607b040d-b878-4bc6-8912-dce4a3701858 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69490ea2a9a340bf91aca38bf5655b45', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6de1f3cac02c4bcdb05dd2337176033d', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 10:56:53.842508 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-607b040d-b878-4bc6-8912-dce4a3701858 service swift] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:56:53.842875 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-607b040d-b878-4bc6-8912-dce4a3701858 service swift] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:56:53.843244 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-607b040d-b878-4bc6-8912-dce4a3701858 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:56:53.852225 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-607b040d-b878-4bc6-8912-dce4a3701858 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 10:56:53.854844 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-607b040d-b878-4bc6-8912-dce4a3701858 service swift] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 10:56:53.864251 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 1122/2215] 10.222.0.25 () {60 vars in 1384 bytes} [Sun Aug 7 10:56:53 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 07 10:56:54.048739 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd7ed8a8-1c0b-45f6-89ec-242678c88a6f None None] REQUEST_METHOD: `POST` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:56:54.049664 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd7ed8a8-1c0b-45f6-89ec-242678c88a6f None None] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:56:54.049972 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd7ed8a8-1c0b-45f6-89ec-242678c88a6f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:56:54.070720 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:DELETE=2,keystone:SELECT=95,keystone:INSERT=3 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 10:56:54.100068 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.auth.core [None req-fd7ed8a8-1c0b-45f6-89ec-242678c88a6f None None] MFA Rules not processed for user `c3a12c3144844d32812e010550bb0921`. Rule list: `[]` (Enabled: `True`). {{(pid=80143) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 10:56:54.101555 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-fd7ed8a8-1c0b-45f6-89ec-242678c88a6f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 10:56:54.209091 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1094/2216] 10.222.0.25 () {58 vars in 958 bytes} [Sun Aug 7 10:56:54 2022] POST /identity/v3/auth/tokens => generated 3270 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 07 10:56:54.335617 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-412baf4a-5ac5-4eaa-bd4c-34c4e7233d81 None None] REQUEST_METHOD: `POST` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:56:54.335910 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-412baf4a-5ac5-4eaa-bd4c-34c4e7233d81 None None] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:56:54.336198 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-412baf4a-5ac5-4eaa-bd4c-34c4e7233d81 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:56:54.344274 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=100 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 10:56:54.357013 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.auth.core [None req-412baf4a-5ac5-4eaa-bd4c-34c4e7233d81 None None] MFA Rules not processed for user `de1d6c24b1a14d10a20d82a4c65793d8`. Rule list: `[]` (Enabled: `True`). {{(pid=80144) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 10:56:54.358470 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.fernet_utils [None req-412baf4a-5ac5-4eaa-bd4c-34c4e7233d81 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80144) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 10:56:54.365859 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 1123/2217] 10.222.0.25 () {58 vars in 967 bytes} [Sun Aug 7 10:56:54 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 07 10:56:54.378836 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd7ed8a8-1c0b-45f6-89ec-242678c88a6f None None] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 10:56:54.391730 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd7ed8a8-1c0b-45f6-89ec-242678c88a6f None None] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 10:56:54.397076 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2343e186-d5fa-425f-bfa9-1cb67bc5e3a3 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69490ea2a9a340bf91aca38bf5655b45', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6de1f3cac02c4bcdb05dd2337176033d', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 10:56:54.397685 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2343e186-d5fa-425f-bfa9-1cb67bc5e3a3 service swift] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:56:54.397966 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2343e186-d5fa-425f-bfa9-1cb67bc5e3a3 service swift] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:56:54.398238 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2343e186-d5fa-425f-bfa9-1cb67bc5e3a3 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:56:54.406655 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2343e186-d5fa-425f-bfa9-1cb67bc5e3a3 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 10:56:54.408296 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2343e186-d5fa-425f-bfa9-1cb67bc5e3a3 service swift] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 10:56:54.416889 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1095/2218] 10.222.0.25 () {60 vars in 1384 bytes} [Sun Aug 7 10:56:54 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 07 10:57:01.845668 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-412baf4a-5ac5-4eaa-bd4c-34c4e7233d81 None None] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 10:57:01.847913 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.fernet_utils [None req-412baf4a-5ac5-4eaa-bd4c-34c4e7233d81 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80144) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 10:57:01.944352 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-412baf4a-5ac5-4eaa-bd4c-34c4e7233d81 None None] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 10:57:01.950892 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-802f5f7f-610e-42bd-8101-1f933217a173 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5ac47e3b09f43d19a23a0f457fe9e23', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 10:57:01.951727 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-802f5f7f-610e-42bd-8101-1f933217a173 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:57:01.952203 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-802f5f7f-610e-42bd-8101-1f933217a173 admin admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:57:01.952579 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-802f5f7f-610e-42bd-8101-1f933217a173 admin admin] PATH_INFO: `/v3/users/aeb2182f9b614007b61823075835fe33` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:57:01.953847 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-802f5f7f-610e-42bd-8101-1f933217a173 admin admin] RBAC: Authorizing `identity:delete_user(user_id=aeb2182f9b614007b61823075835fe33)` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 10:57:01.955916 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-802f5f7f-610e-42bd-8101-1f933217a173 admin admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 10:57:02.051147 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.notifications [None req-802f5f7f-610e-42bd-8101-1f933217a173 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'aeb2182f9b614007b61823075835fe33'} {{(pid=80144) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 10:57:02.068419 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.notifications [None req-802f5f7f-610e-42bd-8101-1f933217a173 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'aeb2182f9b614007b61823075835fe33'} {{(pid=80144) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 10:57:02.078066 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.notifications [None req-802f5f7f-610e-42bd-8101-1f933217a173 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'aeb2182f9b614007b61823075835fe33'} {{(pid=80144) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 10:57:02.079205 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.notifications [None req-802f5f7f-610e-42bd-8101-1f933217a173 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'aeb2182f9b614007b61823075835fe33'} {{(pid=80144) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 10:57:02.079995 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.notifications [None req-802f5f7f-610e-42bd-8101-1f933217a173 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'aeb2182f9b614007b61823075835fe33'} {{(pid=80144) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 10:57:02.101277 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 1124/2219] 10.222.0.25 () {58 vars in 1224 bytes} [Sun Aug 7 10:57:01 2022] DELETE /identity/v3/users/aeb2182f9b614007b61823075835fe33 => generated 0 bytes in 256 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 07 10:57:02.106636 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fb284b5-bf25-44bd-a43c-72c90b61b88e None None] REQUEST_METHOD: `POST` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:57:02.107160 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fb284b5-bf25-44bd-a43c-72c90b61b88e None None] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:57:02.107250 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fb284b5-bf25-44bd-a43c-72c90b61b88e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:57:02.126687 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.auth.core [None req-2fb284b5-bf25-44bd-a43c-72c90b61b88e None None] MFA Rules not processed for user `bac4e6b76fe64217ac8fa44e71cffe6a`. Rule list: `[]` (Enabled: `True`). {{(pid=80143) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 10:57:02.127912 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-2fb284b5-bf25-44bd-a43c-72c90b61b88e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 10:57:02.186897 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1096/2220] 10.222.0.25 () {58 vars in 958 bytes} [Sun Aug 7 10:57:02 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 07 10:57:02.196675 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-802f5f7f-610e-42bd-8101-1f933217a173 admin admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 10:57:02.198093 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.fernet_utils [None req-802f5f7f-610e-42bd-8101-1f933217a173 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80144) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 10:57:02.254881 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-802f5f7f-610e-42bd-8101-1f933217a173 admin admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 10:57:02.258949 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f750e2b-a602-4244-bd19-e0b1923cf391 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be1283c9bf47424c9aa3ad27cab03027', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6de1f3cac02c4bcdb05dd2337176033d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 10:57:02.259560 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f750e2b-a602-4244-bd19-e0b1923cf391 service neutron] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:57:02.259839 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f750e2b-a602-4244-bd19-e0b1923cf391 service neutron] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:57:02.260108 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f750e2b-a602-4244-bd19-e0b1923cf391 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:57:02.269132 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f750e2b-a602-4244-bd19-e0b1923cf391 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 10:57:02.270526 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f750e2b-a602-4244-bd19-e0b1923cf391 service neutron] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 10:57:02.279784 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 1125/2221] 10.222.0.25 () {60 vars in 1365 bytes} [Sun Aug 7 10:57:02 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 07 10:57:02.550129 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fb284b5-bf25-44bd-a43c-72c90b61b88e None None] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 10:57:02.552800 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-2fb284b5-bf25-44bd-a43c-72c90b61b88e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 10:57:02.589907 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fb284b5-bf25-44bd-a43c-72c90b61b88e None None] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 10:57:02.610719 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-452bd561-e066-4d0a-ac24-34a2f089a865 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5ac47e3b09f43d19a23a0f457fe9e23', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 10:57:02.613041 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-452bd561-e066-4d0a-ac24-34a2f089a865 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:57:02.613574 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-452bd561-e066-4d0a-ac24-34a2f089a865 admin admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:57:02.614302 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-452bd561-e066-4d0a-ac24-34a2f089a865 admin admin] PATH_INFO: `/v3/projects/2f306584e65048f98a4128cafc1b210c` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:57:02.617546 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-452bd561-e066-4d0a-ac24-34a2f089a865 admin admin] RBAC: Authorizing `identity:delete_project(project_id=2f306584e65048f98a4128cafc1b210c)` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 10:57:02.619844 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-452bd561-e066-4d0a-ac24-34a2f089a865 admin admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 10:57:02.689293 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-452bd561-e066-4d0a-ac24-34a2f089a865 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2f306584e65048f98a4128cafc1b210c', 'request_id': 'req-452bd561-e066-4d0a-ac24-34a2f089a865'} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 10:57:02.700526 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-452bd561-e066-4d0a-ac24-34a2f089a865 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2f306584e65048f98a4128cafc1b210c', 'request_id': 'req-452bd561-e066-4d0a-ac24-34a2f089a865'} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 10:57:02.708616 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-452bd561-e066-4d0a-ac24-34a2f089a865 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2f306584e65048f98a4128cafc1b210c', 'request_id': 'req-452bd561-e066-4d0a-ac24-34a2f089a865'} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 10:57:02.739386 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-452bd561-e066-4d0a-ac24-34a2f089a865 admin admin] The token cache is being invalidate because project 2f306584e65048f98a4128cafc1b210c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80143) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 07 10:57:02.739869 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-452bd561-e066-4d0a-ac24-34a2f089a865 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 10:57:02.740690 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-452bd561-e066-4d0a-ac24-34a2f089a865 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 10:57:02.742576 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1097/2222] 10.222.0.25 () {58 vars in 1233 bytes} [Sun Aug 7 10:57:02 2022] DELETE /identity/v3/projects/2f306584e65048f98a4128cafc1b210c => generated 0 bytes in 193 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 07 10:57:12.275787 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=81,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 10:57:12.731299 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=84,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 10:57:30.732292 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f750e2b-a602-4244-bd19-e0b1923cf391 service neutron] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 10:57:30.741101 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f750e2b-a602-4244-bd19-e0b1923cf391 service neutron] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 10:57:30.791145 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de5b4750-a514-4986-a092-368cef474e55 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be1283c9bf47424c9aa3ad27cab03027', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6de1f3cac02c4bcdb05dd2337176033d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 10:57:30.791960 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de5b4750-a514-4986-a092-368cef474e55 service neutron] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:57:30.792252 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de5b4750-a514-4986-a092-368cef474e55 service neutron] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:57:30.792522 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de5b4750-a514-4986-a092-368cef474e55 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:57:30.794260 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.fernet_utils [None req-de5b4750-a514-4986-a092-368cef474e55 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80144) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 10:57:30.855066 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de5b4750-a514-4986-a092-368cef474e55 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 10:57:30.856494 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de5b4750-a514-4986-a092-368cef474e55 service neutron] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 10:57:30.866259 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 1126/2223] 10.222.0.25 () {60 vars in 1365 bytes} [Sun Aug 7 10:57:30 2022] GET /identity/v3/auth/tokens => generated 3317 bytes in 135 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 07 10:57:38.724994 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-452bd561-e066-4d0a-ac24-34a2f089a865 admin admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 10:57:38.726853 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-452bd561-e066-4d0a-ac24-34a2f089a865 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 10:57:38.797088 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-452bd561-e066-4d0a-ac24-34a2f089a865 admin admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 10:57:38.801614 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37fe21d6-935e-4e25-8c3e-689d18f831b4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5ac47e3b09f43d19a23a0f457fe9e23', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 10:57:38.802222 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37fe21d6-935e-4e25-8c3e-689d18f831b4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:57:38.802543 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37fe21d6-935e-4e25-8c3e-689d18f831b4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:57:38.802815 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37fe21d6-935e-4e25-8c3e-689d18f831b4 admin admin] PATH_INFO: `/v3/users/167a1e2adda64bdb8634edf4556405ba` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:57:38.803955 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37fe21d6-935e-4e25-8c3e-689d18f831b4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=167a1e2adda64bdb8634edf4556405ba)` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 10:57:38.805842 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37fe21d6-935e-4e25-8c3e-689d18f831b4 admin admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 10:57:38.851004 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-37fe21d6-935e-4e25-8c3e-689d18f831b4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '167a1e2adda64bdb8634edf4556405ba'} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 10:57:38.858997 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-37fe21d6-935e-4e25-8c3e-689d18f831b4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '167a1e2adda64bdb8634edf4556405ba'} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 10:57:38.871488 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-37fe21d6-935e-4e25-8c3e-689d18f831b4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '167a1e2adda64bdb8634edf4556405ba'} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 10:57:38.872159 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-37fe21d6-935e-4e25-8c3e-689d18f831b4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '167a1e2adda64bdb8634edf4556405ba'} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 10:57:38.879462 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-37fe21d6-935e-4e25-8c3e-689d18f831b4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '167a1e2adda64bdb8634edf4556405ba'} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 10:57:38.885261 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1098/2224] 10.222.0.25 () {58 vars in 1224 bytes} [Sun Aug 7 10:57:38 2022] DELETE /identity/v3/users/167a1e2adda64bdb8634edf4556405ba => generated 0 bytes in 161 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 07 10:57:38.889815 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c702f15-5749-40ed-bbfb-d83553580805 None None] REQUEST_METHOD: `POST` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:57:38.890301 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c702f15-5749-40ed-bbfb-d83553580805 None None] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:57:38.890404 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c702f15-5749-40ed-bbfb-d83553580805 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:57:38.909729 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.auth.core [None req-0c702f15-5749-40ed-bbfb-d83553580805 None None] MFA Rules not processed for user `bac4e6b76fe64217ac8fa44e71cffe6a`. Rule list: `[]` (Enabled: `True`). {{(pid=80144) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 10:57:38.910945 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.fernet_utils [None req-0c702f15-5749-40ed-bbfb-d83553580805 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80144) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 10:57:38.968163 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 1127/2225] 10.222.0.25 () {58 vars in 958 bytes} [Sun Aug 7 10:57:38 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 07 10:57:38.977652 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37fe21d6-935e-4e25-8c3e-689d18f831b4 admin admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 10:57:38.979716 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-37fe21d6-935e-4e25-8c3e-689d18f831b4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 10:57:39.040433 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37fe21d6-935e-4e25-8c3e-689d18f831b4 admin admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 10:57:39.044313 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36371d11-b11b-4a00-a055-876f3bdf8cad service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be1283c9bf47424c9aa3ad27cab03027', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6de1f3cac02c4bcdb05dd2337176033d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 10:57:39.044924 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36371d11-b11b-4a00-a055-876f3bdf8cad service neutron] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:57:39.045212 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36371d11-b11b-4a00-a055-876f3bdf8cad service neutron] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:57:39.045485 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36371d11-b11b-4a00-a055-876f3bdf8cad service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:57:39.054349 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36371d11-b11b-4a00-a055-876f3bdf8cad service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 10:57:39.055816 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36371d11-b11b-4a00-a055-876f3bdf8cad service neutron] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 10:57:39.065675 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1099/2226] 10.222.0.25 () {60 vars in 1365 bytes} [Sun Aug 7 10:57:38 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 07 10:57:39.334981 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c702f15-5749-40ed-bbfb-d83553580805 None None] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 10:57:39.336634 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.fernet_utils [None req-0c702f15-5749-40ed-bbfb-d83553580805 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80144) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 10:57:39.348467 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c702f15-5749-40ed-bbfb-d83553580805 None None] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 10:57:39.353152 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d76b9af-d95a-4ee3-92d9-41f2ed7e35cb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5ac47e3b09f43d19a23a0f457fe9e23', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 10:57:39.353805 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d76b9af-d95a-4ee3-92d9-41f2ed7e35cb admin admin] REQUEST_METHOD: `DELETE` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:57:39.354089 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d76b9af-d95a-4ee3-92d9-41f2ed7e35cb admin admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:57:39.354364 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d76b9af-d95a-4ee3-92d9-41f2ed7e35cb admin admin] PATH_INFO: `/v3/projects/bfa4e257f9ba4b93b55e28effac87706` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:57:39.355524 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d76b9af-d95a-4ee3-92d9-41f2ed7e35cb admin admin] RBAC: Authorizing `identity:delete_project(project_id=bfa4e257f9ba4b93b55e28effac87706)` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 10:57:39.357438 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d76b9af-d95a-4ee3-92d9-41f2ed7e35cb admin admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 10:57:39.405677 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=97,keystone:DELETE=3 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 10:57:39.408801 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.notifications [None req-7d76b9af-d95a-4ee3-92d9-41f2ed7e35cb admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'bfa4e257f9ba4b93b55e28effac87706', 'request_id': 'req-7d76b9af-d95a-4ee3-92d9-41f2ed7e35cb'} {{(pid=80144) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 10:57:39.415387 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.notifications [None req-7d76b9af-d95a-4ee3-92d9-41f2ed7e35cb admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'bfa4e257f9ba4b93b55e28effac87706', 'request_id': 'req-7d76b9af-d95a-4ee3-92d9-41f2ed7e35cb'} {{(pid=80144) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 10:57:39.423605 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.notifications [None req-7d76b9af-d95a-4ee3-92d9-41f2ed7e35cb admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'bfa4e257f9ba4b93b55e28effac87706', 'request_id': 'req-7d76b9af-d95a-4ee3-92d9-41f2ed7e35cb'} {{(pid=80144) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 10:57:39.435901 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.notifications [None req-7d76b9af-d95a-4ee3-92d9-41f2ed7e35cb admin admin] The token cache is being invalidate because project bfa4e257f9ba4b93b55e28effac87706 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80144) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 07 10:57:39.436196 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.notifications [None req-7d76b9af-d95a-4ee3-92d9-41f2ed7e35cb admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80144) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 10:57:39.436833 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.notifications [None req-7d76b9af-d95a-4ee3-92d9-41f2ed7e35cb admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80144) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 10:57:39.438317 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 1128/2227] 10.222.0.25 () {58 vars in 1233 bytes} [Sun Aug 7 10:57:39 2022] DELETE /identity/v3/projects/bfa4e257f9ba4b93b55e28effac87706 => generated 0 bytes in 104 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 07 10:57:39.443207 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8629cb2e-2d70-4c51-89bf-a8a14bb0e15a None None] REQUEST_METHOD: `POST` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:57:39.443525 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8629cb2e-2d70-4c51-89bf-a8a14bb0e15a None None] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:57:39.443795 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8629cb2e-2d70-4c51-89bf-a8a14bb0e15a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:57:39.463293 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.auth.core [None req-8629cb2e-2d70-4c51-89bf-a8a14bb0e15a None None] MFA Rules not processed for user `bac4e6b76fe64217ac8fa44e71cffe6a`. Rule list: `[]` (Enabled: `True`). {{(pid=80143) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 10:57:39.464488 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-8629cb2e-2d70-4c51-89bf-a8a14bb0e15a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 10:57:39.481398 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 10:57:39.526020 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1100/2228] 10.222.0.25 () {58 vars in 958 bytes} [Sun Aug 7 10:57:39 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 07 10:57:39.533747 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90934340-21d1-4d2e-b24d-9eb6e5fdc9cd None None] REQUEST_METHOD: `POST` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:57:39.533990 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90934340-21d1-4d2e-b24d-9eb6e5fdc9cd None None] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:57:39.534280 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90934340-21d1-4d2e-b24d-9eb6e5fdc9cd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:57:39.554444 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.auth.core [None req-90934340-21d1-4d2e-b24d-9eb6e5fdc9cd None None] MFA Rules not processed for user `bac4e6b76fe64217ac8fa44e71cffe6a`. Rule list: `[]` (Enabled: `True`). {{(pid=80144) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 10:57:39.555676 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.fernet_utils [None req-90934340-21d1-4d2e-b24d-9eb6e5fdc9cd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80144) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 10:57:39.564068 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 1129/2229] 10.222.0.25 () {58 vars in 958 bytes} [Sun Aug 7 10:57:39 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 07 10:57:39.568402 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8629cb2e-2d70-4c51-89bf-a8a14bb0e15a None None] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 10:57:39.578256 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8629cb2e-2d70-4c51-89bf-a8a14bb0e15a None None] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 10:57:39.582773 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90444ba6-ab64-424e-8303-f58bb6ce3303 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5ac47e3b09f43d19a23a0f457fe9e23', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 10:57:39.583435 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90444ba6-ab64-424e-8303-f58bb6ce3303 admin admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:57:39.583717 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90444ba6-ab64-424e-8303-f58bb6ce3303 admin admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:57:39.583995 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90444ba6-ab64-424e-8303-f58bb6ce3303 admin admin] PATH_INFO: `/v3/domains` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:57:39.584641 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90444ba6-ab64-424e-8303-f58bb6ce3303 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80143) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 10:57:39.584976 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90444ba6-ab64-424e-8303-f58bb6ce3303 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 10:57:39.586563 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90444ba6-ab64-424e-8303-f58bb6ce3303 admin admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 10:57:39.598204 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1101/2230] 10.222.0.25 () {58 vars in 1153 bytes} [Sun Aug 7 10:57:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 10:57:39.602495 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90934340-21d1-4d2e-b24d-9eb6e5fdc9cd None None] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 10:57:39.612664 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90934340-21d1-4d2e-b24d-9eb6e5fdc9cd None None] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 10:57:39.617346 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08fbd7ca-4513-438c-bda4-b45c243d24eb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5ac47e3b09f43d19a23a0f457fe9e23', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 10:57:39.617939 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08fbd7ca-4513-438c-bda4-b45c243d24eb admin admin] REQUEST_METHOD: `POST` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:57:39.618223 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08fbd7ca-4513-438c-bda4-b45c243d24eb admin admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:57:39.618499 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08fbd7ca-4513-438c-bda4-b45c243d24eb admin admin] PATH_INFO: `/v3/projects` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:57:39.619783 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08fbd7ca-4513-438c-bda4-b45c243d24eb admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 10:57:39.621668 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08fbd7ca-4513-438c-bda4-b45c243d24eb admin admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 10:57:39.650142 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 1130/2231] 10.222.0.25 () {60 vars in 1153 bytes} [Sun Aug 7 10:57:39 2022] POST /identity/v3/projects => generated 383 bytes in 46 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 07 10:57:39.652804 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90444ba6-ab64-424e-8303-f58bb6ce3303 admin admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 10:57:39.662857 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90444ba6-ab64-424e-8303-f58bb6ce3303 admin admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 10:57:39.722642 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6871329-cafc-4d3d-8479-7ade5f4cbcd0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5ac47e3b09f43d19a23a0f457fe9e23', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 10:57:39.723282 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6871329-cafc-4d3d-8479-7ade5f4cbcd0 admin admin] REQUEST_METHOD: `POST` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:57:39.723570 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6871329-cafc-4d3d-8479-7ade5f4cbcd0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:57:39.723848 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6871329-cafc-4d3d-8479-7ade5f4cbcd0 admin admin] PATH_INFO: `/v3/users` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:57:39.724471 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6871329-cafc-4d3d-8479-7ade5f4cbcd0 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 10:57:39.726388 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6871329-cafc-4d3d-8479-7ade5f4cbcd0 admin admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 10:57:39.749386 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1102/2232] 10.222.0.25 () {60 vars in 1144 bytes} [Sun Aug 7 10:57:39 2022] POST /identity/v3/users => generated 292 bytes in 97 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 07 10:57:39.753601 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08fbd7ca-4513-438c-bda4-b45c243d24eb admin admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 10:57:39.764680 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08fbd7ca-4513-438c-bda4-b45c243d24eb admin admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 10:57:39.769260 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdcaf5a1-e406-43f4-8cf8-913dd9cef112 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5ac47e3b09f43d19a23a0f457fe9e23', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 10:57:39.769864 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdcaf5a1-e406-43f4-8cf8-913dd9cef112 admin admin] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:57:39.770154 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdcaf5a1-e406-43f4-8cf8-913dd9cef112 admin admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:57:39.770432 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdcaf5a1-e406-43f4-8cf8-913dd9cef112 admin admin] PATH_INFO: `/v3/roles` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:57:39.771133 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdcaf5a1-e406-43f4-8cf8-913dd9cef112 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 10:57:39.772706 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdcaf5a1-e406-43f4-8cf8-913dd9cef112 admin admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 10:57:39.783241 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 1131/2233] 10.222.0.25 () {58 vars in 1122 bytes} [Sun Aug 7 10:57:39 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 07 10:57:39.787064 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6871329-cafc-4d3d-8479-7ade5f4cbcd0 admin admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 10:57:39.797561 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6871329-cafc-4d3d-8479-7ade5f4cbcd0 admin admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 10:57:39.802227 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d632713e-d9a1-45e7-a1f2-6746e6c64afd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5ac47e3b09f43d19a23a0f457fe9e23', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 10:57:39.802868 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d632713e-d9a1-45e7-a1f2-6746e6c64afd admin admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:57:39.803254 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d632713e-d9a1-45e7-a1f2-6746e6c64afd admin admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:57:39.803604 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d632713e-d9a1-45e7-a1f2-6746e6c64afd admin admin] PATH_INFO: `/v3/roles` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:57:39.804280 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d632713e-d9a1-45e7-a1f2-6746e6c64afd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 10:57:39.805855 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d632713e-d9a1-45e7-a1f2-6746e6c64afd admin admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 10:57:39.816924 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1103/2234] 10.222.0.25 () {58 vars in 1122 bytes} [Sun Aug 7 10:57:39 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 07 10:57:39.821012 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdcaf5a1-e406-43f4-8cf8-913dd9cef112 admin admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 10:57:39.831516 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdcaf5a1-e406-43f4-8cf8-913dd9cef112 admin admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 10:57:39.836019 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89130b75-170f-4ba0-bc0b-7dce8eb735e8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5ac47e3b09f43d19a23a0f457fe9e23', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 10:57:39.836657 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89130b75-170f-4ba0-bc0b-7dce8eb735e8 admin admin] REQUEST_METHOD: `PUT` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:57:39.836936 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89130b75-170f-4ba0-bc0b-7dce8eb735e8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:57:39.837207 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89130b75-170f-4ba0-bc0b-7dce8eb735e8 admin admin] PATH_INFO: `/v3/projects/81ab48958e894bdbb5cd46049356a8a0/users/c4597d928ad24b6c81420f7158e5e98b/roles/b98dfb3b9fee4644af2887a3ac3fd2e6` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:57:39.851488 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89130b75-170f-4ba0-bc0b-7dce8eb735e8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=81ab48958e894bdbb5cd46049356a8a0, user_id=c4597d928ad24b6c81420f7158e5e98b, role_id=b98dfb3b9fee4644af2887a3ac3fd2e6)` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 10:57:39.854142 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89130b75-170f-4ba0-bc0b-7dce8eb735e8 admin admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 10:57:39.867010 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 1132/2235] 10.222.0.25 () {60 vars in 1483 bytes} [Sun Aug 7 10:57:39 2022] PUT /identity/v3/projects/81ab48958e894bdbb5cd46049356a8a0/users/c4597d928ad24b6c81420f7158e5e98b/roles/b98dfb3b9fee4644af2887a3ac3fd2e6 => generated 0 bytes in 46 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 07 10:57:39.872698 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe466c57-110e-4054-99a4-c1da625c8302 None None] REQUEST_METHOD: `POST` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:57:39.872987 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe466c57-110e-4054-99a4-c1da625c8302 None None] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:57:39.873276 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe466c57-110e-4054-99a4-c1da625c8302 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:57:39.893367 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.auth.core [None req-fe466c57-110e-4054-99a4-c1da625c8302 None None] MFA Rules not processed for user `bac4e6b76fe64217ac8fa44e71cffe6a`. Rule list: `[]` (Enabled: `True`). {{(pid=80143) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 10:57:39.894586 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-fe466c57-110e-4054-99a4-c1da625c8302 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 10:57:39.953269 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1104/2236] 10.222.0.25 () {58 vars in 958 bytes} [Sun Aug 7 10:57:39 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 07 10:57:39.963141 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89130b75-170f-4ba0-bc0b-7dce8eb735e8 admin admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 10:57:39.964940 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.fernet_utils [None req-89130b75-170f-4ba0-bc0b-7dce8eb735e8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80144) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 10:57:40.021536 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89130b75-170f-4ba0-bc0b-7dce8eb735e8 admin admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 10:57:40.025186 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b153b276-9134-4208-89c9-bd69f6c604a9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be1283c9bf47424c9aa3ad27cab03027', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6de1f3cac02c4bcdb05dd2337176033d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 10:57:40.025794 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b153b276-9134-4208-89c9-bd69f6c604a9 service neutron] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:57:40.026078 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b153b276-9134-4208-89c9-bd69f6c604a9 service neutron] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:57:40.026355 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b153b276-9134-4208-89c9-bd69f6c604a9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:57:40.035273 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b153b276-9134-4208-89c9-bd69f6c604a9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 10:57:40.036637 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b153b276-9134-4208-89c9-bd69f6c604a9 service neutron] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 10:57:40.045872 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 1133/2237] 10.222.0.25 () {60 vars in 1365 bytes} [Sun Aug 7 10:57:39 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 07 10:57:40.671483 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4849f8b0-fd6b-4851-9462-cb376daf394f None None] REQUEST_METHOD: `POST` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:57:40.671850 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4849f8b0-fd6b-4851-9462-cb376daf394f None None] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:57:40.672196 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4849f8b0-fd6b-4851-9462-cb376daf394f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:57:40.701013 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.auth.core [None req-4849f8b0-fd6b-4851-9462-cb376daf394f None None] MFA Rules not processed for user `c4597d928ad24b6c81420f7158e5e98b`. Rule list: `[]` (Enabled: `True`). {{(pid=80143) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 10:57:40.702506 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-4849f8b0-fd6b-4851-9462-cb376daf394f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 10:57:40.720863 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 10:57:40.773706 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1105/2238] 10.222.0.25 () {58 vars in 958 bytes} [Sun Aug 7 10:57:40 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 07 10:57:40.785161 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b153b276-9134-4208-89c9-bd69f6c604a9 service neutron] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 10:57:40.786547 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.fernet_utils [None req-b153b276-9134-4208-89c9-bd69f6c604a9 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80144) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 10:57:40.845772 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b153b276-9134-4208-89c9-bd69f6c604a9 service neutron] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 10:57:40.849275 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e896cd85-dbbb-4b1b-9278-06c1515bba49 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88dbd6dfd1164994a3e16da553561910', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6de1f3cac02c4bcdb05dd2337176033d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 10:57:40.849825 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e896cd85-dbbb-4b1b-9278-06c1515bba49 service cinder] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:57:40.850101 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e896cd85-dbbb-4b1b-9278-06c1515bba49 service cinder] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:57:40.850369 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e896cd85-dbbb-4b1b-9278-06c1515bba49 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:57:40.859886 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e896cd85-dbbb-4b1b-9278-06c1515bba49 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 10:57:40.861293 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e896cd85-dbbb-4b1b-9278-06c1515bba49 service cinder] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 10:57:40.870203 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 1134/2239] 10.222.0.25 () {60 vars in 1365 bytes} [Sun Aug 7 10:57:40 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 07 10:57:42.217744 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e9fbfee-4a38-472b-828a-191d28e865ea None None] REQUEST_METHOD: `POST` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:57:42.218252 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e9fbfee-4a38-472b-828a-191d28e865ea None None] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:57:42.218375 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e9fbfee-4a38-472b-828a-191d28e865ea None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:57:42.237832 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.auth.core [None req-9e9fbfee-4a38-472b-828a-191d28e865ea None None] MFA Rules not processed for user `c4597d928ad24b6c81420f7158e5e98b`. Rule list: `[]` (Enabled: `True`). {{(pid=80143) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 10:57:42.239225 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-9e9fbfee-4a38-472b-828a-191d28e865ea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 10:57:42.246083 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1106/2240] 10.222.0.25 () {58 vars in 958 bytes} [Sun Aug 7 10:57:42 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 07 10:57:43.298732 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e896cd85-dbbb-4b1b-9278-06c1515bba49 service cinder] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 10:57:43.300244 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.fernet_utils [None req-e896cd85-dbbb-4b1b-9278-06c1515bba49 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80144) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 10:57:43.319322 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:DELETE=4,keystone:SELECT=93,keystone:INSERT=3 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 10:57:43.371989 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e896cd85-dbbb-4b1b-9278-06c1515bba49 service cinder] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 10:57:43.376976 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-def41205-7c0e-4f4b-a4f7-fe42f70cd486 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6ee2e4c08af04c219ed395d7ca7085a6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6de1f3cac02c4bcdb05dd2337176033d', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 10:57:43.377582 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-def41205-7c0e-4f4b-a4f7-fe42f70cd486 service placement] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:57:43.377867 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-def41205-7c0e-4f4b-a4f7-fe42f70cd486 service placement] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:57:43.378143 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-def41205-7c0e-4f4b-a4f7-fe42f70cd486 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:57:43.379908 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.fernet_utils [None req-def41205-7c0e-4f4b-a4f7-fe42f70cd486 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80144) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 10:57:43.391128 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-def41205-7c0e-4f4b-a4f7-fe42f70cd486 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 10:57:43.392714 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-def41205-7c0e-4f4b-a4f7-fe42f70cd486 service placement] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 10:57:43.402489 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 1135/2241] 10.222.0.25 () {60 vars in 1365 bytes} [Sun Aug 7 10:57:43 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 104 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 07 10:57:52.232204 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=22 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 10:57:53.398382 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=25 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 10:58:15.881291 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d649ef9-0b75-49c9-b4d9-d9525cb59e7f None None] REQUEST_METHOD: `POST` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:58:15.881801 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d649ef9-0b75-49c9-b4d9-d9525cb59e7f None None] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:58:15.881899 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d649ef9-0b75-49c9-b4d9-d9525cb59e7f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:58:15.900530 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.auth.core [None req-1d649ef9-0b75-49c9-b4d9-d9525cb59e7f None None] MFA Rules not processed for user `de1d6c24b1a14d10a20d82a4c65793d8`. Rule list: `[]` (Enabled: `True`). {{(pid=80143) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 10:58:15.902228 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-1d649ef9-0b75-49c9-b4d9-d9525cb59e7f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 10:58:15.957373 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1107/2242] 10.222.0.25 () {58 vars in 1009 bytes} [Sun Aug 7 10:58:15 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 77 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 07 10:58:16.008727 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-def41205-7c0e-4f4b-a4f7-fe42f70cd486 service placement] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 10:58:16.010172 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.fernet_utils [None req-def41205-7c0e-4f4b-a4f7-fe42f70cd486 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80144) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 10:58:16.081825 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-def41205-7c0e-4f4b-a4f7-fe42f70cd486 service placement] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 10:58:16.086656 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc3ee446-f2f7-4298-ba92-67562743aea5 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69490ea2a9a340bf91aca38bf5655b45', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6de1f3cac02c4bcdb05dd2337176033d', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 10:58:16.087291 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc3ee446-f2f7-4298-ba92-67562743aea5 service swift] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:58:16.087569 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc3ee446-f2f7-4298-ba92-67562743aea5 service swift] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:58:16.087837 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc3ee446-f2f7-4298-ba92-67562743aea5 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:58:16.095659 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc3ee446-f2f7-4298-ba92-67562743aea5 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 10:58:16.097229 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc3ee446-f2f7-4298-ba92-67562743aea5 service swift] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 10:58:16.105041 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 1136/2243] 10.222.0.25 () {60 vars in 1384 bytes} [Sun Aug 7 10:58:16 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 07 10:58:25.951031 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=25 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 10:58:26.101840 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=32 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 10:58:39.987176 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6b78e8d-f611-4d6e-a722-62a9eaf1d7af None None] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:58:39.987785 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6b78e8d-f611-4d6e-a722-62a9eaf1d7af None None] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:58:39.987785 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6b78e8d-f611-4d6e-a722-62a9eaf1d7af None None] PATH_INFO: `/` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:58:39.989338 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1108/2244] 10.222.0.25 () {54 vars in 921 bytes} [Sun Aug 7 10:58:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 07 10:58:39.996977 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-891096c4-f075-463e-8689-53076f252e9a None None] REQUEST_METHOD: `POST` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:58:39.997508 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-891096c4-f075-463e-8689-53076f252e9a None None] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:58:39.997616 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-891096c4-f075-463e-8689-53076f252e9a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:58:40.019673 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.auth.core [None req-891096c4-f075-463e-8689-53076f252e9a None None] MFA Rules not processed for user `31d16d1b96104b4eb690398432b9829e`. Rule list: `[]` (Enabled: `True`). {{(pid=80144) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 10:58:40.020893 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.fernet_utils [None req-891096c4-f075-463e-8689-53076f252e9a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80144) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 10:58:40.215740 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 1137/2245] 10.222.0.25 () {58 vars in 1018 bytes} [Sun Aug 7 10:58:39 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 220 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 07 10:58:40.227049 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6b78e8d-f611-4d6e-a722-62a9eaf1d7af None None] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 10:58:40.228692 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-d6b78e8d-f611-4d6e-a722-62a9eaf1d7af None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 10:58:40.286405 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6b78e8d-f611-4d6e-a722-62a9eaf1d7af None None] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 10:58:40.289946 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91ceb54d-d907-4002-935a-3f53e4ae9e3d service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a11186d9d0d6488b953ede2f1628148d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6de1f3cac02c4bcdb05dd2337176033d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 10:58:40.290549 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91ceb54d-d907-4002-935a-3f53e4ae9e3d service glance] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:58:40.290833 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91ceb54d-d907-4002-935a-3f53e4ae9e3d service glance] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:58:40.291142 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91ceb54d-d907-4002-935a-3f53e4ae9e3d service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:58:40.301807 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91ceb54d-d907-4002-935a-3f53e4ae9e3d service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 10:58:40.303427 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91ceb54d-d907-4002-935a-3f53e4ae9e3d service glance] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 10:58:40.314781 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1109/2246] 10.222.0.25 () {60 vars in 1365 bytes} [Sun Aug 7 10:58:40 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 07 10:58:40.383370 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e40231c-2bb1-4643-a9be-a9ab198310cf None None] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:58:40.383681 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e40231c-2bb1-4643-a9be-a9ab198310cf None None] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:58:40.383952 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e40231c-2bb1-4643-a9be-a9ab198310cf None None] PATH_INFO: `/` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:58:40.385283 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 1138/2247] 10.222.0.25 () {54 vars in 921 bytes} [Sun Aug 7 10:58:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 07 10:58:40.390162 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1174668c-4c4a-4efd-91ef-e07e2eb72429 None None] REQUEST_METHOD: `POST` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:58:40.390457 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1174668c-4c4a-4efd-91ef-e07e2eb72429 None None] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:58:40.390763 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1174668c-4c4a-4efd-91ef-e07e2eb72429 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:58:40.410172 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.auth.core [None req-1174668c-4c4a-4efd-91ef-e07e2eb72429 None None] MFA Rules not processed for user `be1283c9bf47424c9aa3ad27cab03027`. Rule list: `[]` (Enabled: `True`). {{(pid=80143) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 10:58:40.411434 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-1174668c-4c4a-4efd-91ef-e07e2eb72429 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 10:58:40.419128 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1110/2248] 10.222.0.25 () {58 vars in 1018 bytes} [Sun Aug 7 10:58:40 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 07 10:58:40.431789 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e40231c-2bb1-4643-a9be-a9ab198310cf None None] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 10:58:40.434144 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.fernet_utils [None req-2e40231c-2bb1-4643-a9be-a9ab198310cf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80144) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 10:58:40.447178 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e40231c-2bb1-4643-a9be-a9ab198310cf None None] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 10:58:40.450685 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40f4b732-fe0c-4e10-a9e9-fcbd7c3c461e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be1283c9bf47424c9aa3ad27cab03027', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6de1f3cac02c4bcdb05dd2337176033d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 10:58:40.451309 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40f4b732-fe0c-4e10-a9e9-fcbd7c3c461e service neutron] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:58:40.451593 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40f4b732-fe0c-4e10-a9e9-fcbd7c3c461e service neutron] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:58:40.451890 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40f4b732-fe0c-4e10-a9e9-fcbd7c3c461e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:58:40.458738 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40f4b732-fe0c-4e10-a9e9-fcbd7c3c461e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 10:58:40.460163 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40f4b732-fe0c-4e10-a9e9-fcbd7c3c461e service neutron] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 10:58:40.468168 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 1139/2249] 10.222.0.25 () {60 vars in 1365 bytes} [Sun Aug 7 10:58:40 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 07 10:58:40.553522 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a0712e3-4244-41f2-a619-c25b31d39404 None None] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:58:40.553919 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a0712e3-4244-41f2-a619-c25b31d39404 None None] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:58:40.554225 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a0712e3-4244-41f2-a619-c25b31d39404 None None] PATH_INFO: `/` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:58:40.555631 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1111/2250] 10.222.0.25 () {54 vars in 921 bytes} [Sun Aug 7 10:58:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 07 10:58:40.561638 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5852d63-df1e-4840-9002-362b39857bba None None] REQUEST_METHOD: `POST` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:58:40.562017 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5852d63-df1e-4840-9002-362b39857bba None None] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:58:40.562364 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5852d63-df1e-4840-9002-362b39857bba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:58:40.581583 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.auth.core [None req-a5852d63-df1e-4840-9002-362b39857bba None None] MFA Rules not processed for user `31d16d1b96104b4eb690398432b9829e`. Rule list: `[]` (Enabled: `True`). {{(pid=80144) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 10:58:40.582775 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.fernet_utils [None req-a5852d63-df1e-4840-9002-362b39857bba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80144) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 10:58:40.592685 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 1140/2251] 10.222.0.25 () {58 vars in 1018 bytes} [Sun Aug 7 10:58:40 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 07 10:58:40.603051 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a0712e3-4244-41f2-a619-c25b31d39404 None None] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 10:58:40.615339 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a0712e3-4244-41f2-a619-c25b31d39404 None None] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 10:58:40.619057 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1340857f-f97b-48a1-a9b3-ca382b7de79e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be1283c9bf47424c9aa3ad27cab03027', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6de1f3cac02c4bcdb05dd2337176033d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 10:58:40.619652 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1340857f-f97b-48a1-a9b3-ca382b7de79e service neutron] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:58:40.619952 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1340857f-f97b-48a1-a9b3-ca382b7de79e service neutron] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:58:40.620226 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1340857f-f97b-48a1-a9b3-ca382b7de79e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:58:40.630192 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1340857f-f97b-48a1-a9b3-ca382b7de79e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 10:58:40.631645 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1340857f-f97b-48a1-a9b3-ca382b7de79e service neutron] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 10:58:40.642167 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1112/2252] 10.222.0.25 () {60 vars in 1365 bytes} [Sun Aug 7 10:58:40 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 07 10:58:50.576998 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=40 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 10:58:50.637566 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=35 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 10:58:51.414092 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6231edde-2c73-4f33-998d-dc3c696033b3 None None] REQUEST_METHOD: `POST` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:58:51.414382 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6231edde-2c73-4f33-998d-dc3c696033b3 None None] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:58:51.414661 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6231edde-2c73-4f33-998d-dc3c696033b3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:58:51.434035 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.auth.core [None req-6231edde-2c73-4f33-998d-dc3c696033b3 None None] MFA Rules not processed for user `de1d6c24b1a14d10a20d82a4c65793d8`. Rule list: `[]` (Enabled: `True`). {{(pid=80144) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 10:58:51.435266 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.fernet_utils [None req-6231edde-2c73-4f33-998d-dc3c696033b3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80144) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 10:58:51.442582 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 1141/2253] 10.222.0.25 () {58 vars in 1009 bytes} [Sun Aug 7 10:58:51 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 07 10:58:51.455417 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1340857f-f97b-48a1-a9b3-ca382b7de79e service neutron] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 10:58:51.467297 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1340857f-f97b-48a1-a9b3-ca382b7de79e service neutron] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 10:58:51.472254 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0212072b-1f72-4ce5-890b-18d26576d9b7 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69490ea2a9a340bf91aca38bf5655b45', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6de1f3cac02c4bcdb05dd2337176033d', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 10:58:51.472848 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0212072b-1f72-4ce5-890b-18d26576d9b7 service swift] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:58:51.473132 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0212072b-1f72-4ce5-890b-18d26576d9b7 service swift] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:58:51.473408 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0212072b-1f72-4ce5-890b-18d26576d9b7 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:58:51.481376 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0212072b-1f72-4ce5-890b-18d26576d9b7 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 10:58:51.483237 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0212072b-1f72-4ce5-890b-18d26576d9b7 service swift] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 10:58:51.491300 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1113/2254] 10.222.0.25 () {60 vars in 1384 bytes} [Sun Aug 7 10:58:51 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 07 10:59:01.429193 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=3 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 10:59:01.488053 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=6 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 10:59:05.249524 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6231edde-2c73-4f33-998d-dc3c696033b3 None None] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 10:59:05.259913 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6231edde-2c73-4f33-998d-dc3c696033b3 None None] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 10:59:05.264118 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-221882fb-751d-4cc4-aec1-3bbc0cae8eb0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be1283c9bf47424c9aa3ad27cab03027', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6de1f3cac02c4bcdb05dd2337176033d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 10:59:05.264837 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-221882fb-751d-4cc4-aec1-3bbc0cae8eb0 service neutron] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:59:05.265185 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-221882fb-751d-4cc4-aec1-3bbc0cae8eb0 service neutron] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:59:05.265532 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-221882fb-751d-4cc4-aec1-3bbc0cae8eb0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:59:05.267415 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.fernet_utils [None req-221882fb-751d-4cc4-aec1-3bbc0cae8eb0 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80144) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 10:59:05.282558 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-221882fb-751d-4cc4-aec1-3bbc0cae8eb0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 10:59:05.284078 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-221882fb-751d-4cc4-aec1-3bbc0cae8eb0 service neutron] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 10:59:05.297645 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 1142/2255] 10.222.0.25 () {60 vars in 1365 bytes} [Sun Aug 7 10:59:05 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 48 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 07 10:59:06.461578 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0212072b-1f72-4ce5-890b-18d26576d9b7 service swift] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 10:59:06.472076 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0212072b-1f72-4ce5-890b-18d26576d9b7 service swift] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 10:59:06.475821 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1723f477-855a-4909-af02-d417a2bfefa7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be1283c9bf47424c9aa3ad27cab03027', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6de1f3cac02c4bcdb05dd2337176033d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 10:59:06.476418 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1723f477-855a-4909-af02-d417a2bfefa7 service neutron] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:59:06.476699 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1723f477-855a-4909-af02-d417a2bfefa7 service neutron] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:59:06.476979 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1723f477-855a-4909-af02-d417a2bfefa7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:59:06.478698 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-1723f477-855a-4909-af02-d417a2bfefa7 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 10:59:06.492732 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1723f477-855a-4909-af02-d417a2bfefa7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 10:59:06.494083 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1723f477-855a-4909-af02-d417a2bfefa7 service neutron] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 10:59:06.505882 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1114/2256] 10.222.0.25 () {60 vars in 1365 bytes} [Sun Aug 7 10:59:06 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 45 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 07 10:59:15.290594 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=6 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 10:59:16.501547 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=6 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 10:59:21.030692 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-221882fb-751d-4cc4-aec1-3bbc0cae8eb0 service neutron] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 10:59:21.032210 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.fernet_utils [None req-221882fb-751d-4cc4-aec1-3bbc0cae8eb0 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80144) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 10:59:21.046002 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-221882fb-751d-4cc4-aec1-3bbc0cae8eb0 service neutron] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 10:59:21.051646 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea6ccca6-c467-4a95-bdf2-ae7e6cbefc8a service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31d16d1b96104b4eb690398432b9829e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6de1f3cac02c4bcdb05dd2337176033d', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'reader', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 10:59:21.052268 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea6ccca6-c467-4a95-bdf2-ae7e6cbefc8a service nova] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:59:21.052545 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea6ccca6-c467-4a95-bdf2-ae7e6cbefc8a service nova] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:59:21.052812 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea6ccca6-c467-4a95-bdf2-ae7e6cbefc8a service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:59:21.054412 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.fernet_utils [None req-ea6ccca6-c467-4a95-bdf2-ae7e6cbefc8a service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80144) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 10:59:21.114859 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea6ccca6-c467-4a95-bdf2-ae7e6cbefc8a service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 10:59:21.116464 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea6ccca6-c467-4a95-bdf2-ae7e6cbefc8a service nova] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 10:59:21.125331 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 1143/2257] 10.222.0.25 () {60 vars in 1365 bytes} [Sun Aug 7 10:59:21 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 07 10:59:29.378062 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f486887b-eca5-41d8-9a79-b46292aa638b None None] REQUEST_METHOD: `POST` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:59:29.378524 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f486887b-eca5-41d8-9a79-b46292aa638b None None] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:59:29.378803 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f486887b-eca5-41d8-9a79-b46292aa638b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:59:29.401840 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.auth.core [None req-f486887b-eca5-41d8-9a79-b46292aa638b None None] MFA Rules not processed for user `c4597d928ad24b6c81420f7158e5e98b`. Rule list: `[]` (Enabled: `True`). {{(pid=80143) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 10:59:29.403098 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-f486887b-eca5-41d8-9a79-b46292aa638b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 10:59:29.424282 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1115/2258] 10.222.0.25 () {58 vars in 958 bytes} [Sun Aug 7 10:59:29 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 44 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 07 10:59:29.710242 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea6ccca6-c467-4a95-bdf2-ae7e6cbefc8a service nova] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 10:59:29.721218 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea6ccca6-c467-4a95-bdf2-ae7e6cbefc8a service nova] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 10:59:29.725425 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27284ce5-5f69-476e-a67d-54a59ae8fab6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be1283c9bf47424c9aa3ad27cab03027', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6de1f3cac02c4bcdb05dd2337176033d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 10:59:29.726208 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27284ce5-5f69-476e-a67d-54a59ae8fab6 service neutron] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:59:29.726671 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27284ce5-5f69-476e-a67d-54a59ae8fab6 service neutron] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:59:29.727126 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27284ce5-5f69-476e-a67d-54a59ae8fab6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:59:29.728947 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.fernet_utils [None req-27284ce5-5f69-476e-a67d-54a59ae8fab6 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80144) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 10:59:29.740875 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27284ce5-5f69-476e-a67d-54a59ae8fab6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 10:59:29.742270 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27284ce5-5f69-476e-a67d-54a59ae8fab6 service neutron] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 10:59:29.750187 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 1144/2259] 10.222.0.25 () {60 vars in 1365 bytes} [Sun Aug 7 10:59:29 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 07 10:59:30.061279 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f486887b-eca5-41d8-9a79-b46292aa638b None None] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 10:59:30.062736 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-f486887b-eca5-41d8-9a79-b46292aa638b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 10:59:30.078536 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f486887b-eca5-41d8-9a79-b46292aa638b None None] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 10:59:30.083546 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42570131-56d9-42e4-a7f2-074da2b2a627 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6ee2e4c08af04c219ed395d7ca7085a6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6de1f3cac02c4bcdb05dd2337176033d', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 10:59:30.084163 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42570131-56d9-42e4-a7f2-074da2b2a627 service placement] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:59:30.084446 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42570131-56d9-42e4-a7f2-074da2b2a627 service placement] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:59:30.084722 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42570131-56d9-42e4-a7f2-074da2b2a627 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:59:30.087589 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-42570131-56d9-42e4-a7f2-074da2b2a627 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 10:59:30.107646 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42570131-56d9-42e4-a7f2-074da2b2a627 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 10:59:30.107646 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42570131-56d9-42e4-a7f2-074da2b2a627 service placement] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 10:59:30.113784 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1116/2260] 10.222.0.25 () {60 vars in 1365 bytes} [Sun Aug 7 10:59:30 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 53 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 07 10:59:31.022598 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27284ce5-5f69-476e-a67d-54a59ae8fab6 service neutron] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 10:59:31.031289 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27284ce5-5f69-476e-a67d-54a59ae8fab6 service neutron] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 10:59:31.034950 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9219c439-6488-4b04-b4f7-19dc1d3e0df9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be1283c9bf47424c9aa3ad27cab03027', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6de1f3cac02c4bcdb05dd2337176033d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 10:59:31.035556 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9219c439-6488-4b04-b4f7-19dc1d3e0df9 service neutron] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:59:31.035830 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9219c439-6488-4b04-b4f7-19dc1d3e0df9 service neutron] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:59:31.036094 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9219c439-6488-4b04-b4f7-19dc1d3e0df9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:59:31.037893 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.fernet_utils [None req-9219c439-6488-4b04-b4f7-19dc1d3e0df9 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80144) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 10:59:31.048650 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9219c439-6488-4b04-b4f7-19dc1d3e0df9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 10:59:31.050267 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9219c439-6488-4b04-b4f7-19dc1d3e0df9 service neutron] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 10:59:31.058494 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 1145/2261] 10.222.0.25 () {60 vars in 1365 bytes} [Sun Aug 7 10:59:31 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 07 10:59:32.450242 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42570131-56d9-42e4-a7f2-074da2b2a627 service placement] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 10:59:32.451695 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-42570131-56d9-42e4-a7f2-074da2b2a627 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 10:59:32.467004 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42570131-56d9-42e4-a7f2-074da2b2a627 service placement] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 10:59:32.472798 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd8a8019-2f84-4add-a086-f3df3c0021e6 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31d16d1b96104b4eb690398432b9829e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6de1f3cac02c4bcdb05dd2337176033d', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'reader', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 10:59:32.473340 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd8a8019-2f84-4add-a086-f3df3c0021e6 service nova] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:59:32.473626 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd8a8019-2f84-4add-a086-f3df3c0021e6 service nova] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:59:32.473904 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd8a8019-2f84-4add-a086-f3df3c0021e6 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:59:32.475639 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-cd8a8019-2f84-4add-a086-f3df3c0021e6 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 10:59:32.535239 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd8a8019-2f84-4add-a086-f3df3c0021e6 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 10:59:32.537683 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd8a8019-2f84-4add-a086-f3df3c0021e6 service nova] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 10:59:32.548080 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1117/2262] 10.222.0.25 () {60 vars in 1365 bytes} [Sun Aug 7 10:59:32 2022] GET /identity/v3/auth/tokens => generated 3270 bytes in 98 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 07 10:59:33.695965 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-212b68d3-be47-44e7-94ff-2c0b6e459022 None None] REQUEST_METHOD: `POST` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:59:33.695965 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-212b68d3-be47-44e7-94ff-2c0b6e459022 None None] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:59:33.696461 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-212b68d3-be47-44e7-94ff-2c0b6e459022 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:59:33.718053 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.auth.core [None req-212b68d3-be47-44e7-94ff-2c0b6e459022 None None] MFA Rules not processed for user `de1d6c24b1a14d10a20d82a4c65793d8`. Rule list: `[]` (Enabled: `True`). {{(pid=80144) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 10:59:33.719311 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.fernet_utils [None req-212b68d3-be47-44e7-94ff-2c0b6e459022 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80144) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 10:59:33.727624 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 1146/2263] 10.222.0.25 () {58 vars in 967 bytes} [Sun Aug 7 10:59:33 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 07 10:59:33.751346 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd8a8019-2f84-4add-a086-f3df3c0021e6 service nova] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 10:59:33.763854 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd8a8019-2f84-4add-a086-f3df3c0021e6 service nova] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 10:59:33.769086 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7808edf-f057-4e6f-a700-e28a32f95929 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69490ea2a9a340bf91aca38bf5655b45', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6de1f3cac02c4bcdb05dd2337176033d', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 10:59:33.769704 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7808edf-f057-4e6f-a700-e28a32f95929 service swift] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:59:33.769985 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7808edf-f057-4e6f-a700-e28a32f95929 service swift] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:59:33.770263 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7808edf-f057-4e6f-a700-e28a32f95929 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:59:33.779540 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7808edf-f057-4e6f-a700-e28a32f95929 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 10:59:33.781095 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7808edf-f057-4e6f-a700-e28a32f95929 service swift] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 10:59:33.789010 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1118/2264] 10.222.0.25 () {60 vars in 1384 bytes} [Sun Aug 7 10:59:33 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Aug 07 10:59:37.897158 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cf987e1-e017-44d4-8501-c9ce50d70a0b None None] REQUEST_METHOD: `POST` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:59:37.897712 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cf987e1-e017-44d4-8501-c9ce50d70a0b None None] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:59:37.897712 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cf987e1-e017-44d4-8501-c9ce50d70a0b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:59:37.916222 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.auth.core [None req-2cf987e1-e017-44d4-8501-c9ce50d70a0b None None] MFA Rules not processed for user `06231f178e0d494084915b9368fa3b9b`. Rule list: `[]` (Enabled: `True`). {{(pid=80144) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 10:59:37.917401 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.fernet_utils [None req-2cf987e1-e017-44d4-8501-c9ce50d70a0b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80144) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 10:59:37.972424 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 1147/2265] 10.222.0.25 () {58 vars in 958 bytes} [Sun Aug 7 10:59:37 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 76 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 07 10:59:38.319268 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7808edf-f057-4e6f-a700-e28a32f95929 service swift] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 10:59:38.328788 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7808edf-f057-4e6f-a700-e28a32f95929 service swift] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 10:59:38.332328 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-128656ee-8035-4302-9764-b0cdfd82f775 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be1283c9bf47424c9aa3ad27cab03027', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6de1f3cac02c4bcdb05dd2337176033d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 10:59:38.332918 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-128656ee-8035-4302-9764-b0cdfd82f775 service neutron] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:59:38.333199 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-128656ee-8035-4302-9764-b0cdfd82f775 service neutron] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:59:38.333474 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-128656ee-8035-4302-9764-b0cdfd82f775 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:59:38.335346 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-128656ee-8035-4302-9764-b0cdfd82f775 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 10:59:38.344685 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-128656ee-8035-4302-9764-b0cdfd82f775 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 10:59:38.346093 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-128656ee-8035-4302-9764-b0cdfd82f775 service neutron] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 10:59:38.353885 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1119/2266] 10.222.0.25 () {60 vars in 1365 bytes} [Sun Aug 7 10:59:38 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 35 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 07 10:59:42.649052 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cf987e1-e017-44d4-8501-c9ce50d70a0b None None] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 10:59:42.668504 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cf987e1-e017-44d4-8501-c9ce50d70a0b None None] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 10:59:42.674241 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4905f024-c425-41ea-b218-8ad048a87ed7 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31d16d1b96104b4eb690398432b9829e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6de1f3cac02c4bcdb05dd2337176033d', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'reader', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 10:59:42.674995 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4905f024-c425-41ea-b218-8ad048a87ed7 service nova] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:59:42.675278 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4905f024-c425-41ea-b218-8ad048a87ed7 service nova] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:59:42.675554 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4905f024-c425-41ea-b218-8ad048a87ed7 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:59:42.677276 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.fernet_utils [None req-4905f024-c425-41ea-b218-8ad048a87ed7 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80144) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 10:59:42.691062 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4905f024-c425-41ea-b218-8ad048a87ed7 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 10:59:42.692930 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4905f024-c425-41ea-b218-8ad048a87ed7 service nova] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 10:59:42.705034 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 1148/2267] 10.222.0.25 () {60 vars in 1365 bytes} [Sun Aug 7 10:59:42 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 56 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 07 10:59:48.350464 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=49 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 10:59:50.974035 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-128656ee-8035-4302-9764-b0cdfd82f775 service neutron] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 10:59:50.987845 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-128656ee-8035-4302-9764-b0cdfd82f775 service neutron] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 10:59:50.993296 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-689377ab-ec25-45bf-89a1-77784c27f883 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31d16d1b96104b4eb690398432b9829e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6de1f3cac02c4bcdb05dd2337176033d', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'reader', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 10:59:50.994084 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-689377ab-ec25-45bf-89a1-77784c27f883 service nova] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 10:59:50.994432 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-689377ab-ec25-45bf-89a1-77784c27f883 service nova] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 10:59:50.994773 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-689377ab-ec25-45bf-89a1-77784c27f883 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 10:59:50.996621 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-689377ab-ec25-45bf-89a1-77784c27f883 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 10:59:51.008863 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-689377ab-ec25-45bf-89a1-77784c27f883 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 10:59:51.010494 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-689377ab-ec25-45bf-89a1-77784c27f883 service nova] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 10:59:51.021669 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1120/2268] 10.222.0.25 () {60 vars in 1365 bytes} [Sun Aug 7 10:59:50 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 48 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 07 10:59:52.700358 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=74 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 11:00:01.017281 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=6 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 11:00:01.972906 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4905f024-c425-41ea-b218-8ad048a87ed7 service nova] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 11:00:01.984755 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4905f024-c425-41ea-b218-8ad048a87ed7 service nova] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 11:00:01.990254 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4786d860-5770-45cb-88a4-b0e21e514836 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31d16d1b96104b4eb690398432b9829e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6de1f3cac02c4bcdb05dd2337176033d', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'reader', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 11:00:01.990942 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4786d860-5770-45cb-88a4-b0e21e514836 service nova] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 11:00:01.991237 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4786d860-5770-45cb-88a4-b0e21e514836 service nova] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 11:00:01.991508 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4786d860-5770-45cb-88a4-b0e21e514836 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 11:00:01.993210 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.fernet_utils [None req-4786d860-5770-45cb-88a4-b0e21e514836 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80144) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 11:00:02.005822 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4786d860-5770-45cb-88a4-b0e21e514836 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 11:00:02.007408 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4786d860-5770-45cb-88a4-b0e21e514836 service nova] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 11:00:02.017670 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 1149/2269] 10.222.0.25 () {60 vars in 1365 bytes} [Sun Aug 7 11:00:01 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 45 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 07 11:00:07.440636 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-689377ab-ec25-45bf-89a1-77784c27f883 service nova] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 11:00:07.453962 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-689377ab-ec25-45bf-89a1-77784c27f883 service nova] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 11:00:07.459957 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2eac1a8-583b-410a-b492-1d649aa00a33 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31d16d1b96104b4eb690398432b9829e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6de1f3cac02c4bcdb05dd2337176033d', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'reader', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 11:00:07.460560 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2eac1a8-583b-410a-b492-1d649aa00a33 service nova] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 11:00:07.460838 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2eac1a8-583b-410a-b492-1d649aa00a33 service nova] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 11:00:07.461105 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2eac1a8-583b-410a-b492-1d649aa00a33 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 11:00:07.462825 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-c2eac1a8-583b-410a-b492-1d649aa00a33 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 11:00:07.520802 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2eac1a8-583b-410a-b492-1d649aa00a33 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 11:00:07.522381 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2eac1a8-583b-410a-b492-1d649aa00a33 service nova] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 11:00:07.531672 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1121/2270] 10.222.0.25 () {60 vars in 1365 bytes} [Sun Aug 7 11:00:07 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 07 11:00:12.013389 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=6 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 11:00:17.527369 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=28 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 11:00:22.465668 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1589ce8-6c63-43ea-a3a5-e67f8d6b28c0 None None] REQUEST_METHOD: `POST` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 11:00:22.466181 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1589ce8-6c63-43ea-a3a5-e67f8d6b28c0 None None] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 11:00:22.466293 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1589ce8-6c63-43ea-a3a5-e67f8d6b28c0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 11:00:22.484842 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.auth.core [None req-d1589ce8-6c63-43ea-a3a5-e67f8d6b28c0 None None] MFA Rules not processed for user `c3a12c3144844d32812e010550bb0921`. Rule list: `[]` (Enabled: `True`). {{(pid=80144) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 11:00:22.486018 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.fernet_utils [None req-d1589ce8-6c63-43ea-a3a5-e67f8d6b28c0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80144) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 11:00:22.492693 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 1150/2271] 10.222.0.25 () {58 vars in 958 bytes} [Sun Aug 7 11:00:22 2022] POST /identity/v3/auth/tokens => generated 3270 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 07 11:00:32.480052 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=3 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 11:00:46.892143 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2eac1a8-583b-410a-b492-1d649aa00a33 service nova] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 11:00:46.904811 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2eac1a8-583b-410a-b492-1d649aa00a33 service nova] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 11:00:46.910055 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81d15fb2-9d73-4de2-8607-e9e43ae8ddbf service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6ee2e4c08af04c219ed395d7ca7085a6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6de1f3cac02c4bcdb05dd2337176033d', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 11:00:46.911004 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81d15fb2-9d73-4de2-8607-e9e43ae8ddbf service placement] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 11:00:46.911305 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81d15fb2-9d73-4de2-8607-e9e43ae8ddbf service placement] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 11:00:46.911534 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81d15fb2-9d73-4de2-8607-e9e43ae8ddbf service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 11:00:46.913458 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-81d15fb2-9d73-4de2-8607-e9e43ae8ddbf service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 11:00:46.926155 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81d15fb2-9d73-4de2-8607-e9e43ae8ddbf service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 11:00:46.927770 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81d15fb2-9d73-4de2-8607-e9e43ae8ddbf service placement] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 11:00:46.939466 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1122/2272] 10.222.0.25 () {60 vars in 1365 bytes} [Sun Aug 7 11:00:46 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 48 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 07 11:00:46.945393 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1589ce8-6c63-43ea-a3a5-e67f8d6b28c0 None None] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 11:00:46.959053 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1589ce8-6c63-43ea-a3a5-e67f8d6b28c0 None None] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 11:00:46.965566 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8309d3b9-eede-41be-9493-f56400b91d8d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5ac47e3b09f43d19a23a0f457fe9e23', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 11:00:46.966277 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8309d3b9-eede-41be-9493-f56400b91d8d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 11:00:46.966629 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8309d3b9-eede-41be-9493-f56400b91d8d admin admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 11:00:46.966997 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8309d3b9-eede-41be-9493-f56400b91d8d admin admin] PATH_INFO: `/v3/users/c4597d928ad24b6c81420f7158e5e98b` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 11:00:46.968213 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8309d3b9-eede-41be-9493-f56400b91d8d admin admin] RBAC: Authorizing `identity:delete_user(user_id=c4597d928ad24b6c81420f7158e5e98b)` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 11:00:46.970214 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8309d3b9-eede-41be-9493-f56400b91d8d admin admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 11:00:47.016985 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.notifications [None req-8309d3b9-eede-41be-9493-f56400b91d8d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c4597d928ad24b6c81420f7158e5e98b'} {{(pid=80144) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 11:00:47.023883 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.notifications [None req-8309d3b9-eede-41be-9493-f56400b91d8d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c4597d928ad24b6c81420f7158e5e98b'} {{(pid=80144) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 11:00:47.032754 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.notifications [None req-8309d3b9-eede-41be-9493-f56400b91d8d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c4597d928ad24b6c81420f7158e5e98b'} {{(pid=80144) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 11:00:47.033445 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.notifications [None req-8309d3b9-eede-41be-9493-f56400b91d8d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c4597d928ad24b6c81420f7158e5e98b'} {{(pid=80144) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 11:00:47.034051 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.notifications [None req-8309d3b9-eede-41be-9493-f56400b91d8d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c4597d928ad24b6c81420f7158e5e98b'} {{(pid=80144) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 11:00:47.051770 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 1151/2273] 10.222.0.25 () {58 vars in 1224 bytes} [Sun Aug 7 11:00:46 2022] DELETE /identity/v3/users/c4597d928ad24b6c81420f7158e5e98b => generated 0 bytes in 107 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 07 11:00:47.226782 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81d15fb2-9d73-4de2-8607-e9e43ae8ddbf service placement] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 11:00:47.246449 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81d15fb2-9d73-4de2-8607-e9e43ae8ddbf service placement] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 11:00:47.319978 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d965b16-87df-4232-a0b0-e6333f739ba5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5ac47e3b09f43d19a23a0f457fe9e23', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 11:00:47.320619 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d965b16-87df-4232-a0b0-e6333f739ba5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 11:00:47.320905 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d965b16-87df-4232-a0b0-e6333f739ba5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 11:00:47.321172 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d965b16-87df-4232-a0b0-e6333f739ba5 admin admin] PATH_INFO: `/v3/projects/81ab48958e894bdbb5cd46049356a8a0` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 11:00:47.322527 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d965b16-87df-4232-a0b0-e6333f739ba5 admin admin] RBAC: Authorizing `identity:delete_project(project_id=81ab48958e894bdbb5cd46049356a8a0)` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 11:00:47.324624 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d965b16-87df-4232-a0b0-e6333f739ba5 admin admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 11:00:47.372601 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-4d965b16-87df-4232-a0b0-e6333f739ba5 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '81ab48958e894bdbb5cd46049356a8a0', 'request_id': 'req-4d965b16-87df-4232-a0b0-e6333f739ba5'} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 11:00:47.388712 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-4d965b16-87df-4232-a0b0-e6333f739ba5 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '81ab48958e894bdbb5cd46049356a8a0', 'request_id': 'req-4d965b16-87df-4232-a0b0-e6333f739ba5'} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 11:00:47.388712 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-4d965b16-87df-4232-a0b0-e6333f739ba5 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '81ab48958e894bdbb5cd46049356a8a0', 'request_id': 'req-4d965b16-87df-4232-a0b0-e6333f739ba5'} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 11:00:47.397543 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-4d965b16-87df-4232-a0b0-e6333f739ba5 admin admin] The token cache is being invalidate because project 81ab48958e894bdbb5cd46049356a8a0 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80143) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 07 11:00:47.397838 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-4d965b16-87df-4232-a0b0-e6333f739ba5 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 11:00:47.398503 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-4d965b16-87df-4232-a0b0-e6333f739ba5 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 11:00:47.400213 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1123/2274] 10.222.0.25 () {58 vars in 1233 bytes} [Sun Aug 7 11:00:47 2022] DELETE /identity/v3/projects/81ab48958e894bdbb5cd46049356a8a0 => generated 0 bytes in 174 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 07 11:00:47.404966 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a20a3817-7295-4c90-978c-29dfa7c1c688 None None] REQUEST_METHOD: `POST` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 11:00:47.405252 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a20a3817-7295-4c90-978c-29dfa7c1c688 None None] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 11:00:47.405551 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a20a3817-7295-4c90-978c-29dfa7c1c688 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 11:00:47.425271 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.auth.core [None req-a20a3817-7295-4c90-978c-29dfa7c1c688 None None] MFA Rules not processed for user `bac4e6b76fe64217ac8fa44e71cffe6a`. Rule list: `[]` (Enabled: `True`). {{(pid=80144) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 11:00:47.426468 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.fernet_utils [None req-a20a3817-7295-4c90-978c-29dfa7c1c688 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80144) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 11:00:47.484661 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 1152/2275] 10.222.0.25 () {58 vars in 958 bytes} [Sun Aug 7 11:00:47 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 07 11:00:47.492145 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19aafcb3-71cc-421e-a0c9-9efa1ae7c9d6 None None] REQUEST_METHOD: `POST` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 11:00:47.492436 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19aafcb3-71cc-421e-a0c9-9efa1ae7c9d6 None None] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 11:00:47.492726 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19aafcb3-71cc-421e-a0c9-9efa1ae7c9d6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 11:00:47.513714 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.auth.core [None req-19aafcb3-71cc-421e-a0c9-9efa1ae7c9d6 None None] MFA Rules not processed for user `bac4e6b76fe64217ac8fa44e71cffe6a`. Rule list: `[]` (Enabled: `True`). {{(pid=80143) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 11:00:47.514942 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-19aafcb3-71cc-421e-a0c9-9efa1ae7c9d6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 11:00:47.522856 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1124/2276] 10.222.0.25 () {58 vars in 958 bytes} [Sun Aug 7 11:00:47 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 07 11:00:47.527143 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a20a3817-7295-4c90-978c-29dfa7c1c688 None None] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 11:00:47.537459 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a20a3817-7295-4c90-978c-29dfa7c1c688 None None] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 11:00:47.542632 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-376dcfa1-cd33-41d4-9b41-7121f30f7ace admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5ac47e3b09f43d19a23a0f457fe9e23', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 11:00:47.543254 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-376dcfa1-cd33-41d4-9b41-7121f30f7ace admin admin] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 11:00:47.543533 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-376dcfa1-cd33-41d4-9b41-7121f30f7ace admin admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 11:00:47.543802 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-376dcfa1-cd33-41d4-9b41-7121f30f7ace admin admin] PATH_INFO: `/v3/domains` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 11:00:47.544504 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-376dcfa1-cd33-41d4-9b41-7121f30f7ace admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80144) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 11:00:47.544834 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-376dcfa1-cd33-41d4-9b41-7121f30f7ace admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 11:00:47.546709 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-376dcfa1-cd33-41d4-9b41-7121f30f7ace admin admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 11:00:47.559290 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 1153/2277] 10.222.0.25 () {58 vars in 1153 bytes} [Sun Aug 7 11:00:47 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 11:00:47.563535 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19aafcb3-71cc-421e-a0c9-9efa1ae7c9d6 None None] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 11:00:47.572790 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19aafcb3-71cc-421e-a0c9-9efa1ae7c9d6 None None] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 11:00:47.577291 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0acfea12-05ae-4c0a-89d6-192440f0a48c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5ac47e3b09f43d19a23a0f457fe9e23', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 11:00:47.577883 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0acfea12-05ae-4c0a-89d6-192440f0a48c admin admin] REQUEST_METHOD: `POST` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 11:00:47.578162 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0acfea12-05ae-4c0a-89d6-192440f0a48c admin admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 11:00:47.578435 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0acfea12-05ae-4c0a-89d6-192440f0a48c admin admin] PATH_INFO: `/v3/projects` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 11:00:47.579099 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0acfea12-05ae-4c0a-89d6-192440f0a48c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 11:00:47.581006 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0acfea12-05ae-4c0a-89d6-192440f0a48c admin admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 11:00:47.609904 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1125/2278] 10.222.0.25 () {60 vars in 1153 bytes} [Sun Aug 7 11:00:47 2022] POST /identity/v3/projects => generated 389 bytes in 47 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 07 11:00:47.613790 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-376dcfa1-cd33-41d4-9b41-7121f30f7ace admin admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 11:00:47.624270 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-376dcfa1-cd33-41d4-9b41-7121f30f7ace admin admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 11:00:47.680603 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85076ed1-552f-4a48-b19b-f63d01cca745 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5ac47e3b09f43d19a23a0f457fe9e23', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 11:00:47.681207 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85076ed1-552f-4a48-b19b-f63d01cca745 admin admin] REQUEST_METHOD: `POST` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 11:00:47.681519 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85076ed1-552f-4a48-b19b-f63d01cca745 admin admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 11:00:47.681796 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85076ed1-552f-4a48-b19b-f63d01cca745 admin admin] PATH_INFO: `/v3/users` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 11:00:47.682427 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85076ed1-552f-4a48-b19b-f63d01cca745 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 11:00:47.684361 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85076ed1-552f-4a48-b19b-f63d01cca745 admin admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 11:00:47.692075 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 11:00:47.707789 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 1154/2279] 10.222.0.25 () {60 vars in 1144 bytes} [Sun Aug 7 11:00:47 2022] POST /identity/v3/users => generated 295 bytes in 94 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 07 11:00:47.711727 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0acfea12-05ae-4c0a-89d6-192440f0a48c admin admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 11:00:47.722523 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0acfea12-05ae-4c0a-89d6-192440f0a48c admin admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 11:00:47.727153 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5755088-baa6-4875-bebf-693b63e09b72 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5ac47e3b09f43d19a23a0f457fe9e23', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 11:00:47.727758 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5755088-baa6-4875-bebf-693b63e09b72 admin admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 11:00:47.728039 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5755088-baa6-4875-bebf-693b63e09b72 admin admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 11:00:47.728991 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5755088-baa6-4875-bebf-693b63e09b72 admin admin] PATH_INFO: `/v3/roles` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 11:00:47.729786 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5755088-baa6-4875-bebf-693b63e09b72 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 11:00:47.731420 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5755088-baa6-4875-bebf-693b63e09b72 admin admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 11:00:47.741469 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1126/2280] 10.222.0.25 () {58 vars in 1122 bytes} [Sun Aug 7 11:00:47 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 07 11:00:47.745328 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85076ed1-552f-4a48-b19b-f63d01cca745 admin admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 11:00:47.755088 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85076ed1-552f-4a48-b19b-f63d01cca745 admin admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 11:00:47.759653 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c68daf2-18ba-43c7-8744-ba6072b88a53 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5ac47e3b09f43d19a23a0f457fe9e23', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 11:00:47.760239 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c68daf2-18ba-43c7-8744-ba6072b88a53 admin admin] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 11:00:47.760511 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c68daf2-18ba-43c7-8744-ba6072b88a53 admin admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 11:00:47.760777 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c68daf2-18ba-43c7-8744-ba6072b88a53 admin admin] PATH_INFO: `/v3/roles` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 11:00:47.761441 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c68daf2-18ba-43c7-8744-ba6072b88a53 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 11:00:47.763025 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c68daf2-18ba-43c7-8744-ba6072b88a53 admin admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 11:00:47.773147 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 1155/2281] 10.222.0.25 () {58 vars in 1122 bytes} [Sun Aug 7 11:00:47 2022] GET /identity/v3/roles => generated 1420 bytes in 28 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 07 11:00:47.777645 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5755088-baa6-4875-bebf-693b63e09b72 admin admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 11:00:47.787655 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5755088-baa6-4875-bebf-693b63e09b72 admin admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 11:00:47.792959 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e3f0ddb-9a4e-4ac0-bcd7-ca5839264151 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5ac47e3b09f43d19a23a0f457fe9e23', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 11:00:47.793609 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e3f0ddb-9a4e-4ac0-bcd7-ca5839264151 admin admin] REQUEST_METHOD: `PUT` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 11:00:47.793895 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e3f0ddb-9a4e-4ac0-bcd7-ca5839264151 admin admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 11:00:47.794175 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e3f0ddb-9a4e-4ac0-bcd7-ca5839264151 admin admin] PATH_INFO: `/v3/projects/1bb09f7eb6cc4e44a3b7a3473d9be843/users/72d8a5d8dd3a45fdb8e78e228d2e421e/roles/b98dfb3b9fee4644af2887a3ac3fd2e6` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 11:00:47.808926 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e3f0ddb-9a4e-4ac0-bcd7-ca5839264151 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1bb09f7eb6cc4e44a3b7a3473d9be843, user_id=72d8a5d8dd3a45fdb8e78e228d2e421e, role_id=b98dfb3b9fee4644af2887a3ac3fd2e6)` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 11:00:47.811657 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e3f0ddb-9a4e-4ac0-bcd7-ca5839264151 admin admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 11:00:47.824379 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1127/2282] 10.222.0.25 () {60 vars in 1483 bytes} [Sun Aug 7 11:00:47 2022] PUT /identity/v3/projects/1bb09f7eb6cc4e44a3b7a3473d9be843/users/72d8a5d8dd3a45fdb8e78e228d2e421e/roles/b98dfb3b9fee4644af2887a3ac3fd2e6 => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 07 11:00:47.834412 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dda7a78-1fc2-4a57-a790-56e44a035582 None None] REQUEST_METHOD: `POST` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 11:00:47.834694 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dda7a78-1fc2-4a57-a790-56e44a035582 None None] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 11:00:47.834987 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dda7a78-1fc2-4a57-a790-56e44a035582 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 11:00:47.862536 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.auth.core [None req-4dda7a78-1fc2-4a57-a790-56e44a035582 None None] MFA Rules not processed for user `72d8a5d8dd3a45fdb8e78e228d2e421e`. Rule list: `[]` (Enabled: `True`). {{(pid=80144) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 11:00:47.863739 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.fernet_utils [None req-4dda7a78-1fc2-4a57-a790-56e44a035582 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80144) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 11:00:47.927476 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 1156/2283] 10.222.0.25 () {58 vars in 958 bytes} [Sun Aug 7 11:00:47 2022] POST /identity/v3/auth/tokens => generated 3266 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 07 11:00:47.938401 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e3f0ddb-9a4e-4ac0-bcd7-ca5839264151 admin admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 11:00:47.940421 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-9e3f0ddb-9a4e-4ac0-bcd7-ca5839264151 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 11:00:47.955336 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 11:00:48.000260 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e3f0ddb-9a4e-4ac0-bcd7-ca5839264151 admin admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 11:00:48.003854 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42660be3-6977-44bc-857d-0ba32b64f86d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88dbd6dfd1164994a3e16da553561910', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6de1f3cac02c4bcdb05dd2337176033d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 11:00:48.004453 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42660be3-6977-44bc-857d-0ba32b64f86d service cinder] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 11:00:48.004733 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42660be3-6977-44bc-857d-0ba32b64f86d service cinder] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 11:00:48.005003 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42660be3-6977-44bc-857d-0ba32b64f86d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 11:00:48.013780 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42660be3-6977-44bc-857d-0ba32b64f86d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 11:00:48.015229 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42660be3-6977-44bc-857d-0ba32b64f86d service cinder] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 11:00:48.023753 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1128/2284] 10.222.0.25 () {60 vars in 1365 bytes} [Sun Aug 7 11:00:47 2022] GET /identity/v3/auth/tokens => generated 3266 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 07 11:00:55.414791 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dda7a78-1fc2-4a57-a790-56e44a035582 None None] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 11:00:55.416851 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.fernet_utils [None req-4dda7a78-1fc2-4a57-a790-56e44a035582 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80144) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 11:00:55.491891 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dda7a78-1fc2-4a57-a790-56e44a035582 None None] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 11:00:55.498237 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-933866d6-5bd1-4afb-905a-278b3e56aa58 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31d16d1b96104b4eb690398432b9829e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6de1f3cac02c4bcdb05dd2337176033d', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'reader', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 11:00:55.498857 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-933866d6-5bd1-4afb-905a-278b3e56aa58 service nova] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 11:00:55.499180 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-933866d6-5bd1-4afb-905a-278b3e56aa58 service nova] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 11:00:55.499458 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-933866d6-5bd1-4afb-905a-278b3e56aa58 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 11:00:55.501240 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.fernet_utils [None req-933866d6-5bd1-4afb-905a-278b3e56aa58 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80144) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 11:00:55.562382 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-933866d6-5bd1-4afb-905a-278b3e56aa58 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 11:00:55.564620 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-933866d6-5bd1-4afb-905a-278b3e56aa58 service nova] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 11:00:55.578971 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 1157/2285] 10.222.0.25 () {60 vars in 1365 bytes} [Sun Aug 7 11:00:55 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 160 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 07 11:00:58.019990 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=20 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 11:01:05.569775 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:INSERT=3,keystone:SELECT=95 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 11:01:07.966550 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80399352-698a-445f-b461-9b08f19b3b48 None None] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 11:01:07.967168 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80399352-698a-445f-b461-9b08f19b3b48 None None] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 11:01:07.967168 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80399352-698a-445f-b461-9b08f19b3b48 None None] PATH_INFO: `/` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 11:01:07.968499 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1129/2286] 10.222.0.25 () {54 vars in 921 bytes} [Sun Aug 7 11:01:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 07 11:01:07.974976 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-419f4310-e792-4148-8140-6cd89d0042e1 None None] REQUEST_METHOD: `POST` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 11:01:07.975455 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-419f4310-e792-4148-8140-6cd89d0042e1 None None] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 11:01:07.975571 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-419f4310-e792-4148-8140-6cd89d0042e1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 11:01:07.998100 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.auth.core [None req-419f4310-e792-4148-8140-6cd89d0042e1 None None] MFA Rules not processed for user `be1283c9bf47424c9aa3ad27cab03027`. Rule list: `[]` (Enabled: `True`). {{(pid=80144) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 11:01:07.999333 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.fernet_utils [None req-419f4310-e792-4148-8140-6cd89d0042e1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80144) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 11:01:08.057837 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 1158/2287] 10.222.0.25 () {58 vars in 1018 bytes} [Sun Aug 7 11:01:07 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 07 11:01:08.070156 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80399352-698a-445f-b461-9b08f19b3b48 None None] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 11:01:08.071933 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-80399352-698a-445f-b461-9b08f19b3b48 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 11:01:08.086581 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80399352-698a-445f-b461-9b08f19b3b48 None None] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 11:01:08.090313 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a5fc0c4-181e-42f9-9c29-221e90995c16 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be1283c9bf47424c9aa3ad27cab03027', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6de1f3cac02c4bcdb05dd2337176033d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 11:01:08.090943 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a5fc0c4-181e-42f9-9c29-221e90995c16 service neutron] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 11:01:08.091242 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a5fc0c4-181e-42f9-9c29-221e90995c16 service neutron] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 11:01:08.091522 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a5fc0c4-181e-42f9-9c29-221e90995c16 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 11:01:08.104530 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a5fc0c4-181e-42f9-9c29-221e90995c16 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 11:01:08.104530 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a5fc0c4-181e-42f9-9c29-221e90995c16 service neutron] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 11:01:08.110091 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1130/2288] 10.222.0.25 () {60 vars in 1365 bytes} [Sun Aug 7 11:01:08 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 07 11:01:08.197099 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-095ede62-a929-45af-b79c-f50f9bdd1c3f None None] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 11:01:08.197451 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-095ede62-a929-45af-b79c-f50f9bdd1c3f None None] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 11:01:08.197950 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-095ede62-a929-45af-b79c-f50f9bdd1c3f None None] PATH_INFO: `/` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 11:01:08.199738 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 1159/2289] 10.222.0.25 () {54 vars in 921 bytes} [Sun Aug 7 11:01:08 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 07 11:01:08.204943 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-519201fe-d443-4176-8299-c3bbdae7b85d None None] REQUEST_METHOD: `POST` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 11:01:08.205302 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-519201fe-d443-4176-8299-c3bbdae7b85d None None] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 11:01:08.205678 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-519201fe-d443-4176-8299-c3bbdae7b85d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 11:01:08.226309 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.auth.core [None req-519201fe-d443-4176-8299-c3bbdae7b85d None None] MFA Rules not processed for user `31d16d1b96104b4eb690398432b9829e`. Rule list: `[]` (Enabled: `True`). {{(pid=80143) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 11:01:08.227678 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-519201fe-d443-4176-8299-c3bbdae7b85d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 11:01:08.238494 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1131/2290] 10.222.0.25 () {58 vars in 1018 bytes} [Sun Aug 7 11:01:08 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 07 11:01:08.249895 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-095ede62-a929-45af-b79c-f50f9bdd1c3f None None] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 11:01:08.261679 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-095ede62-a929-45af-b79c-f50f9bdd1c3f None None] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 11:01:08.265816 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91f2588e-42f1-419f-859a-52f42560e95c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be1283c9bf47424c9aa3ad27cab03027', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6de1f3cac02c4bcdb05dd2337176033d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 11:01:08.266430 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91f2588e-42f1-419f-859a-52f42560e95c service neutron] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 11:01:08.266703 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91f2588e-42f1-419f-859a-52f42560e95c service neutron] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 11:01:08.267002 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91f2588e-42f1-419f-859a-52f42560e95c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 11:01:08.277455 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91f2588e-42f1-419f-859a-52f42560e95c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 11:01:08.278841 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91f2588e-42f1-419f-859a-52f42560e95c service neutron] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 11:01:08.289213 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 1160/2291] 10.222.0.25 () {60 vars in 1365 bytes} [Sun Aug 7 11:01:08 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 07 11:01:10.441165 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-519201fe-d443-4176-8299-c3bbdae7b85d None None] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 11:01:10.553014 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-519201fe-d443-4176-8299-c3bbdae7b85d None None] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 11:01:10.558330 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dae4d84-a6b4-4895-885e-287341eeeb4d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5ac47e3b09f43d19a23a0f457fe9e23', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 11:01:10.559025 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dae4d84-a6b4-4895-885e-287341eeeb4d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 11:01:10.559314 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dae4d84-a6b4-4895-885e-287341eeeb4d admin admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 11:01:10.559592 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dae4d84-a6b4-4895-885e-287341eeeb4d admin admin] PATH_INFO: `/v3/users/72d8a5d8dd3a45fdb8e78e228d2e421e` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 11:01:10.560823 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dae4d84-a6b4-4895-885e-287341eeeb4d admin admin] RBAC: Authorizing `identity:delete_user(user_id=72d8a5d8dd3a45fdb8e78e228d2e421e)` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 11:01:10.562819 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dae4d84-a6b4-4895-885e-287341eeeb4d admin admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 11:01:10.627656 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-5dae4d84-a6b4-4895-885e-287341eeeb4d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '72d8a5d8dd3a45fdb8e78e228d2e421e'} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 11:01:10.635807 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-5dae4d84-a6b4-4895-885e-287341eeeb4d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '72d8a5d8dd3a45fdb8e78e228d2e421e'} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 11:01:10.647540 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-5dae4d84-a6b4-4895-885e-287341eeeb4d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '72d8a5d8dd3a45fdb8e78e228d2e421e'} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 11:01:10.648268 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-5dae4d84-a6b4-4895-885e-287341eeeb4d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '72d8a5d8dd3a45fdb8e78e228d2e421e'} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 11:01:10.656613 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-5dae4d84-a6b4-4895-885e-287341eeeb4d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '72d8a5d8dd3a45fdb8e78e228d2e421e'} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 11:01:10.662800 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1132/2292] 10.222.0.25 () {58 vars in 1224 bytes} [Sun Aug 7 11:01:10 2022] DELETE /identity/v3/users/72d8a5d8dd3a45fdb8e78e228d2e421e => generated 0 bytes in 222 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 07 11:01:10.668534 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0af8a794-9feb-461d-992e-cbdde8cc685a None None] REQUEST_METHOD: `POST` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 11:01:10.669026 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0af8a794-9feb-461d-992e-cbdde8cc685a None None] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 11:01:10.669182 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0af8a794-9feb-461d-992e-cbdde8cc685a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 11:01:10.688362 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.auth.core [None req-0af8a794-9feb-461d-992e-cbdde8cc685a None None] MFA Rules not processed for user `bac4e6b76fe64217ac8fa44e71cffe6a`. Rule list: `[]` (Enabled: `True`). {{(pid=80144) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 11:01:10.689553 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.fernet_utils [None req-0af8a794-9feb-461d-992e-cbdde8cc685a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80144) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 11:01:10.753326 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 1161/2293] 10.222.0.25 () {58 vars in 958 bytes} [Sun Aug 7 11:01:10 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 07 11:01:10.763991 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dae4d84-a6b4-4895-885e-287341eeeb4d admin admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 11:01:10.765498 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-5dae4d84-a6b4-4895-885e-287341eeeb4d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 11:01:10.826644 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dae4d84-a6b4-4895-885e-287341eeeb4d admin admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 11:01:10.830280 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f795e67-f457-46b2-a8f4-2329600b788f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be1283c9bf47424c9aa3ad27cab03027', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6de1f3cac02c4bcdb05dd2337176033d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 11:01:10.830881 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f795e67-f457-46b2-a8f4-2329600b788f service neutron] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 11:01:10.831190 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f795e67-f457-46b2-a8f4-2329600b788f service neutron] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 11:01:10.831462 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f795e67-f457-46b2-a8f4-2329600b788f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 11:01:10.840584 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f795e67-f457-46b2-a8f4-2329600b788f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 11:01:10.841988 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f795e67-f457-46b2-a8f4-2329600b788f service neutron] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 11:01:10.851340 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1133/2294] 10.222.0.25 () {60 vars in 1365 bytes} [Sun Aug 7 11:01:10 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 07 11:01:11.136699 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0af8a794-9feb-461d-992e-cbdde8cc685a None None] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 11:01:11.138179 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.fernet_utils [None req-0af8a794-9feb-461d-992e-cbdde8cc685a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80144) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 11:01:11.149878 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0af8a794-9feb-461d-992e-cbdde8cc685a None None] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 11:01:11.157253 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-472fb970-7d8c-464e-b393-bf19704a6a1e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5ac47e3b09f43d19a23a0f457fe9e23', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 11:01:11.164649 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-472fb970-7d8c-464e-b393-bf19704a6a1e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 11:01:11.165010 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-472fb970-7d8c-464e-b393-bf19704a6a1e admin admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 11:01:11.165440 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-472fb970-7d8c-464e-b393-bf19704a6a1e admin admin] PATH_INFO: `/v3/projects/1bb09f7eb6cc4e44a3b7a3473d9be843` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 11:01:11.167383 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-472fb970-7d8c-464e-b393-bf19704a6a1e admin admin] RBAC: Authorizing `identity:delete_project(project_id=1bb09f7eb6cc4e44a3b7a3473d9be843)` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 11:01:11.170970 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-472fb970-7d8c-464e-b393-bf19704a6a1e admin admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 11:01:11.218740 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.notifications [None req-472fb970-7d8c-464e-b393-bf19704a6a1e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1bb09f7eb6cc4e44a3b7a3473d9be843', 'request_id': 'req-472fb970-7d8c-464e-b393-bf19704a6a1e'} {{(pid=80144) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 11:01:11.224534 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.notifications [None req-472fb970-7d8c-464e-b393-bf19704a6a1e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1bb09f7eb6cc4e44a3b7a3473d9be843', 'request_id': 'req-472fb970-7d8c-464e-b393-bf19704a6a1e'} {{(pid=80144) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 11:01:11.231685 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.notifications [None req-472fb970-7d8c-464e-b393-bf19704a6a1e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1bb09f7eb6cc4e44a3b7a3473d9be843', 'request_id': 'req-472fb970-7d8c-464e-b393-bf19704a6a1e'} {{(pid=80144) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 11:01:11.242741 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.notifications [None req-472fb970-7d8c-464e-b393-bf19704a6a1e admin admin] The token cache is being invalidate because project 1bb09f7eb6cc4e44a3b7a3473d9be843 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80144) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 07 11:01:11.243077 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.notifications [None req-472fb970-7d8c-464e-b393-bf19704a6a1e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80144) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 11:01:11.243706 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.notifications [None req-472fb970-7d8c-464e-b393-bf19704a6a1e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80144) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 11:01:11.245272 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 1162/2295] 10.222.0.25 () {58 vars in 1233 bytes} [Sun Aug 7 11:01:11 2022] DELETE /identity/v3/projects/1bb09f7eb6cc4e44a3b7a3473d9be843 => generated 0 bytes in 109 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 07 11:01:12.142781 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-710ce6fa-24a4-4416-bac4-03253b3a26e9 None None] REQUEST_METHOD: `POST` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 11:01:12.143241 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-710ce6fa-24a4-4416-bac4-03253b3a26e9 None None] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 11:01:12.143377 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-710ce6fa-24a4-4416-bac4-03253b3a26e9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 11:01:12.150812 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 11:01:12.164361 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.auth.core [None req-710ce6fa-24a4-4416-bac4-03253b3a26e9 None None] MFA Rules not processed for user `de1d6c24b1a14d10a20d82a4c65793d8`. Rule list: `[]` (Enabled: `True`). {{(pid=80143) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 11:01:12.165590 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-710ce6fa-24a4-4416-bac4-03253b3a26e9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 11:01:12.220380 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1134/2296] 10.222.0.25 () {58 vars in 967 bytes} [Sun Aug 7 11:01:12 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 78 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 07 11:01:12.267438 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-472fb970-7d8c-464e-b393-bf19704a6a1e admin admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 11:01:12.268823 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.fernet_utils [None req-472fb970-7d8c-464e-b393-bf19704a6a1e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80144) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 11:01:12.291933 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 11:01:12.341102 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-472fb970-7d8c-464e-b393-bf19704a6a1e admin admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 11:01:12.345912 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2bea17c-635a-4adf-98fa-ef15d944c886 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69490ea2a9a340bf91aca38bf5655b45', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6de1f3cac02c4bcdb05dd2337176033d', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 11:01:12.346509 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2bea17c-635a-4adf-98fa-ef15d944c886 service swift] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 11:01:12.346785 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2bea17c-635a-4adf-98fa-ef15d944c886 service swift] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 11:01:12.347110 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2bea17c-635a-4adf-98fa-ef15d944c886 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 11:01:12.355422 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2bea17c-635a-4adf-98fa-ef15d944c886 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 11:01:12.356969 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2bea17c-635a-4adf-98fa-ef15d944c886 service swift] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 11:01:12.364951 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 1163/2297] 10.222.0.25 () {60 vars in 1384 bytes} [Sun Aug 7 11:01:12 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 98 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 07 11:01:12.694002 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-710ce6fa-24a4-4416-bac4-03253b3a26e9 None None] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 11:01:12.695438 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-710ce6fa-24a4-4416-bac4-03253b3a26e9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 11:01:12.755934 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-710ce6fa-24a4-4416-bac4-03253b3a26e9 None None] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 11:01:12.759605 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8671610-73f5-4892-b92e-8a5e7fbbf589 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be1283c9bf47424c9aa3ad27cab03027', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6de1f3cac02c4bcdb05dd2337176033d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 11:01:12.760387 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8671610-73f5-4892-b92e-8a5e7fbbf589 service neutron] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 11:01:12.760688 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8671610-73f5-4892-b92e-8a5e7fbbf589 service neutron] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 11:01:12.760966 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8671610-73f5-4892-b92e-8a5e7fbbf589 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 11:01:12.762741 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-d8671610-73f5-4892-b92e-8a5e7fbbf589 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 11:01:12.828797 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8671610-73f5-4892-b92e-8a5e7fbbf589 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 11:01:12.830234 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8671610-73f5-4892-b92e-8a5e7fbbf589 service neutron] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 11:01:12.840728 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1135/2298] 10.222.0.25 () {60 vars in 1365 bytes} [Sun Aug 7 11:01:12 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 147 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 07 11:01:19.574578 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2bea17c-635a-4adf-98fa-ef15d944c886 service swift] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 11:01:19.576065 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.fernet_utils [None req-e2bea17c-635a-4adf-98fa-ef15d944c886 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80144) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 11:01:19.645100 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8671610-73f5-4892-b92e-8a5e7fbbf589 service neutron] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 11:01:19.651887 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2bea17c-635a-4adf-98fa-ef15d944c886 service swift] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 11:01:19.652569 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-d8671610-73f5-4892-b92e-8a5e7fbbf589 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 11:01:19.657185 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aaab2ed-0a34-4065-9d36-aa42f4d086ff service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6ee2e4c08af04c219ed395d7ca7085a6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6de1f3cac02c4bcdb05dd2337176033d', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 11:01:19.657802 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aaab2ed-0a34-4065-9d36-aa42f4d086ff service placement] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 11:01:19.658084 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aaab2ed-0a34-4065-9d36-aa42f4d086ff service placement] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 11:01:19.658358 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aaab2ed-0a34-4065-9d36-aa42f4d086ff service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 11:01:19.660127 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.fernet_utils [None req-2aaab2ed-0a34-4065-9d36-aa42f4d086ff service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80144) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 11:01:19.664394 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8671610-73f5-4892-b92e-8a5e7fbbf589 service neutron] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 11:01:19.670749 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca7ff47d-e160-4350-9767-9f89d252b1d6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5ac47e3b09f43d19a23a0f457fe9e23', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 11:01:19.671427 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca7ff47d-e160-4350-9767-9f89d252b1d6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 11:01:19.671705 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca7ff47d-e160-4350-9767-9f89d252b1d6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 11:01:19.671978 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca7ff47d-e160-4350-9767-9f89d252b1d6 admin admin] PATH_INFO: `/v3/users/ee9ca3b9c7284fd1a600cbd3add41f56` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 11:01:19.672086 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2aaab2ed-0a34-4065-9d36-aa42f4d086ff service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 11:01:19.673228 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca7ff47d-e160-4350-9767-9f89d252b1d6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ee9ca3b9c7284fd1a600cbd3add41f56)` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 11:01:19.673561 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2aaab2ed-0a34-4065-9d36-aa42f4d086ff service placement] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 11:01:19.675175 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca7ff47d-e160-4350-9767-9f89d252b1d6 admin admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 11:01:19.686604 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 1164/2299] 10.222.0.25 () {60 vars in 1365 bytes} [Sun Aug 7 11:01:19 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 112 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 07 11:01:19.733286 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-ca7ff47d-e160-4350-9767-9f89d252b1d6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ee9ca3b9c7284fd1a600cbd3add41f56'} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 11:01:19.738555 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=93,keystone:DELETE=7 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 11:01:19.741760 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-ca7ff47d-e160-4350-9767-9f89d252b1d6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ee9ca3b9c7284fd1a600cbd3add41f56'} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 11:01:19.758041 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-ca7ff47d-e160-4350-9767-9f89d252b1d6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ee9ca3b9c7284fd1a600cbd3add41f56'} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 11:01:19.758760 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-ca7ff47d-e160-4350-9767-9f89d252b1d6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ee9ca3b9c7284fd1a600cbd3add41f56'} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 11:01:19.767999 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-ca7ff47d-e160-4350-9767-9f89d252b1d6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ee9ca3b9c7284fd1a600cbd3add41f56'} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 11:01:19.774437 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1136/2300] 10.222.0.25 () {58 vars in 1224 bytes} [Sun Aug 7 11:01:19 2022] DELETE /identity/v3/users/ee9ca3b9c7284fd1a600cbd3add41f56 => generated 0 bytes in 130 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 07 11:01:19.959801 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aaab2ed-0a34-4065-9d36-aa42f4d086ff service placement] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 11:01:19.970913 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aaab2ed-0a34-4065-9d36-aa42f4d086ff service placement] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 11:01:20.030300 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e2367be-5c29-4e89-86e1-c7071096010d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5ac47e3b09f43d19a23a0f457fe9e23', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 11:01:20.030834 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e2367be-5c29-4e89-86e1-c7071096010d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 11:01:20.031140 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e2367be-5c29-4e89-86e1-c7071096010d admin admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 11:01:20.031444 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e2367be-5c29-4e89-86e1-c7071096010d admin admin] PATH_INFO: `/v3/projects/112b9a6f7ead454695e7efd26c0d845b` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 11:01:20.032749 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e2367be-5c29-4e89-86e1-c7071096010d admin admin] RBAC: Authorizing `identity:delete_project(project_id=112b9a6f7ead454695e7efd26c0d845b)` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 11:01:20.034649 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e2367be-5c29-4e89-86e1-c7071096010d admin admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 11:01:20.075209 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=97,keystone:DELETE=3 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 11:01:20.083103 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.notifications [None req-6e2367be-5c29-4e89-86e1-c7071096010d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '112b9a6f7ead454695e7efd26c0d845b', 'request_id': 'req-6e2367be-5c29-4e89-86e1-c7071096010d'} {{(pid=80144) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 11:01:20.089085 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.notifications [None req-6e2367be-5c29-4e89-86e1-c7071096010d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '112b9a6f7ead454695e7efd26c0d845b', 'request_id': 'req-6e2367be-5c29-4e89-86e1-c7071096010d'} {{(pid=80144) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 11:01:20.096268 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.notifications [None req-6e2367be-5c29-4e89-86e1-c7071096010d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '112b9a6f7ead454695e7efd26c0d845b', 'request_id': 'req-6e2367be-5c29-4e89-86e1-c7071096010d'} {{(pid=80144) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 11:01:20.108269 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.notifications [None req-6e2367be-5c29-4e89-86e1-c7071096010d admin admin] The token cache is being invalidate because project 112b9a6f7ead454695e7efd26c0d845b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80144) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 07 11:01:20.108568 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.notifications [None req-6e2367be-5c29-4e89-86e1-c7071096010d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80144) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 11:01:20.109196 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.notifications [None req-6e2367be-5c29-4e89-86e1-c7071096010d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80144) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 11:01:20.110659 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 1165/2301] 10.222.0.25 () {58 vars in 1233 bytes} [Sun Aug 7 11:01:19 2022] DELETE /identity/v3/projects/112b9a6f7ead454695e7efd26c0d845b => generated 0 bytes in 151 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 07 11:01:20.114702 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca7ff47d-e160-4350-9767-9f89d252b1d6 admin admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 11:01:20.116447 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-ca7ff47d-e160-4350-9767-9f89d252b1d6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 11:01:20.129978 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca7ff47d-e160-4350-9767-9f89d252b1d6 admin admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 11:01:20.190713 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f9133cf-3610-42ba-8fec-54b9f34bdf17 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5ac47e3b09f43d19a23a0f457fe9e23', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 11:01:20.191500 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f9133cf-3610-42ba-8fec-54b9f34bdf17 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 11:01:20.191771 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f9133cf-3610-42ba-8fec-54b9f34bdf17 admin admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 11:01:20.192060 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f9133cf-3610-42ba-8fec-54b9f34bdf17 admin admin] PATH_INFO: `/v3/users/a0be3da3bf484da8b95fa985e43df9f5` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 11:01:20.195049 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f9133cf-3610-42ba-8fec-54b9f34bdf17 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a0be3da3bf484da8b95fa985e43df9f5)` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 11:01:20.197962 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f9133cf-3610-42ba-8fec-54b9f34bdf17 admin admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 11:01:20.249845 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-3f9133cf-3610-42ba-8fec-54b9f34bdf17 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a0be3da3bf484da8b95fa985e43df9f5'} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 11:01:20.258313 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-3f9133cf-3610-42ba-8fec-54b9f34bdf17 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a0be3da3bf484da8b95fa985e43df9f5'} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 11:01:20.270422 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-3f9133cf-3610-42ba-8fec-54b9f34bdf17 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a0be3da3bf484da8b95fa985e43df9f5'} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 11:01:20.271156 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-3f9133cf-3610-42ba-8fec-54b9f34bdf17 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a0be3da3bf484da8b95fa985e43df9f5'} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 11:01:20.279816 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-3f9133cf-3610-42ba-8fec-54b9f34bdf17 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a0be3da3bf484da8b95fa985e43df9f5'} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 11:01:20.285721 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1137/2302] 10.222.0.25 () {58 vars in 1224 bytes} [Sun Aug 7 11:01:20 2022] DELETE /identity/v3/users/a0be3da3bf484da8b95fa985e43df9f5 => generated 0 bytes in 171 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 07 11:01:20.460066 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e2367be-5c29-4e89-86e1-c7071096010d admin admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 11:01:20.461910 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.fernet_utils [None req-6e2367be-5c29-4e89-86e1-c7071096010d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80144) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 11:01:20.481061 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e2367be-5c29-4e89-86e1-c7071096010d admin admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 11:01:20.545264 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b85c4757-8ace-4f8a-b8e6-310a13771e94 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5ac47e3b09f43d19a23a0f457fe9e23', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 11:01:20.546451 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b85c4757-8ace-4f8a-b8e6-310a13771e94 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 11:01:20.546728 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b85c4757-8ace-4f8a-b8e6-310a13771e94 admin admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 11:01:20.547050 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b85c4757-8ace-4f8a-b8e6-310a13771e94 admin admin] PATH_INFO: `/v3/projects/7950a7f5cbac420b992ad97be30314cf` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 11:01:20.548277 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b85c4757-8ace-4f8a-b8e6-310a13771e94 admin admin] RBAC: Authorizing `identity:delete_project(project_id=7950a7f5cbac420b992ad97be30314cf)` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 11:01:20.550167 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b85c4757-8ace-4f8a-b8e6-310a13771e94 admin admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 11:01:20.600474 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.notifications [None req-b85c4757-8ace-4f8a-b8e6-310a13771e94 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7950a7f5cbac420b992ad97be30314cf', 'request_id': 'req-b85c4757-8ace-4f8a-b8e6-310a13771e94'} {{(pid=80144) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 11:01:20.606525 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.notifications [None req-b85c4757-8ace-4f8a-b8e6-310a13771e94 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7950a7f5cbac420b992ad97be30314cf', 'request_id': 'req-b85c4757-8ace-4f8a-b8e6-310a13771e94'} {{(pid=80144) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 11:01:20.615073 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.notifications [None req-b85c4757-8ace-4f8a-b8e6-310a13771e94 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7950a7f5cbac420b992ad97be30314cf', 'request_id': 'req-b85c4757-8ace-4f8a-b8e6-310a13771e94'} {{(pid=80144) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 11:01:20.627481 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.notifications [None req-b85c4757-8ace-4f8a-b8e6-310a13771e94 admin admin] The token cache is being invalidate because project 7950a7f5cbac420b992ad97be30314cf was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80144) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 07 11:01:20.627805 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.notifications [None req-b85c4757-8ace-4f8a-b8e6-310a13771e94 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80144) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 11:01:20.628500 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.notifications [None req-b85c4757-8ace-4f8a-b8e6-310a13771e94 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80144) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 11:01:20.630226 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 1166/2303] 10.222.0.25 () {58 vars in 1233 bytes} [Sun Aug 7 11:01:20 2022] DELETE /identity/v3/projects/7950a7f5cbac420b992ad97be30314cf => generated 0 bytes in 171 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 07 11:01:29.062480 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdaa068a-b175-4a78-a1df-d85eb896a03a None None] REQUEST_METHOD: `POST` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 11:01:29.063010 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdaa068a-b175-4a78-a1df-d85eb896a03a None None] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 11:01:29.063113 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdaa068a-b175-4a78-a1df-d85eb896a03a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 11:01:29.082260 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.auth.core [None req-cdaa068a-b175-4a78-a1df-d85eb896a03a None None] MFA Rules not processed for user `2b757e3f47e44b9ba3645d12b20f7502`. Rule list: `[]` (Enabled: `True`). {{(pid=80143) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 11:01:29.083486 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-cdaa068a-b175-4a78-a1df-d85eb896a03a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 11:01:29.127684 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=85,keystone:DELETE=13,keystone:INSERT=2 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 11:01:29.140183 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1138/2304] 10.222.0.25 () {58 vars in 958 bytes} [Sun Aug 7 11:01:29 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 07 11:01:30.624390 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=60,keystone:DELETE=11,keystone:INSERT=2 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 11:01:39.137440 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=3 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 11:01:50.469056 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b85c4757-8ace-4f8a-b8e6-310a13771e94 admin admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 11:01:50.470690 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.fernet_utils [None req-b85c4757-8ace-4f8a-b8e6-310a13771e94 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80144) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 11:01:50.546997 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b85c4757-8ace-4f8a-b8e6-310a13771e94 admin admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 11:01:50.553879 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39e64f62-fdd4-4bb5-9921-0e2e5f22c36f service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31d16d1b96104b4eb690398432b9829e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6de1f3cac02c4bcdb05dd2337176033d', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'reader', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 11:01:50.554584 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39e64f62-fdd4-4bb5-9921-0e2e5f22c36f service nova] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 11:01:50.554866 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39e64f62-fdd4-4bb5-9921-0e2e5f22c36f service nova] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 11:01:50.555166 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39e64f62-fdd4-4bb5-9921-0e2e5f22c36f service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 11:01:50.557198 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.fernet_utils [None req-39e64f62-fdd4-4bb5-9921-0e2e5f22c36f service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80144) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 11:01:50.625860 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39e64f62-fdd4-4bb5-9921-0e2e5f22c36f service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 11:01:50.627479 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39e64f62-fdd4-4bb5-9921-0e2e5f22c36f service nova] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 11:01:50.636552 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 1167/2305] 10.222.0.25 () {60 vars in 1365 bytes} [Sun Aug 7 11:01:50 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 168 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 07 11:02:00.632297 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=56 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 11:02:03.817469 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7784400-892f-4eac-b57b-a8376a342fec None None] REQUEST_METHOD: `POST` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 11:02:03.817932 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7784400-892f-4eac-b57b-a8376a342fec None None] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 11:02:03.818036 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7784400-892f-4eac-b57b-a8376a342fec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 11:02:03.837701 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.auth.core [None req-c7784400-892f-4eac-b57b-a8376a342fec None None] MFA Rules not processed for user `2b64b05230b940eba992a063ea7cd778`. Rule list: `[]` (Enabled: `True`). {{(pid=80143) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 11:02:03.838914 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-c7784400-892f-4eac-b57b-a8376a342fec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 11:02:03.899414 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1139/2306] 10.222.0.25 () {58 vars in 958 bytes} [Sun Aug 7 11:02:03 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 07 11:02:11.569876 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6ee6690-8179-469d-9292-bc0e57d3af2c None None] REQUEST_METHOD: `POST` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 11:02:11.570402 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6ee6690-8179-469d-9292-bc0e57d3af2c None None] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 11:02:11.570500 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6ee6690-8179-469d-9292-bc0e57d3af2c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 11:02:11.601189 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.auth.core [None req-f6ee6690-8179-469d-9292-bc0e57d3af2c None None] MFA Rules not processed for user `06231f178e0d494084915b9368fa3b9b`. Rule list: `[]` (Enabled: `True`). {{(pid=80144) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 11:02:11.602394 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.fernet_utils [None req-f6ee6690-8179-469d-9292-bc0e57d3af2c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80144) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 11:02:11.629932 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 1168/2307] 10.222.0.25 () {58 vars in 958 bytes} [Sun Aug 7 11:02:11 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 61 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 07 11:02:13.892399 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=25 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 11:02:14.929299 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7784400-892f-4eac-b57b-a8376a342fec None None] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 11:02:14.930874 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-c7784400-892f-4eac-b57b-a8376a342fec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 11:02:15.019505 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7784400-892f-4eac-b57b-a8376a342fec None None] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 11:02:15.023107 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c698ed0-3592-4f5e-95b4-a7cd2350113c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be1283c9bf47424c9aa3ad27cab03027', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6de1f3cac02c4bcdb05dd2337176033d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 11:02:15.023710 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c698ed0-3592-4f5e-95b4-a7cd2350113c service neutron] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 11:02:15.024010 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c698ed0-3592-4f5e-95b4-a7cd2350113c service neutron] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 11:02:15.024265 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c698ed0-3592-4f5e-95b4-a7cd2350113c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 11:02:15.026015 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-6c698ed0-3592-4f5e-95b4-a7cd2350113c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 11:02:15.116191 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c698ed0-3592-4f5e-95b4-a7cd2350113c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 11:02:15.117608 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c698ed0-3592-4f5e-95b4-a7cd2350113c service neutron] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 11:02:15.137246 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1140/2308] 10.222.0.25 () {60 vars in 1365 bytes} [Sun Aug 7 11:02:14 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 209 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 07 11:02:18.529463 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6ee6690-8179-469d-9292-bc0e57d3af2c None None] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 11:02:18.531029 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.fernet_utils [None req-f6ee6690-8179-469d-9292-bc0e57d3af2c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80144) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 11:02:18.545757 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6ee6690-8179-469d-9292-bc0e57d3af2c None None] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 11:02:18.550961 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ce46f39-e46d-4764-a251-b653d6f5a646 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5ac47e3b09f43d19a23a0f457fe9e23', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 11:02:18.551439 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ce46f39-e46d-4764-a251-b653d6f5a646 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 11:02:18.551903 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ce46f39-e46d-4764-a251-b653d6f5a646 admin admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 11:02:18.552175 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ce46f39-e46d-4764-a251-b653d6f5a646 admin admin] PATH_INFO: `/v3/users/c3a12c3144844d32812e010550bb0921` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 11:02:18.553298 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ce46f39-e46d-4764-a251-b653d6f5a646 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c3a12c3144844d32812e010550bb0921)` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 11:02:18.555263 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ce46f39-e46d-4764-a251-b653d6f5a646 admin admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 11:02:18.618028 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.notifications [None req-5ce46f39-e46d-4764-a251-b653d6f5a646 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c3a12c3144844d32812e010550bb0921'} {{(pid=80144) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 11:02:18.625888 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.notifications [None req-5ce46f39-e46d-4764-a251-b653d6f5a646 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c3a12c3144844d32812e010550bb0921'} {{(pid=80144) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 11:02:18.635177 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.notifications [None req-5ce46f39-e46d-4764-a251-b653d6f5a646 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c3a12c3144844d32812e010550bb0921'} {{(pid=80144) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 11:02:18.636494 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.notifications [None req-5ce46f39-e46d-4764-a251-b653d6f5a646 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c3a12c3144844d32812e010550bb0921'} {{(pid=80144) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 11:02:18.637893 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.notifications [None req-5ce46f39-e46d-4764-a251-b653d6f5a646 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c3a12c3144844d32812e010550bb0921'} {{(pid=80144) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 11:02:18.669594 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 1169/2309] 10.222.0.25 () {58 vars in 1224 bytes} [Sun Aug 7 11:02:18 2022] DELETE /identity/v3/users/c3a12c3144844d32812e010550bb0921 => generated 0 bytes in 141 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 07 11:02:18.867116 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c698ed0-3592-4f5e-95b4-a7cd2350113c service neutron] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 11:02:18.883934 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c698ed0-3592-4f5e-95b4-a7cd2350113c service neutron] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 11:02:18.935687 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=100 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 11:02:18.951393 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f431b2da-212a-41ba-9b80-ff9ed6d98797 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5ac47e3b09f43d19a23a0f457fe9e23', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 11:02:18.952060 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f431b2da-212a-41ba-9b80-ff9ed6d98797 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 11:02:18.952337 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f431b2da-212a-41ba-9b80-ff9ed6d98797 admin admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 11:02:18.952609 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f431b2da-212a-41ba-9b80-ff9ed6d98797 admin admin] PATH_INFO: `/v3/projects/9df6ca365836428198f2d20fd9829feb` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 11:02:18.953721 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f431b2da-212a-41ba-9b80-ff9ed6d98797 admin admin] RBAC: Authorizing `identity:delete_project(project_id=9df6ca365836428198f2d20fd9829feb)` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 11:02:18.955697 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f431b2da-212a-41ba-9b80-ff9ed6d98797 admin admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 11:02:19.004197 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-f431b2da-212a-41ba-9b80-ff9ed6d98797 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9df6ca365836428198f2d20fd9829feb', 'request_id': 'req-f431b2da-212a-41ba-9b80-ff9ed6d98797'} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 11:02:19.010949 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-f431b2da-212a-41ba-9b80-ff9ed6d98797 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9df6ca365836428198f2d20fd9829feb', 'request_id': 'req-f431b2da-212a-41ba-9b80-ff9ed6d98797'} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 11:02:19.017447 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-f431b2da-212a-41ba-9b80-ff9ed6d98797 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9df6ca365836428198f2d20fd9829feb', 'request_id': 'req-f431b2da-212a-41ba-9b80-ff9ed6d98797'} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 11:02:19.031770 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-f431b2da-212a-41ba-9b80-ff9ed6d98797 admin admin] The token cache is being invalidate because project 9df6ca365836428198f2d20fd9829feb was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80143) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 07 11:02:19.032077 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-f431b2da-212a-41ba-9b80-ff9ed6d98797 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 11:02:19.032701 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-f431b2da-212a-41ba-9b80-ff9ed6d98797 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 11:02:19.034197 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1141/2310] 10.222.0.25 () {58 vars in 1233 bytes} [Sun Aug 7 11:02:18 2022] DELETE /identity/v3/projects/9df6ca365836428198f2d20fd9829feb => generated 0 bytes in 168 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 07 11:02:19.039223 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d594270e-223b-4dc3-927f-02a3042d5ccb None None] REQUEST_METHOD: `POST` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 11:02:19.039511 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d594270e-223b-4dc3-927f-02a3042d5ccb None None] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 11:02:19.039789 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d594270e-223b-4dc3-927f-02a3042d5ccb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 11:02:19.099294 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.auth.core [None req-d594270e-223b-4dc3-927f-02a3042d5ccb None None] MFA Rules not processed for user `bac4e6b76fe64217ac8fa44e71cffe6a`. Rule list: `[]` (Enabled: `True`). {{(pid=80144) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 11:02:19.100521 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.fernet_utils [None req-d594270e-223b-4dc3-927f-02a3042d5ccb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80144) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 11:02:19.164873 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 1170/2311] 10.222.0.25 () {58 vars in 958 bytes} [Sun Aug 7 11:02:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 07 11:02:19.173925 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68739649-e5f6-4865-8faf-5c87475462e6 None None] REQUEST_METHOD: `POST` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 11:02:19.174209 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68739649-e5f6-4865-8faf-5c87475462e6 None None] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 11:02:19.174485 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68739649-e5f6-4865-8faf-5c87475462e6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 11:02:19.195710 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.auth.core [None req-68739649-e5f6-4865-8faf-5c87475462e6 None None] MFA Rules not processed for user `bac4e6b76fe64217ac8fa44e71cffe6a`. Rule list: `[]` (Enabled: `True`). {{(pid=80143) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 11:02:19.197421 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-68739649-e5f6-4865-8faf-5c87475462e6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 11:02:19.206253 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1142/2312] 10.222.0.25 () {58 vars in 958 bytes} [Sun Aug 7 11:02:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 07 11:02:19.210582 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d594270e-223b-4dc3-927f-02a3042d5ccb None None] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 11:02:19.221121 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d594270e-223b-4dc3-927f-02a3042d5ccb None None] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 11:02:19.225807 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c6227d6-d620-4585-8b11-9db7f4f98d19 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5ac47e3b09f43d19a23a0f457fe9e23', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 11:02:19.226416 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c6227d6-d620-4585-8b11-9db7f4f98d19 admin admin] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 11:02:19.226697 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c6227d6-d620-4585-8b11-9db7f4f98d19 admin admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 11:02:19.227006 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c6227d6-d620-4585-8b11-9db7f4f98d19 admin admin] PATH_INFO: `/v3/domains` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 11:02:19.227653 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c6227d6-d620-4585-8b11-9db7f4f98d19 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80144) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 11:02:19.227956 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c6227d6-d620-4585-8b11-9db7f4f98d19 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 11:02:19.230275 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c6227d6-d620-4585-8b11-9db7f4f98d19 admin admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 11:02:19.233883 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 11:02:19.245127 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 1171/2313] 10.222.0.25 () {58 vars in 1153 bytes} [Sun Aug 7 11:02:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 11:02:19.249191 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68739649-e5f6-4865-8faf-5c87475462e6 None None] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 11:02:19.258850 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68739649-e5f6-4865-8faf-5c87475462e6 None None] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 11:02:19.264464 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-608ceb84-2e87-4dd4-88e3-e45e286188a1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5ac47e3b09f43d19a23a0f457fe9e23', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 11:02:19.265086 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-608ceb84-2e87-4dd4-88e3-e45e286188a1 admin admin] REQUEST_METHOD: `POST` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 11:02:19.265370 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-608ceb84-2e87-4dd4-88e3-e45e286188a1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 11:02:19.265646 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-608ceb84-2e87-4dd4-88e3-e45e286188a1 admin admin] PATH_INFO: `/v3/projects` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 11:02:19.266309 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-608ceb84-2e87-4dd4-88e3-e45e286188a1 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 11:02:19.268276 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-608ceb84-2e87-4dd4-88e3-e45e286188a1 admin admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 11:02:19.301729 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1143/2314] 10.222.0.25 () {60 vars in 1153 bytes} [Sun Aug 7 11:02:19 2022] POST /identity/v3/projects => generated 365 bytes in 53 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 07 11:02:19.306234 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c6227d6-d620-4585-8b11-9db7f4f98d19 admin admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 11:02:19.317038 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c6227d6-d620-4585-8b11-9db7f4f98d19 admin admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 11:02:19.387025 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2718bf6-7e92-40d1-8b6f-33699f5db78f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5ac47e3b09f43d19a23a0f457fe9e23', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 11:02:19.387594 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2718bf6-7e92-40d1-8b6f-33699f5db78f admin admin] REQUEST_METHOD: `POST` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 11:02:19.387861 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2718bf6-7e92-40d1-8b6f-33699f5db78f admin admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 11:02:19.388137 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2718bf6-7e92-40d1-8b6f-33699f5db78f admin admin] PATH_INFO: `/v3/users` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 11:02:19.388763 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2718bf6-7e92-40d1-8b6f-33699f5db78f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 11:02:19.390917 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2718bf6-7e92-40d1-8b6f-33699f5db78f admin admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 11:02:19.429637 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 1172/2315] 10.222.0.25 () {60 vars in 1144 bytes} [Sun Aug 7 11:02:19 2022] POST /identity/v3/users => generated 283 bytes in 124 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 07 11:02:19.434244 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-608ceb84-2e87-4dd4-88e3-e45e286188a1 admin admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 11:02:19.445521 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-608ceb84-2e87-4dd4-88e3-e45e286188a1 admin admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 11:02:19.452001 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a0bb9e5-ba40-4d9e-a146-141d46f427a4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5ac47e3b09f43d19a23a0f457fe9e23', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 11:02:19.452624 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a0bb9e5-ba40-4d9e-a146-141d46f427a4 admin admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 11:02:19.452903 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a0bb9e5-ba40-4d9e-a146-141d46f427a4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 11:02:19.453178 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a0bb9e5-ba40-4d9e-a146-141d46f427a4 admin admin] PATH_INFO: `/v3/roles` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 11:02:19.453830 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a0bb9e5-ba40-4d9e-a146-141d46f427a4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 11:02:19.455677 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a0bb9e5-ba40-4d9e-a146-141d46f427a4 admin admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 11:02:19.472095 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2718bf6-7e92-40d1-8b6f-33699f5db78f admin admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 11:02:19.478780 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1144/2316] 10.222.0.25 () {58 vars in 1122 bytes} [Sun Aug 7 11:02:19 2022] GET /identity/v3/roles => generated 1420 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 07 11:02:19.483498 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2718bf6-7e92-40d1-8b6f-33699f5db78f admin admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 11:02:19.488547 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a5df9a1-76ba-438c-ada1-eefd379594d1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5ac47e3b09f43d19a23a0f457fe9e23', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 11:02:19.489165 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a5df9a1-76ba-438c-ada1-eefd379594d1 admin admin] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 11:02:19.489445 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a5df9a1-76ba-438c-ada1-eefd379594d1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 11:02:19.489717 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a5df9a1-76ba-438c-ada1-eefd379594d1 admin admin] PATH_INFO: `/v3/roles` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 11:02:19.490359 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a5df9a1-76ba-438c-ada1-eefd379594d1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 11:02:19.492233 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a5df9a1-76ba-438c-ada1-eefd379594d1 admin admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 11:02:19.514948 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 1173/2317] 10.222.0.25 () {58 vars in 1122 bytes} [Sun Aug 7 11:02:19 2022] GET /identity/v3/roles => generated 1420 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 07 11:02:19.519175 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a0bb9e5-ba40-4d9e-a146-141d46f427a4 admin admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 11:02:19.530576 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a0bb9e5-ba40-4d9e-a146-141d46f427a4 admin admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 11:02:19.536327 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8497d7cc-e941-4182-8d23-3949c968f3cb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5ac47e3b09f43d19a23a0f457fe9e23', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 11:02:19.536823 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8497d7cc-e941-4182-8d23-3949c968f3cb admin admin] REQUEST_METHOD: `PUT` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 11:02:19.537100 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8497d7cc-e941-4182-8d23-3949c968f3cb admin admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 11:02:19.537375 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8497d7cc-e941-4182-8d23-3949c968f3cb admin admin] PATH_INFO: `/v3/projects/494491475f5442a3b25e74255bb79146/users/cfe75789ca644031be04bd71b30623f6/roles/b98dfb3b9fee4644af2887a3ac3fd2e6` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 11:02:19.554026 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8497d7cc-e941-4182-8d23-3949c968f3cb admin admin] RBAC: Authorizing `identity:create_grant(project_id=494491475f5442a3b25e74255bb79146, user_id=cfe75789ca644031be04bd71b30623f6, role_id=b98dfb3b9fee4644af2887a3ac3fd2e6)` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 11:02:19.557125 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8497d7cc-e941-4182-8d23-3949c968f3cb admin admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 11:02:19.575402 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1145/2318] 10.222.0.25 () {60 vars in 1483 bytes} [Sun Aug 7 11:02:19 2022] PUT /identity/v3/projects/494491475f5442a3b25e74255bb79146/users/cfe75789ca644031be04bd71b30623f6/roles/b98dfb3b9fee4644af2887a3ac3fd2e6 => generated 0 bytes in 57 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 07 11:02:19.586748 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e19c685-391e-4a08-abd7-0560f0bbc5b2 None None] REQUEST_METHOD: `POST` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 11:02:19.587330 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e19c685-391e-4a08-abd7-0560f0bbc5b2 None None] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 11:02:19.587330 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e19c685-391e-4a08-abd7-0560f0bbc5b2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 11:02:19.617471 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.auth.core [None req-3e19c685-391e-4a08-abd7-0560f0bbc5b2 None None] MFA Rules not processed for user `cfe75789ca644031be04bd71b30623f6`. Rule list: `[]` (Enabled: `True`). {{(pid=80144) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 11:02:19.618941 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.fernet_utils [None req-3e19c685-391e-4a08-abd7-0560f0bbc5b2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80144) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 11:02:19.688530 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 1174/2319] 10.222.0.25 () {58 vars in 958 bytes} [Sun Aug 7 11:02:19 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 07 11:02:19.699734 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8497d7cc-e941-4182-8d23-3949c968f3cb admin admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 11:02:19.701690 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-8497d7cc-e941-4182-8d23-3949c968f3cb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 11:02:19.770586 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8497d7cc-e941-4182-8d23-3949c968f3cb admin admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 11:02:19.775063 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-756dc3e8-71f4-4a67-a604-edd743a0daec service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88dbd6dfd1164994a3e16da553561910', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6de1f3cac02c4bcdb05dd2337176033d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 11:02:19.775685 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-756dc3e8-71f4-4a67-a604-edd743a0daec service cinder] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 11:02:19.775969 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-756dc3e8-71f4-4a67-a604-edd743a0daec service cinder] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 11:02:19.776361 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-756dc3e8-71f4-4a67-a604-edd743a0daec service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 11:02:19.785418 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-756dc3e8-71f4-4a67-a604-edd743a0daec service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 11:02:19.786869 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-756dc3e8-71f4-4a67-a604-edd743a0daec service cinder] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 11:02:19.797930 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1146/2320] 10.222.0.25 () {60 vars in 1365 bytes} [Sun Aug 7 11:02:19 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 99 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 07 11:02:28.190826 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e19c685-391e-4a08-abd7-0560f0bbc5b2 None None] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 11:02:28.257289 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e19c685-391e-4a08-abd7-0560f0bbc5b2 None None] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 11:02:28.261868 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a26e37c-25e8-4ea5-973b-824a3bd1129f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5ac47e3b09f43d19a23a0f457fe9e23', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 11:02:28.262497 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a26e37c-25e8-4ea5-973b-824a3bd1129f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 11:02:28.262780 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a26e37c-25e8-4ea5-973b-824a3bd1129f admin admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 11:02:28.263081 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a26e37c-25e8-4ea5-973b-824a3bd1129f admin admin] PATH_INFO: `/v3/users/cfe75789ca644031be04bd71b30623f6` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 11:02:28.264203 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a26e37c-25e8-4ea5-973b-824a3bd1129f admin admin] RBAC: Authorizing `identity:delete_user(user_id=cfe75789ca644031be04bd71b30623f6)` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 11:02:28.266093 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a26e37c-25e8-4ea5-973b-824a3bd1129f admin admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 11:02:28.270816 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 11:02:28.313981 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.notifications [None req-1a26e37c-25e8-4ea5-973b-824a3bd1129f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'cfe75789ca644031be04bd71b30623f6'} {{(pid=80144) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 11:02:28.320841 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.notifications [None req-1a26e37c-25e8-4ea5-973b-824a3bd1129f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'cfe75789ca644031be04bd71b30623f6'} {{(pid=80144) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 11:02:28.328741 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.notifications [None req-1a26e37c-25e8-4ea5-973b-824a3bd1129f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'cfe75789ca644031be04bd71b30623f6'} {{(pid=80144) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 11:02:28.329462 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.notifications [None req-1a26e37c-25e8-4ea5-973b-824a3bd1129f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'cfe75789ca644031be04bd71b30623f6'} {{(pid=80144) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 11:02:28.330376 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.notifications [None req-1a26e37c-25e8-4ea5-973b-824a3bd1129f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'cfe75789ca644031be04bd71b30623f6'} {{(pid=80144) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 11:02:28.350689 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 1175/2321] 10.222.0.25 () {58 vars in 1224 bytes} [Sun Aug 7 11:02:28 2022] DELETE /identity/v3/users/cfe75789ca644031be04bd71b30623f6 => generated 0 bytes in 160 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 07 11:02:28.355907 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e5084c3-bf10-4c77-9044-d6a27247ea6e None None] REQUEST_METHOD: `POST` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 11:02:28.356387 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e5084c3-bf10-4c77-9044-d6a27247ea6e None None] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 11:02:28.356496 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e5084c3-bf10-4c77-9044-d6a27247ea6e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 11:02:28.375045 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.auth.core [None req-8e5084c3-bf10-4c77-9044-d6a27247ea6e None None] MFA Rules not processed for user `bac4e6b76fe64217ac8fa44e71cffe6a`. Rule list: `[]` (Enabled: `True`). {{(pid=80143) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 11:02:28.376291 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-8e5084c3-bf10-4c77-9044-d6a27247ea6e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 11:02:28.385312 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 11:02:28.441387 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1147/2322] 10.222.0.25 () {58 vars in 958 bytes} [Sun Aug 7 11:02:28 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 07 11:02:28.452058 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a26e37c-25e8-4ea5-973b-824a3bd1129f admin admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 11:02:28.453467 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.fernet_utils [None req-1a26e37c-25e8-4ea5-973b-824a3bd1129f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80144) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 11:02:28.512487 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a26e37c-25e8-4ea5-973b-824a3bd1129f admin admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 11:02:28.516148 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b7c2b77-9a00-4637-81cc-5c8d18ddb0e9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be1283c9bf47424c9aa3ad27cab03027', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6de1f3cac02c4bcdb05dd2337176033d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 11:02:28.516754 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b7c2b77-9a00-4637-81cc-5c8d18ddb0e9 service neutron] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 11:02:28.517032 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b7c2b77-9a00-4637-81cc-5c8d18ddb0e9 service neutron] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 11:02:28.517311 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b7c2b77-9a00-4637-81cc-5c8d18ddb0e9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 11:02:28.527622 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b7c2b77-9a00-4637-81cc-5c8d18ddb0e9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 11:02:28.529048 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b7c2b77-9a00-4637-81cc-5c8d18ddb0e9 service neutron] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 11:02:28.539263 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 1176/2323] 10.222.0.25 () {60 vars in 1365 bytes} [Sun Aug 7 11:02:28 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 07 11:02:28.812300 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e5084c3-bf10-4c77-9044-d6a27247ea6e None None] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 11:02:28.814569 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-8e5084c3-bf10-4c77-9044-d6a27247ea6e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 11:02:28.828127 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e5084c3-bf10-4c77-9044-d6a27247ea6e None None] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 11:02:28.832841 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b1ab9d2-86c4-401e-bd4a-ce3acb05e5bc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5ac47e3b09f43d19a23a0f457fe9e23', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 11:02:28.833511 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b1ab9d2-86c4-401e-bd4a-ce3acb05e5bc admin admin] REQUEST_METHOD: `DELETE` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 11:02:28.833797 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b1ab9d2-86c4-401e-bd4a-ce3acb05e5bc admin admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 11:02:28.834080 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b1ab9d2-86c4-401e-bd4a-ce3acb05e5bc admin admin] PATH_INFO: `/v3/projects/494491475f5442a3b25e74255bb79146` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 11:02:28.835322 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b1ab9d2-86c4-401e-bd4a-ce3acb05e5bc admin admin] RBAC: Authorizing `identity:delete_project(project_id=494491475f5442a3b25e74255bb79146)` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 11:02:28.837247 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b1ab9d2-86c4-401e-bd4a-ce3acb05e5bc admin admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 11:02:28.889870 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-2b1ab9d2-86c4-401e-bd4a-ce3acb05e5bc admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '494491475f5442a3b25e74255bb79146', 'request_id': 'req-2b1ab9d2-86c4-401e-bd4a-ce3acb05e5bc'} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 11:02:28.895307 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-2b1ab9d2-86c4-401e-bd4a-ce3acb05e5bc admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '494491475f5442a3b25e74255bb79146', 'request_id': 'req-2b1ab9d2-86c4-401e-bd4a-ce3acb05e5bc'} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 11:02:28.901386 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-2b1ab9d2-86c4-401e-bd4a-ce3acb05e5bc admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '494491475f5442a3b25e74255bb79146', 'request_id': 'req-2b1ab9d2-86c4-401e-bd4a-ce3acb05e5bc'} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 11:02:28.913314 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-2b1ab9d2-86c4-401e-bd4a-ce3acb05e5bc admin admin] The token cache is being invalidate because project 494491475f5442a3b25e74255bb79146 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80143) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 07 11:02:28.913609 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-2b1ab9d2-86c4-401e-bd4a-ce3acb05e5bc admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 11:02:28.914231 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-2b1ab9d2-86c4-401e-bd4a-ce3acb05e5bc admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 11:02:28.915745 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1148/2324] 10.222.0.25 () {58 vars in 1233 bytes} [Sun Aug 7 11:02:28 2022] DELETE /identity/v3/projects/494491475f5442a3b25e74255bb79146 => generated 0 bytes in 105 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 07 11:02:28.920966 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09daec80-ca9b-4b4e-85f1-512f82baf219 None None] REQUEST_METHOD: `POST` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 11:02:28.921250 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09daec80-ca9b-4b4e-85f1-512f82baf219 None None] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 11:02:28.921525 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09daec80-ca9b-4b4e-85f1-512f82baf219 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 11:02:28.941465 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.auth.core [None req-09daec80-ca9b-4b4e-85f1-512f82baf219 None None] MFA Rules not processed for user `bac4e6b76fe64217ac8fa44e71cffe6a`. Rule list: `[]` (Enabled: `True`). {{(pid=80144) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 11:02:28.942571 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.fernet_utils [None req-09daec80-ca9b-4b4e-85f1-512f82baf219 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80144) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 11:02:29.003002 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 1177/2325] 10.222.0.25 () {58 vars in 958 bytes} [Sun Aug 7 11:02:28 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 07 11:02:29.011054 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa23c150-2061-4753-b54c-ff201f2493db None None] REQUEST_METHOD: `POST` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 11:02:29.012281 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa23c150-2061-4753-b54c-ff201f2493db None None] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 11:02:29.012739 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa23c150-2061-4753-b54c-ff201f2493db None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 11:02:29.036563 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.auth.core [None req-aa23c150-2061-4753-b54c-ff201f2493db None None] MFA Rules not processed for user `bac4e6b76fe64217ac8fa44e71cffe6a`. Rule list: `[]` (Enabled: `True`). {{(pid=80143) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 11:02:29.037774 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-aa23c150-2061-4753-b54c-ff201f2493db None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 11:02:29.049016 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1149/2326] 10.222.0.25 () {58 vars in 958 bytes} [Sun Aug 7 11:02:29 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 07 11:02:29.053849 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09daec80-ca9b-4b4e-85f1-512f82baf219 None None] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 11:02:29.068664 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09daec80-ca9b-4b4e-85f1-512f82baf219 None None] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 11:02:29.074323 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-201270db-e96f-4926-a99b-e02ff794015c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5ac47e3b09f43d19a23a0f457fe9e23', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 11:02:29.075321 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-201270db-e96f-4926-a99b-e02ff794015c admin admin] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 11:02:29.075609 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-201270db-e96f-4926-a99b-e02ff794015c admin admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 11:02:29.075885 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-201270db-e96f-4926-a99b-e02ff794015c admin admin] PATH_INFO: `/v3/domains` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 11:02:29.076534 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-201270db-e96f-4926-a99b-e02ff794015c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80144) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 11:02:29.076845 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-201270db-e96f-4926-a99b-e02ff794015c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 11:02:29.079171 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-201270db-e96f-4926-a99b-e02ff794015c admin admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 11:02:29.092331 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 1178/2327] 10.222.0.25 () {58 vars in 1153 bytes} [Sun Aug 7 11:02:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 11:02:29.097976 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa23c150-2061-4753-b54c-ff201f2493db None None] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 11:02:29.107183 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa23c150-2061-4753-b54c-ff201f2493db None None] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 11:02:29.112142 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd40f759-82c2-4530-8a47-fde6e0168cc8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5ac47e3b09f43d19a23a0f457fe9e23', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 11:02:29.112765 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd40f759-82c2-4530-8a47-fde6e0168cc8 admin admin] REQUEST_METHOD: `POST` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 11:02:29.113035 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd40f759-82c2-4530-8a47-fde6e0168cc8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 11:02:29.113311 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd40f759-82c2-4530-8a47-fde6e0168cc8 admin admin] PATH_INFO: `/v3/projects` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 11:02:29.113985 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd40f759-82c2-4530-8a47-fde6e0168cc8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 11:02:29.115934 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd40f759-82c2-4530-8a47-fde6e0168cc8 admin admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 11:02:29.145729 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1150/2328] 10.222.0.25 () {60 vars in 1153 bytes} [Sun Aug 7 11:02:29 2022] POST /identity/v3/projects => generated 373 bytes in 49 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 07 11:02:29.150355 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-201270db-e96f-4926-a99b-e02ff794015c admin admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 11:02:29.160539 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-201270db-e96f-4926-a99b-e02ff794015c admin admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 11:02:29.170143 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 11:02:29.224855 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-457d1833-14ed-4a98-b6fa-ca7c8a86cbb6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5ac47e3b09f43d19a23a0f457fe9e23', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 11:02:29.225529 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-457d1833-14ed-4a98-b6fa-ca7c8a86cbb6 admin admin] REQUEST_METHOD: `POST` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 11:02:29.225796 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-457d1833-14ed-4a98-b6fa-ca7c8a86cbb6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 11:02:29.226076 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-457d1833-14ed-4a98-b6fa-ca7c8a86cbb6 admin admin] PATH_INFO: `/v3/users` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 11:02:29.226710 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-457d1833-14ed-4a98-b6fa-ca7c8a86cbb6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 11:02:29.228668 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-457d1833-14ed-4a98-b6fa-ca7c8a86cbb6 admin admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 11:02:29.255487 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 1179/2329] 10.222.0.25 () {60 vars in 1144 bytes} [Sun Aug 7 11:02:29 2022] POST /identity/v3/users => generated 287 bytes in 102 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 07 11:02:29.256711 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd40f759-82c2-4530-8a47-fde6e0168cc8 admin admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 11:02:29.268423 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd40f759-82c2-4530-8a47-fde6e0168cc8 admin admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 11:02:29.273123 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3be9b54-4ff8-412b-8463-fd76da6f0e26 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5ac47e3b09f43d19a23a0f457fe9e23', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 11:02:29.273725 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3be9b54-4ff8-412b-8463-fd76da6f0e26 admin admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 11:02:29.274005 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3be9b54-4ff8-412b-8463-fd76da6f0e26 admin admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 11:02:29.274280 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3be9b54-4ff8-412b-8463-fd76da6f0e26 admin admin] PATH_INFO: `/v3/roles` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 11:02:29.274951 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3be9b54-4ff8-412b-8463-fd76da6f0e26 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 11:02:29.276529 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3be9b54-4ff8-412b-8463-fd76da6f0e26 admin admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 11:02:29.287916 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1151/2330] 10.222.0.25 () {58 vars in 1122 bytes} [Sun Aug 7 11:02:29 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 07 11:02:29.293394 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-457d1833-14ed-4a98-b6fa-ca7c8a86cbb6 admin admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 11:02:29.303654 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-457d1833-14ed-4a98-b6fa-ca7c8a86cbb6 admin admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 11:02:29.308989 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34fc5d9e-883a-487c-954d-fe85e17a7f17 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5ac47e3b09f43d19a23a0f457fe9e23', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 11:02:29.309598 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34fc5d9e-883a-487c-954d-fe85e17a7f17 admin admin] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 11:02:29.309881 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34fc5d9e-883a-487c-954d-fe85e17a7f17 admin admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 11:02:29.310156 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34fc5d9e-883a-487c-954d-fe85e17a7f17 admin admin] PATH_INFO: `/v3/roles` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 11:02:29.310801 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34fc5d9e-883a-487c-954d-fe85e17a7f17 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 11:02:29.312502 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34fc5d9e-883a-487c-954d-fe85e17a7f17 admin admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 11:02:29.324638 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 1180/2331] 10.222.0.25 () {58 vars in 1122 bytes} [Sun Aug 7 11:02:29 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 07 11:02:29.328809 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3be9b54-4ff8-412b-8463-fd76da6f0e26 admin admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 11:02:29.341331 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3be9b54-4ff8-412b-8463-fd76da6f0e26 admin admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 11:02:29.346199 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4dc0315-37b7-4b85-bd2b-93642e3da39b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5ac47e3b09f43d19a23a0f457fe9e23', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 11:02:29.346908 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4dc0315-37b7-4b85-bd2b-93642e3da39b admin admin] REQUEST_METHOD: `PUT` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 11:02:29.347223 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4dc0315-37b7-4b85-bd2b-93642e3da39b admin admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 11:02:29.347499 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4dc0315-37b7-4b85-bd2b-93642e3da39b admin admin] PATH_INFO: `/v3/projects/36a5dc2aa11449dcb60f1561fc7d6a12/users/1a41a3a73ed04a058e2dde1b0ea463ac/roles/b98dfb3b9fee4644af2887a3ac3fd2e6` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 11:02:29.365142 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4dc0315-37b7-4b85-bd2b-93642e3da39b admin admin] RBAC: Authorizing `identity:create_grant(project_id=36a5dc2aa11449dcb60f1561fc7d6a12, user_id=1a41a3a73ed04a058e2dde1b0ea463ac, role_id=b98dfb3b9fee4644af2887a3ac3fd2e6)` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 11:02:29.367842 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4dc0315-37b7-4b85-bd2b-93642e3da39b admin admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 11:02:29.381434 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1152/2332] 10.222.0.25 () {60 vars in 1483 bytes} [Sun Aug 7 11:02:29 2022] PUT /identity/v3/projects/36a5dc2aa11449dcb60f1561fc7d6a12/users/1a41a3a73ed04a058e2dde1b0ea463ac/roles/b98dfb3b9fee4644af2887a3ac3fd2e6 => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 07 11:02:29.394329 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0413ab2-ea0d-49f5-8c42-7394a05c8254 None None] REQUEST_METHOD: `POST` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 11:02:29.394656 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0413ab2-ea0d-49f5-8c42-7394a05c8254 None None] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 11:02:29.394872 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0413ab2-ea0d-49f5-8c42-7394a05c8254 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 11:02:29.426317 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.auth.core [None req-f0413ab2-ea0d-49f5-8c42-7394a05c8254 None None] MFA Rules not processed for user `1a41a3a73ed04a058e2dde1b0ea463ac`. Rule list: `[]` (Enabled: `True`). {{(pid=80144) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 11:02:29.427561 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.fernet_utils [None req-f0413ab2-ea0d-49f5-8c42-7394a05c8254 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80144) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 11:02:29.498545 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 1181/2333] 10.222.0.25 () {58 vars in 958 bytes} [Sun Aug 7 11:02:29 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 07 11:02:29.503236 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4dc0315-37b7-4b85-bd2b-93642e3da39b admin admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 11:02:29.553851 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 11:02:29.597878 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4dc0315-37b7-4b85-bd2b-93642e3da39b admin admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 11:02:29.602830 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d296a35-6c51-4de0-9486-54659513554b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5ac47e3b09f43d19a23a0f457fe9e23', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 11:02:29.603472 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d296a35-6c51-4de0-9486-54659513554b admin admin] REQUEST_METHOD: `POST` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 11:02:29.604276 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d296a35-6c51-4de0-9486-54659513554b admin admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 11:02:29.604553 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d296a35-6c51-4de0-9486-54659513554b admin admin] PATH_INFO: `/v3/projects` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 11:02:29.606433 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d296a35-6c51-4de0-9486-54659513554b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 11:02:29.609401 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d296a35-6c51-4de0-9486-54659513554b admin admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 11:02:29.649554 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1153/2334] 10.222.0.25 () {60 vars in 1153 bytes} [Sun Aug 7 11:02:29 2022] POST /identity/v3/projects => generated 373 bytes in 147 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 07 11:02:29.654576 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0413ab2-ea0d-49f5-8c42-7394a05c8254 None None] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 11:02:29.665266 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0413ab2-ea0d-49f5-8c42-7394a05c8254 None None] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 11:02:29.723283 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1533fe9-0807-4545-be4d-9589576db397 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5ac47e3b09f43d19a23a0f457fe9e23', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 11:02:29.723893 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1533fe9-0807-4545-be4d-9589576db397 admin admin] REQUEST_METHOD: `POST` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 11:02:29.724172 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1533fe9-0807-4545-be4d-9589576db397 admin admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 11:02:29.724444 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1533fe9-0807-4545-be4d-9589576db397 admin admin] PATH_INFO: `/v3/users` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 11:02:29.725068 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1533fe9-0807-4545-be4d-9589576db397 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 11:02:29.726987 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1533fe9-0807-4545-be4d-9589576db397 admin admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 11:02:29.748261 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 1182/2335] 10.222.0.25 () {60 vars in 1144 bytes} [Sun Aug 7 11:02:29 2022] POST /identity/v3/users => generated 293 bytes in 94 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 07 11:02:29.752512 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d296a35-6c51-4de0-9486-54659513554b admin admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 11:02:29.762885 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d296a35-6c51-4de0-9486-54659513554b admin admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 11:02:29.767474 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4890b32-4e40-4ffd-9f31-5f453bcc57da admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5ac47e3b09f43d19a23a0f457fe9e23', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 11:02:29.768062 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4890b32-4e40-4ffd-9f31-5f453bcc57da admin admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 11:02:29.768367 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4890b32-4e40-4ffd-9f31-5f453bcc57da admin admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 11:02:29.768640 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4890b32-4e40-4ffd-9f31-5f453bcc57da admin admin] PATH_INFO: `/v3/roles` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 11:02:29.769273 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4890b32-4e40-4ffd-9f31-5f453bcc57da admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 11:02:29.770847 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4890b32-4e40-4ffd-9f31-5f453bcc57da admin admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 11:02:29.782345 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1154/2336] 10.222.0.25 () {58 vars in 1122 bytes} [Sun Aug 7 11:02:29 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 07 11:02:29.786649 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1533fe9-0807-4545-be4d-9589576db397 admin admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 11:02:29.797473 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1533fe9-0807-4545-be4d-9589576db397 admin admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 11:02:29.801989 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1b8e36e-f0d2-447a-8ddf-ab50ba4fae5e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5ac47e3b09f43d19a23a0f457fe9e23', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 11:02:29.802635 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1b8e36e-f0d2-447a-8ddf-ab50ba4fae5e admin admin] REQUEST_METHOD: `PUT` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 11:02:29.802913 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1b8e36e-f0d2-447a-8ddf-ab50ba4fae5e admin admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 11:02:29.803220 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1b8e36e-f0d2-447a-8ddf-ab50ba4fae5e admin admin] PATH_INFO: `/v3/projects/2074470c836347c59fefb5ecd98e1abe/users/8cbbc109469b4cd6aa434780fdd3eb6b/roles/81b1f1bcee2a473088fbbd327c1c5b23` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 11:02:29.811829 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 11:02:29.819616 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1b8e36e-f0d2-447a-8ddf-ab50ba4fae5e admin admin] RBAC: Authorizing `identity:create_grant(project_id=2074470c836347c59fefb5ecd98e1abe, user_id=8cbbc109469b4cd6aa434780fdd3eb6b, role_id=81b1f1bcee2a473088fbbd327c1c5b23)` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 11:02:29.822221 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1b8e36e-f0d2-447a-8ddf-ab50ba4fae5e admin admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 11:02:29.834544 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 1183/2337] 10.222.0.25 () {60 vars in 1483 bytes} [Sun Aug 7 11:02:29 2022] PUT /identity/v3/projects/2074470c836347c59fefb5ecd98e1abe/users/8cbbc109469b4cd6aa434780fdd3eb6b/roles/81b1f1bcee2a473088fbbd327c1c5b23 => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 07 11:02:29.838540 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4890b32-4e40-4ffd-9f31-5f453bcc57da admin admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 11:02:29.848855 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4890b32-4e40-4ffd-9f31-5f453bcc57da admin admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 11:02:29.903417 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a6d75d1-39f1-49aa-8ae5-e0cf56488e6b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5ac47e3b09f43d19a23a0f457fe9e23', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 11:02:29.904024 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a6d75d1-39f1-49aa-8ae5-e0cf56488e6b admin admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 11:02:29.904304 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a6d75d1-39f1-49aa-8ae5-e0cf56488e6b admin admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 11:02:29.904575 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a6d75d1-39f1-49aa-8ae5-e0cf56488e6b admin admin] PATH_INFO: `/v3/roles` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 11:02:29.905283 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a6d75d1-39f1-49aa-8ae5-e0cf56488e6b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 11:02:29.906886 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a6d75d1-39f1-49aa-8ae5-e0cf56488e6b admin admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 11:02:29.916883 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1155/2338] 10.222.0.25 () {58 vars in 1122 bytes} [Sun Aug 7 11:02:29 2022] GET /identity/v3/roles => generated 1420 bytes in 79 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 07 11:02:29.921844 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1b8e36e-f0d2-447a-8ddf-ab50ba4fae5e admin admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 11:02:29.932302 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1b8e36e-f0d2-447a-8ddf-ab50ba4fae5e admin admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 11:02:29.937196 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9776a40-b121-4a3d-9f8a-65d08b56d26c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5ac47e3b09f43d19a23a0f457fe9e23', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 11:02:29.937851 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9776a40-b121-4a3d-9f8a-65d08b56d26c admin admin] REQUEST_METHOD: `PUT` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 11:02:29.938136 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9776a40-b121-4a3d-9f8a-65d08b56d26c admin admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 11:02:29.938442 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9776a40-b121-4a3d-9f8a-65d08b56d26c admin admin] PATH_INFO: `/v3/projects/2074470c836347c59fefb5ecd98e1abe/users/8cbbc109469b4cd6aa434780fdd3eb6b/roles/81b1f1bcee2a473088fbbd327c1c5b23` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 11:02:29.940408 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9776a40-b121-4a3d-9f8a-65d08b56d26c admin admin] RBAC: Authorizing `identity:create_grant(project_id=2074470c836347c59fefb5ecd98e1abe, user_id=8cbbc109469b4cd6aa434780fdd3eb6b, role_id=81b1f1bcee2a473088fbbd327c1c5b23)` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 11:02:29.943054 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9776a40-b121-4a3d-9f8a-65d08b56d26c admin admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 11:02:29.953865 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 1184/2339] 10.222.0.25 () {60 vars in 1483 bytes} [Sun Aug 7 11:02:29 2022] PUT /identity/v3/projects/2074470c836347c59fefb5ecd98e1abe/users/8cbbc109469b4cd6aa434780fdd3eb6b/roles/81b1f1bcee2a473088fbbd327c1c5b23 => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 07 11:02:29.964214 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-675daaeb-2400-4c06-9346-af09240320fc None None] REQUEST_METHOD: `POST` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 11:02:29.965282 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-675daaeb-2400-4c06-9346-af09240320fc None None] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 11:02:29.965588 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-675daaeb-2400-4c06-9346-af09240320fc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 11:02:29.992886 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.auth.core [None req-675daaeb-2400-4c06-9346-af09240320fc None None] MFA Rules not processed for user `8cbbc109469b4cd6aa434780fdd3eb6b`. Rule list: `[]` (Enabled: `True`). {{(pid=80143) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 11:02:29.994067 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-675daaeb-2400-4c06-9346-af09240320fc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 11:02:30.065779 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1156/2340] 10.222.0.25 () {58 vars in 958 bytes} [Sun Aug 7 11:02:29 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 07 11:02:30.078104 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9776a40-b121-4a3d-9f8a-65d08b56d26c admin admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 11:02:30.079814 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.fernet_utils [None req-f9776a40-b121-4a3d-9f8a-65d08b56d26c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80144) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 11:02:30.136003 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9776a40-b121-4a3d-9f8a-65d08b56d26c admin admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 11:02:30.140200 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6223f612-c850-48d7-9de6-e1f3406f5860 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88dbd6dfd1164994a3e16da553561910', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6de1f3cac02c4bcdb05dd2337176033d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 11:02:30.140792 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6223f612-c850-48d7-9de6-e1f3406f5860 service cinder] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 11:02:30.141068 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6223f612-c850-48d7-9de6-e1f3406f5860 service cinder] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 11:02:30.141339 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6223f612-c850-48d7-9de6-e1f3406f5860 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 11:02:30.150317 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6223f612-c850-48d7-9de6-e1f3406f5860 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 11:02:30.152390 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6223f612-c850-48d7-9de6-e1f3406f5860 service cinder] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 11:02:30.161619 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 1185/2341] 10.222.0.25 () {60 vars in 1365 bytes} [Sun Aug 7 11:02:30 2022] GET /identity/v3/auth/tokens => generated 3317 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 07 11:02:30.239007 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-675daaeb-2400-4c06-9346-af09240320fc None None] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 11:02:30.249889 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-675daaeb-2400-4c06-9346-af09240320fc None None] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 11:02:30.253620 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b62d700a-944f-45f7-a4a9-a309572570ef service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88dbd6dfd1164994a3e16da553561910', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6de1f3cac02c4bcdb05dd2337176033d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 11:02:30.254219 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b62d700a-944f-45f7-a4a9-a309572570ef service cinder] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 11:02:30.254502 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b62d700a-944f-45f7-a4a9-a309572570ef service cinder] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 11:02:30.254776 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b62d700a-944f-45f7-a4a9-a309572570ef service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 11:02:30.285942 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 11:02:30.317452 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b62d700a-944f-45f7-a4a9-a309572570ef service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 11:02:30.319108 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b62d700a-944f-45f7-a4a9-a309572570ef service cinder] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 11:02:30.329844 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1157/2342] 10.222.0.25 () {60 vars in 1365 bytes} [Sun Aug 7 11:02:30 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 07 11:02:40.157477 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=36,keystone:INSERT=2 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 11:02:40.324348 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=14 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 11:03:03.709650 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6223f612-c850-48d7-9de6-e1f3406f5860 service cinder] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 11:03:03.711330 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.fernet_utils [None req-6223f612-c850-48d7-9de6-e1f3406f5860 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80144) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 11:03:03.794222 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6223f612-c850-48d7-9de6-e1f3406f5860 service cinder] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 11:03:03.798037 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c123455b-40c9-4912-a768-9b8fb42447f8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be1283c9bf47424c9aa3ad27cab03027', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6de1f3cac02c4bcdb05dd2337176033d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 11:03:03.798645 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c123455b-40c9-4912-a768-9b8fb42447f8 service neutron] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 11:03:03.798959 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c123455b-40c9-4912-a768-9b8fb42447f8 service neutron] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 11:03:03.799220 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c123455b-40c9-4912-a768-9b8fb42447f8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 11:03:03.800954 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.fernet_utils [None req-c123455b-40c9-4912-a768-9b8fb42447f8 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80144) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 11:03:03.864375 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c123455b-40c9-4912-a768-9b8fb42447f8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 11:03:03.865722 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c123455b-40c9-4912-a768-9b8fb42447f8 service neutron] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 11:03:03.875145 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 1186/2343] 10.222.0.25 () {60 vars in 1365 bytes} [Sun Aug 7 11:03:03 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 166 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 07 11:03:08.157520 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b62d700a-944f-45f7-a4a9-a309572570ef service cinder] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 11:03:08.159001 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-b62d700a-944f-45f7-a4a9-a309572570ef service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 11:03:08.249127 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b62d700a-944f-45f7-a4a9-a309572570ef service cinder] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 11:03:08.254151 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0ff53bf-85ad-42a3-add5-15666b9e5406 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6ee2e4c08af04c219ed395d7ca7085a6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6de1f3cac02c4bcdb05dd2337176033d', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 11:03:08.254747 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0ff53bf-85ad-42a3-add5-15666b9e5406 service placement] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 11:03:08.255054 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0ff53bf-85ad-42a3-add5-15666b9e5406 service placement] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 11:03:08.255329 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0ff53bf-85ad-42a3-add5-15666b9e5406 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 11:03:08.257030 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-a0ff53bf-85ad-42a3-add5-15666b9e5406 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 11:03:08.267968 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0ff53bf-85ad-42a3-add5-15666b9e5406 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 11:03:08.269518 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0ff53bf-85ad-42a3-add5-15666b9e5406 service placement] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 11:03:08.278895 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1158/2344] 10.222.0.25 () {60 vars in 1365 bytes} [Sun Aug 7 11:03:08 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 122 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 07 11:03:13.421338 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c123455b-40c9-4912-a768-9b8fb42447f8 service neutron] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 11:03:13.422873 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.fernet_utils [None req-c123455b-40c9-4912-a768-9b8fb42447f8 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80144) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 11:03:13.434310 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c123455b-40c9-4912-a768-9b8fb42447f8 service neutron] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 11:03:13.439955 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6a7ec9f-0fbd-4aa1-b515-7be9dcb630a6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be1283c9bf47424c9aa3ad27cab03027', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6de1f3cac02c4bcdb05dd2337176033d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 11:03:13.440659 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6a7ec9f-0fbd-4aa1-b515-7be9dcb630a6 service neutron] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 11:03:13.441010 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6a7ec9f-0fbd-4aa1-b515-7be9dcb630a6 service neutron] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 11:03:13.441360 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6a7ec9f-0fbd-4aa1-b515-7be9dcb630a6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 11:03:13.443219 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.fernet_utils [None req-b6a7ec9f-0fbd-4aa1-b515-7be9dcb630a6 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80144) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 11:03:13.521453 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6a7ec9f-0fbd-4aa1-b515-7be9dcb630a6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 11:03:13.526732 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6a7ec9f-0fbd-4aa1-b515-7be9dcb630a6 service neutron] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 11:03:13.537453 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 1187/2345] 10.222.0.25 () {60 vars in 1365 bytes} [Sun Aug 7 11:03:13 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 116 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 07 11:03:15.305066 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0ff53bf-85ad-42a3-add5-15666b9e5406 service placement] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 11:03:15.316776 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0ff53bf-85ad-42a3-add5-15666b9e5406 service placement] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 11:03:15.321464 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0056f0d6-8405-43b2-a49e-fc1e50665e6e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5ac47e3b09f43d19a23a0f457fe9e23', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 11:03:15.322107 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0056f0d6-8405-43b2-a49e-fc1e50665e6e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 11:03:15.322390 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0056f0d6-8405-43b2-a49e-fc1e50665e6e admin admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 11:03:15.322681 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0056f0d6-8405-43b2-a49e-fc1e50665e6e admin admin] PATH_INFO: `/v3/users/1a41a3a73ed04a058e2dde1b0ea463ac` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 11:03:15.324072 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0056f0d6-8405-43b2-a49e-fc1e50665e6e admin admin] RBAC: Authorizing `identity:delete_user(user_id=1a41a3a73ed04a058e2dde1b0ea463ac)` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 11:03:15.326297 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0056f0d6-8405-43b2-a49e-fc1e50665e6e admin admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 11:03:15.400048 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-0056f0d6-8405-43b2-a49e-fc1e50665e6e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1a41a3a73ed04a058e2dde1b0ea463ac'} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 11:03:15.408786 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-0056f0d6-8405-43b2-a49e-fc1e50665e6e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1a41a3a73ed04a058e2dde1b0ea463ac'} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 11:03:15.425428 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-0056f0d6-8405-43b2-a49e-fc1e50665e6e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1a41a3a73ed04a058e2dde1b0ea463ac'} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 11:03:15.426402 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-0056f0d6-8405-43b2-a49e-fc1e50665e6e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1a41a3a73ed04a058e2dde1b0ea463ac'} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 11:03:15.435574 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-0056f0d6-8405-43b2-a49e-fc1e50665e6e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1a41a3a73ed04a058e2dde1b0ea463ac'} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 11:03:15.442700 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1159/2346] 10.222.0.25 () {58 vars in 1224 bytes} [Sun Aug 7 11:03:15 2022] DELETE /identity/v3/users/1a41a3a73ed04a058e2dde1b0ea463ac => generated 0 bytes in 138 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 07 11:03:15.447006 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5269304-bb4a-4e19-a373-2e3e2c3f4171 None None] REQUEST_METHOD: `POST` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 11:03:15.447501 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5269304-bb4a-4e19-a373-2e3e2c3f4171 None None] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 11:03:15.447606 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5269304-bb4a-4e19-a373-2e3e2c3f4171 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 11:03:15.468030 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.auth.core [None req-f5269304-bb4a-4e19-a373-2e3e2c3f4171 None None] MFA Rules not processed for user `bac4e6b76fe64217ac8fa44e71cffe6a`. Rule list: `[]` (Enabled: `True`). {{(pid=80144) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 11:03:15.469492 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.fernet_utils [None req-f5269304-bb4a-4e19-a373-2e3e2c3f4171 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80144) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 11:03:15.497765 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=100 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 11:03:15.539804 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 1188/2347] 10.222.0.25 () {58 vars in 958 bytes} [Sun Aug 7 11:03:15 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 07 11:03:15.549356 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0056f0d6-8405-43b2-a49e-fc1e50665e6e admin admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 11:03:15.551054 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-0056f0d6-8405-43b2-a49e-fc1e50665e6e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 11:03:15.606391 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 11:03:15.621616 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0056f0d6-8405-43b2-a49e-fc1e50665e6e admin admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 11:03:15.625919 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14fa1b76-68e3-4704-b0c1-7765523d3394 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be1283c9bf47424c9aa3ad27cab03027', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6de1f3cac02c4bcdb05dd2337176033d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 11:03:15.626609 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14fa1b76-68e3-4704-b0c1-7765523d3394 service neutron] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 11:03:15.626980 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14fa1b76-68e3-4704-b0c1-7765523d3394 service neutron] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 11:03:15.627324 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14fa1b76-68e3-4704-b0c1-7765523d3394 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 11:03:15.637495 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14fa1b76-68e3-4704-b0c1-7765523d3394 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 11:03:15.639600 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14fa1b76-68e3-4704-b0c1-7765523d3394 service neutron] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 11:03:15.650355 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1160/2348] 10.222.0.25 () {60 vars in 1365 bytes} [Sun Aug 7 11:03:15 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 102 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 07 11:03:15.954040 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5269304-bb4a-4e19-a373-2e3e2c3f4171 None None] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 11:03:15.955709 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.fernet_utils [None req-f5269304-bb4a-4e19-a373-2e3e2c3f4171 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80144) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 11:03:15.990527 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5269304-bb4a-4e19-a373-2e3e2c3f4171 None None] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 11:03:15.995656 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec62a3b7-c644-4c01-9924-30624c0d378c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5ac47e3b09f43d19a23a0f457fe9e23', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 11:03:15.996362 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec62a3b7-c644-4c01-9924-30624c0d378c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 11:03:15.996712 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec62a3b7-c644-4c01-9924-30624c0d378c admin admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 11:03:15.997056 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec62a3b7-c644-4c01-9924-30624c0d378c admin admin] PATH_INFO: `/v3/projects/36a5dc2aa11449dcb60f1561fc7d6a12` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 11:03:15.998825 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec62a3b7-c644-4c01-9924-30624c0d378c admin admin] RBAC: Authorizing `identity:delete_project(project_id=36a5dc2aa11449dcb60f1561fc7d6a12)` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 11:03:16.000848 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec62a3b7-c644-4c01-9924-30624c0d378c admin admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 11:03:16.056726 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.notifications [None req-ec62a3b7-c644-4c01-9924-30624c0d378c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '36a5dc2aa11449dcb60f1561fc7d6a12', 'request_id': 'req-ec62a3b7-c644-4c01-9924-30624c0d378c'} {{(pid=80144) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 11:03:16.062820 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.notifications [None req-ec62a3b7-c644-4c01-9924-30624c0d378c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '36a5dc2aa11449dcb60f1561fc7d6a12', 'request_id': 'req-ec62a3b7-c644-4c01-9924-30624c0d378c'} {{(pid=80144) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 11:03:16.070996 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.notifications [None req-ec62a3b7-c644-4c01-9924-30624c0d378c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '36a5dc2aa11449dcb60f1561fc7d6a12', 'request_id': 'req-ec62a3b7-c644-4c01-9924-30624c0d378c'} {{(pid=80144) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 11:03:16.082528 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.notifications [None req-ec62a3b7-c644-4c01-9924-30624c0d378c admin admin] The token cache is being invalidate because project 36a5dc2aa11449dcb60f1561fc7d6a12 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80144) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 07 11:03:16.082851 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.notifications [None req-ec62a3b7-c644-4c01-9924-30624c0d378c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80144) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 11:03:16.083676 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.notifications [None req-ec62a3b7-c644-4c01-9924-30624c0d378c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80144) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 11:03:16.085172 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 1189/2349] 10.222.0.25 () {58 vars in 1233 bytes} [Sun Aug 7 11:03:15 2022] DELETE /identity/v3/projects/36a5dc2aa11449dcb60f1561fc7d6a12 => generated 0 bytes in 132 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 07 11:03:16.089172 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14fa1b76-68e3-4704-b0c1-7765523d3394 service neutron] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 11:03:16.101088 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14fa1b76-68e3-4704-b0c1-7765523d3394 service neutron] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 11:03:16.161660 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27598e6e-c2be-4c49-9c4d-be6357734fbc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5ac47e3b09f43d19a23a0f457fe9e23', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 11:03:16.162218 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27598e6e-c2be-4c49-9c4d-be6357734fbc admin admin] REQUEST_METHOD: `DELETE` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 11:03:16.162498 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27598e6e-c2be-4c49-9c4d-be6357734fbc admin admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 11:03:16.162772 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27598e6e-c2be-4c49-9c4d-be6357734fbc admin admin] PATH_INFO: `/v3/users/8cbbc109469b4cd6aa434780fdd3eb6b` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 11:03:16.163943 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27598e6e-c2be-4c49-9c4d-be6357734fbc admin admin] RBAC: Authorizing `identity:delete_user(user_id=8cbbc109469b4cd6aa434780fdd3eb6b)` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 11:03:16.165868 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27598e6e-c2be-4c49-9c4d-be6357734fbc admin admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 11:03:16.223969 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-27598e6e-c2be-4c49-9c4d-be6357734fbc admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8cbbc109469b4cd6aa434780fdd3eb6b'} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 11:03:16.231979 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-27598e6e-c2be-4c49-9c4d-be6357734fbc admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8cbbc109469b4cd6aa434780fdd3eb6b'} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 11:03:16.257346 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-27598e6e-c2be-4c49-9c4d-be6357734fbc admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8cbbc109469b4cd6aa434780fdd3eb6b'} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 11:03:16.258361 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-27598e6e-c2be-4c49-9c4d-be6357734fbc admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8cbbc109469b4cd6aa434780fdd3eb6b'} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 11:03:16.268439 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-27598e6e-c2be-4c49-9c4d-be6357734fbc admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8cbbc109469b4cd6aa434780fdd3eb6b'} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 11:03:16.278092 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1161/2350] 10.222.0.25 () {58 vars in 1224 bytes} [Sun Aug 7 11:03:16 2022] DELETE /identity/v3/users/8cbbc109469b4cd6aa434780fdd3eb6b => generated 0 bytes in 189 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 07 11:03:16.778042 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec62a3b7-c644-4c01-9924-30624c0d378c admin admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 11:03:16.780642 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.fernet_utils [None req-ec62a3b7-c644-4c01-9924-30624c0d378c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80144) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 11:03:16.803796 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec62a3b7-c644-4c01-9924-30624c0d378c admin admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 11:03:16.881981 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8aa02b4-a8db-4b0c-b734-eae2301acfda admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5ac47e3b09f43d19a23a0f457fe9e23', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 11:03:16.882724 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8aa02b4-a8db-4b0c-b734-eae2301acfda admin admin] REQUEST_METHOD: `DELETE` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 11:03:16.883039 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8aa02b4-a8db-4b0c-b734-eae2301acfda admin admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 11:03:16.883319 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8aa02b4-a8db-4b0c-b734-eae2301acfda admin admin] PATH_INFO: `/v3/projects/2074470c836347c59fefb5ecd98e1abe` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 11:03:16.884992 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8aa02b4-a8db-4b0c-b734-eae2301acfda admin admin] RBAC: Authorizing `identity:delete_project(project_id=2074470c836347c59fefb5ecd98e1abe)` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 11:03:16.887013 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8aa02b4-a8db-4b0c-b734-eae2301acfda admin admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 11:03:16.945540 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.notifications [None req-b8aa02b4-a8db-4b0c-b734-eae2301acfda admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2074470c836347c59fefb5ecd98e1abe', 'request_id': 'req-b8aa02b4-a8db-4b0c-b734-eae2301acfda'} {{(pid=80144) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 11:03:16.953379 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 11:03:16.958411 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.notifications [None req-b8aa02b4-a8db-4b0c-b734-eae2301acfda admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2074470c836347c59fefb5ecd98e1abe', 'request_id': 'req-b8aa02b4-a8db-4b0c-b734-eae2301acfda'} {{(pid=80144) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 11:03:16.969615 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.notifications [None req-b8aa02b4-a8db-4b0c-b734-eae2301acfda admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2074470c836347c59fefb5ecd98e1abe', 'request_id': 'req-b8aa02b4-a8db-4b0c-b734-eae2301acfda'} {{(pid=80144) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 11:03:16.983886 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.notifications [None req-b8aa02b4-a8db-4b0c-b734-eae2301acfda admin admin] The token cache is being invalidate because project 2074470c836347c59fefb5ecd98e1abe was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80144) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 07 11:03:16.984173 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.notifications [None req-b8aa02b4-a8db-4b0c-b734-eae2301acfda admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80144) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 11:03:16.984882 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.notifications [None req-b8aa02b4-a8db-4b0c-b734-eae2301acfda admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80144) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 11:03:16.986863 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 1190/2351] 10.222.0.25 () {58 vars in 1233 bytes} [Sun Aug 7 11:03:16 2022] DELETE /identity/v3/projects/2074470c836347c59fefb5ecd98e1abe => generated 0 bytes in 209 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 07 11:03:16.991500 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3554896-56cb-4aa9-a262-b534f90d5ece None None] REQUEST_METHOD: `POST` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 11:03:16.991999 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3554896-56cb-4aa9-a262-b534f90d5ece None None] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 11:03:16.992090 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3554896-56cb-4aa9-a262-b534f90d5ece None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 11:03:17.014967 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.auth.core [None req-f3554896-56cb-4aa9-a262-b534f90d5ece None None] MFA Rules not processed for user `bac4e6b76fe64217ac8fa44e71cffe6a`. Rule list: `[]` (Enabled: `True`). {{(pid=80143) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 11:03:17.016191 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-f3554896-56cb-4aa9-a262-b534f90d5ece None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 11:03:17.099092 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1162/2352] 10.222.0.25 () {58 vars in 958 bytes} [Sun Aug 7 11:03:16 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 07 11:03:17.104863 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7a48bfd-45de-4b0f-9fb7-58118b025c88 None None] REQUEST_METHOD: `POST` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 11:03:17.105167 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7a48bfd-45de-4b0f-9fb7-58118b025c88 None None] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 11:03:17.105447 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7a48bfd-45de-4b0f-9fb7-58118b025c88 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 11:03:17.127088 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.auth.core [None req-a7a48bfd-45de-4b0f-9fb7-58118b025c88 None None] MFA Rules not processed for user `bac4e6b76fe64217ac8fa44e71cffe6a`. Rule list: `[]` (Enabled: `True`). {{(pid=80144) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 11:03:17.128399 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.fernet_utils [None req-a7a48bfd-45de-4b0f-9fb7-58118b025c88 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80144) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 11:03:17.136520 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 1191/2353] 10.222.0.25 () {58 vars in 958 bytes} [Sun Aug 7 11:03:17 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 07 11:03:17.141627 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3554896-56cb-4aa9-a262-b534f90d5ece None None] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 11:03:17.147825 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 11:03:17.151808 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3554896-56cb-4aa9-a262-b534f90d5ece None None] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 11:03:17.159348 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb4d9cc6-e695-4803-abcb-feaf9ba46220 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5ac47e3b09f43d19a23a0f457fe9e23', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 11:03:17.160052 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb4d9cc6-e695-4803-abcb-feaf9ba46220 admin admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 11:03:17.160536 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb4d9cc6-e695-4803-abcb-feaf9ba46220 admin admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 11:03:17.161314 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb4d9cc6-e695-4803-abcb-feaf9ba46220 admin admin] PATH_INFO: `/v3/domains` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 11:03:17.162338 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb4d9cc6-e695-4803-abcb-feaf9ba46220 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80143) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 11:03:17.162705 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb4d9cc6-e695-4803-abcb-feaf9ba46220 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 11:03:17.164411 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb4d9cc6-e695-4803-abcb-feaf9ba46220 admin admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 11:03:17.176899 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1163/2354] 10.222.0.25 () {58 vars in 1153 bytes} [Sun Aug 7 11:03:17 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 07 11:03:17.181117 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7a48bfd-45de-4b0f-9fb7-58118b025c88 None None] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 11:03:17.191239 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7a48bfd-45de-4b0f-9fb7-58118b025c88 None None] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 11:03:17.195987 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-324433c9-8d00-4682-b137-d48a72aac905 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5ac47e3b09f43d19a23a0f457fe9e23', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 11:03:17.196612 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-324433c9-8d00-4682-b137-d48a72aac905 admin admin] REQUEST_METHOD: `POST` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 11:03:17.196890 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-324433c9-8d00-4682-b137-d48a72aac905 admin admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 11:03:17.197162 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-324433c9-8d00-4682-b137-d48a72aac905 admin admin] PATH_INFO: `/v3/projects` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 11:03:17.197806 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-324433c9-8d00-4682-b137-d48a72aac905 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 11:03:17.199783 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-324433c9-8d00-4682-b137-d48a72aac905 admin admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 11:03:17.231698 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 1192/2355] 10.222.0.25 () {60 vars in 1153 bytes} [Sun Aug 7 11:03:17 2022] POST /identity/v3/projects => generated 377 bytes in 51 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 07 11:03:17.236403 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb4d9cc6-e695-4803-abcb-feaf9ba46220 admin admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 11:03:17.246935 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb4d9cc6-e695-4803-abcb-feaf9ba46220 admin admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 11:03:17.317276 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcf0597d-5225-48c1-9f19-3a525616483b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5ac47e3b09f43d19a23a0f457fe9e23', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 11:03:17.318404 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcf0597d-5225-48c1-9f19-3a525616483b admin admin] REQUEST_METHOD: `POST` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 11:03:17.319036 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcf0597d-5225-48c1-9f19-3a525616483b admin admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 11:03:17.319646 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcf0597d-5225-48c1-9f19-3a525616483b admin admin] PATH_INFO: `/v3/users` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 11:03:17.322187 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcf0597d-5225-48c1-9f19-3a525616483b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 11:03:17.324823 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcf0597d-5225-48c1-9f19-3a525616483b admin admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 11:03:17.362812 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1164/2356] 10.222.0.25 () {60 vars in 1144 bytes} [Sun Aug 7 11:03:17 2022] POST /identity/v3/users => generated 289 bytes in 124 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 07 11:03:17.364056 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-324433c9-8d00-4682-b137-d48a72aac905 admin admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 11:03:17.375524 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-324433c9-8d00-4682-b137-d48a72aac905 admin admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 11:03:17.380676 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-188403c3-36bb-4674-9aa6-b3f7ca8f2069 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5ac47e3b09f43d19a23a0f457fe9e23', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 11:03:17.381363 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-188403c3-36bb-4674-9aa6-b3f7ca8f2069 admin admin] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 11:03:17.381711 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-188403c3-36bb-4674-9aa6-b3f7ca8f2069 admin admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 11:03:17.382050 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-188403c3-36bb-4674-9aa6-b3f7ca8f2069 admin admin] PATH_INFO: `/v3/roles` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 11:03:17.382765 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-188403c3-36bb-4674-9aa6-b3f7ca8f2069 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 11:03:17.384449 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-188403c3-36bb-4674-9aa6-b3f7ca8f2069 admin admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 11:03:17.399960 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 1193/2357] 10.222.0.25 () {58 vars in 1122 bytes} [Sun Aug 7 11:03:17 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 07 11:03:17.404417 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcf0597d-5225-48c1-9f19-3a525616483b admin admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 11:03:17.418236 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcf0597d-5225-48c1-9f19-3a525616483b admin admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 11:03:17.426949 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf9d92f2-5725-4a00-bb8d-d429a6f7fa1f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5ac47e3b09f43d19a23a0f457fe9e23', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 11:03:17.427788 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf9d92f2-5725-4a00-bb8d-d429a6f7fa1f admin admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 11:03:17.428148 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf9d92f2-5725-4a00-bb8d-d429a6f7fa1f admin admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 11:03:17.428515 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf9d92f2-5725-4a00-bb8d-d429a6f7fa1f admin admin] PATH_INFO: `/v3/roles` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 11:03:17.429271 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf9d92f2-5725-4a00-bb8d-d429a6f7fa1f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 11:03:17.431201 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf9d92f2-5725-4a00-bb8d-d429a6f7fa1f admin admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 11:03:17.444802 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1165/2358] 10.222.0.25 () {58 vars in 1122 bytes} [Sun Aug 7 11:03:17 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 07 11:03:17.448879 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-188403c3-36bb-4674-9aa6-b3f7ca8f2069 admin admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 11:03:17.460106 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-188403c3-36bb-4674-9aa6-b3f7ca8f2069 admin admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 11:03:17.465119 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-203a46dc-0dd1-4bf6-a4e1-2be348546bad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5ac47e3b09f43d19a23a0f457fe9e23', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 11:03:17.465801 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-203a46dc-0dd1-4bf6-a4e1-2be348546bad admin admin] REQUEST_METHOD: `PUT` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 11:03:17.466084 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-203a46dc-0dd1-4bf6-a4e1-2be348546bad admin admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 11:03:17.466388 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-203a46dc-0dd1-4bf6-a4e1-2be348546bad admin admin] PATH_INFO: `/v3/projects/7d2f9bfc71a24e038e555a188b933b23/users/e274f6fe64a7479bb55a7b9dd1470e85/roles/b98dfb3b9fee4644af2887a3ac3fd2e6` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 11:03:17.485317 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-203a46dc-0dd1-4bf6-a4e1-2be348546bad admin admin] RBAC: Authorizing `identity:create_grant(project_id=7d2f9bfc71a24e038e555a188b933b23, user_id=e274f6fe64a7479bb55a7b9dd1470e85, role_id=b98dfb3b9fee4644af2887a3ac3fd2e6)` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 11:03:17.488085 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-203a46dc-0dd1-4bf6-a4e1-2be348546bad admin admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 11:03:17.511502 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 1194/2359] 10.222.0.25 () {60 vars in 1483 bytes} [Sun Aug 7 11:03:17 2022] PUT /identity/v3/projects/7d2f9bfc71a24e038e555a188b933b23/users/e274f6fe64a7479bb55a7b9dd1470e85/roles/b98dfb3b9fee4644af2887a3ac3fd2e6 => generated 0 bytes in 56 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 07 11:03:17.516353 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc015680-e75b-428c-ab8d-81dd68d09246 None None] REQUEST_METHOD: `POST` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 11:03:17.516635 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc015680-e75b-428c-ab8d-81dd68d09246 None None] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 11:03:17.516908 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc015680-e75b-428c-ab8d-81dd68d09246 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 11:03:17.547870 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.auth.core [None req-bc015680-e75b-428c-ab8d-81dd68d09246 None None] MFA Rules not processed for user `e274f6fe64a7479bb55a7b9dd1470e85`. Rule list: `[]` (Enabled: `True`). {{(pid=80143) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 11:03:17.549225 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-bc015680-e75b-428c-ab8d-81dd68d09246 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 11:03:17.617786 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1166/2360] 10.222.0.25 () {58 vars in 958 bytes} [Sun Aug 7 11:03:17 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 07 11:03:17.622190 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-203a46dc-0dd1-4bf6-a4e1-2be348546bad admin admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 11:03:17.686640 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-203a46dc-0dd1-4bf6-a4e1-2be348546bad admin admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 11:03:17.691381 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3af1530f-1659-4ee8-ae2d-7c9da37cd179 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5ac47e3b09f43d19a23a0f457fe9e23', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 11:03:17.691988 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3af1530f-1659-4ee8-ae2d-7c9da37cd179 admin admin] REQUEST_METHOD: `POST` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 11:03:17.692269 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3af1530f-1659-4ee8-ae2d-7c9da37cd179 admin admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 11:03:17.692567 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3af1530f-1659-4ee8-ae2d-7c9da37cd179 admin admin] PATH_INFO: `/v3/projects` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 11:03:17.693200 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3af1530f-1659-4ee8-ae2d-7c9da37cd179 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 11:03:17.695171 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3af1530f-1659-4ee8-ae2d-7c9da37cd179 admin admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 11:03:17.726646 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 1195/2361] 10.222.0.25 () {60 vars in 1153 bytes} [Sun Aug 7 11:03:17 2022] POST /identity/v3/projects => generated 379 bytes in 102 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 07 11:03:17.728092 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc015680-e75b-428c-ab8d-81dd68d09246 None None] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 11:03:17.740842 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc015680-e75b-428c-ab8d-81dd68d09246 None None] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 11:03:17.800973 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d51c7802-3d33-4df2-bc34-94709259524f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5ac47e3b09f43d19a23a0f457fe9e23', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 11:03:17.801585 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d51c7802-3d33-4df2-bc34-94709259524f admin admin] REQUEST_METHOD: `POST` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 11:03:17.801868 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d51c7802-3d33-4df2-bc34-94709259524f admin admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 11:03:17.802139 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d51c7802-3d33-4df2-bc34-94709259524f admin admin] PATH_INFO: `/v3/users` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 11:03:17.802758 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d51c7802-3d33-4df2-bc34-94709259524f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 11:03:17.804714 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d51c7802-3d33-4df2-bc34-94709259524f admin admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 11:03:17.813079 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 11:03:17.827883 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1167/2362] 10.222.0.25 () {60 vars in 1144 bytes} [Sun Aug 7 11:03:17 2022] POST /identity/v3/users => generated 296 bytes in 100 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 07 11:03:17.831741 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3af1530f-1659-4ee8-ae2d-7c9da37cd179 admin admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 11:03:17.843133 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3af1530f-1659-4ee8-ae2d-7c9da37cd179 admin admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 11:03:17.847903 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc2a0474-ac19-449e-9fba-225e5b9526c3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5ac47e3b09f43d19a23a0f457fe9e23', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 11:03:17.848615 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc2a0474-ac19-449e-9fba-225e5b9526c3 admin admin] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 11:03:17.848896 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc2a0474-ac19-449e-9fba-225e5b9526c3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 11:03:17.849172 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc2a0474-ac19-449e-9fba-225e5b9526c3 admin admin] PATH_INFO: `/v3/roles` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 11:03:17.849816 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc2a0474-ac19-449e-9fba-225e5b9526c3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 11:03:17.851435 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc2a0474-ac19-449e-9fba-225e5b9526c3 admin admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 11:03:17.863529 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 1196/2363] 10.222.0.25 () {58 vars in 1122 bytes} [Sun Aug 7 11:03:17 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 07 11:03:17.868397 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d51c7802-3d33-4df2-bc34-94709259524f admin admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 11:03:17.879531 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d51c7802-3d33-4df2-bc34-94709259524f admin admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 11:03:17.884260 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e85cc3b0-950a-43bc-b4a0-a1ce80400c54 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5ac47e3b09f43d19a23a0f457fe9e23', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 11:03:17.885048 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e85cc3b0-950a-43bc-b4a0-a1ce80400c54 admin admin] REQUEST_METHOD: `PUT` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 11:03:17.885166 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e85cc3b0-950a-43bc-b4a0-a1ce80400c54 admin admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 11:03:17.885453 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e85cc3b0-950a-43bc-b4a0-a1ce80400c54 admin admin] PATH_INFO: `/v3/projects/c324497e9ca249b7ab2439d94e6f9e21/users/d470f1e77b6a4f898658cb90d72fd0dd/roles/81b1f1bcee2a473088fbbd327c1c5b23` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 11:03:17.901674 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e85cc3b0-950a-43bc-b4a0-a1ce80400c54 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c324497e9ca249b7ab2439d94e6f9e21, user_id=d470f1e77b6a4f898658cb90d72fd0dd, role_id=81b1f1bcee2a473088fbbd327c1c5b23)` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 11:03:17.904932 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e85cc3b0-950a-43bc-b4a0-a1ce80400c54 admin admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 11:03:17.917452 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1168/2364] 10.222.0.25 () {60 vars in 1483 bytes} [Sun Aug 7 11:03:17 2022] PUT /identity/v3/projects/c324497e9ca249b7ab2439d94e6f9e21/users/d470f1e77b6a4f898658cb90d72fd0dd/roles/81b1f1bcee2a473088fbbd327c1c5b23 => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 07 11:03:17.923112 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc2a0474-ac19-449e-9fba-225e5b9526c3 admin admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 11:03:17.933222 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc2a0474-ac19-449e-9fba-225e5b9526c3 admin admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 11:03:17.986874 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:DELETE=2,keystone:SELECT=94,keystone:INSERT=4 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 11:03:18.003403 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7a0fe8e-a54e-4a5d-a1e6-8b8a5a6bbb7f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5ac47e3b09f43d19a23a0f457fe9e23', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 11:03:18.004022 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7a0fe8e-a54e-4a5d-a1e6-8b8a5a6bbb7f admin admin] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 11:03:18.004305 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7a0fe8e-a54e-4a5d-a1e6-8b8a5a6bbb7f admin admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 11:03:18.004632 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7a0fe8e-a54e-4a5d-a1e6-8b8a5a6bbb7f admin admin] PATH_INFO: `/v3/roles` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 11:03:18.005975 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7a0fe8e-a54e-4a5d-a1e6-8b8a5a6bbb7f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 11:03:18.006885 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7a0fe8e-a54e-4a5d-a1e6-8b8a5a6bbb7f admin admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 11:03:18.017970 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 1197/2365] 10.222.0.25 () {58 vars in 1122 bytes} [Sun Aug 7 11:03:17 2022] GET /identity/v3/roles => generated 1420 bytes in 96 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 07 11:03:18.022198 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e85cc3b0-950a-43bc-b4a0-a1ce80400c54 admin admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 11:03:18.035202 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e85cc3b0-950a-43bc-b4a0-a1ce80400c54 admin admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 11:03:18.041146 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa3d5b71-566c-4d29-a137-a44a9f42d573 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5ac47e3b09f43d19a23a0f457fe9e23', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 11:03:18.041920 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa3d5b71-566c-4d29-a137-a44a9f42d573 admin admin] REQUEST_METHOD: `PUT` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 11:03:18.042269 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa3d5b71-566c-4d29-a137-a44a9f42d573 admin admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 11:03:18.043249 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa3d5b71-566c-4d29-a137-a44a9f42d573 admin admin] PATH_INFO: `/v3/projects/c324497e9ca249b7ab2439d94e6f9e21/users/d470f1e77b6a4f898658cb90d72fd0dd/roles/81b1f1bcee2a473088fbbd327c1c5b23` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 11:03:18.046262 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa3d5b71-566c-4d29-a137-a44a9f42d573 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c324497e9ca249b7ab2439d94e6f9e21, user_id=d470f1e77b6a4f898658cb90d72fd0dd, role_id=81b1f1bcee2a473088fbbd327c1c5b23)` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 11:03:18.049011 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa3d5b71-566c-4d29-a137-a44a9f42d573 admin admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 11:03:18.064703 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1169/2366] 10.222.0.25 () {60 vars in 1483 bytes} [Sun Aug 7 11:03:18 2022] PUT /identity/v3/projects/c324497e9ca249b7ab2439d94e6f9e21/users/d470f1e77b6a4f898658cb90d72fd0dd/roles/81b1f1bcee2a473088fbbd327c1c5b23 => generated 0 bytes in 43 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 07 11:03:18.076174 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fb2e86a-2d07-4583-a266-7b41a8429815 None None] REQUEST_METHOD: `POST` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 11:03:18.076543 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fb2e86a-2d07-4583-a266-7b41a8429815 None None] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 11:03:18.076897 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fb2e86a-2d07-4583-a266-7b41a8429815 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 11:03:18.113089 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.auth.core [None req-3fb2e86a-2d07-4583-a266-7b41a8429815 None None] MFA Rules not processed for user `d470f1e77b6a4f898658cb90d72fd0dd`. Rule list: `[]` (Enabled: `True`). {{(pid=80144) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 11:03:18.114600 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.fernet_utils [None req-3fb2e86a-2d07-4583-a266-7b41a8429815 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80144) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 11:03:18.196397 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 1198/2367] 10.222.0.25 () {58 vars in 958 bytes} [Sun Aug 7 11:03:18 2022] POST /identity/v3/auth/tokens => generated 3323 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 07 11:03:18.208188 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa3d5b71-566c-4d29-a137-a44a9f42d573 admin admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 11:03:18.209978 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-aa3d5b71-566c-4d29-a137-a44a9f42d573 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 11:03:18.290233 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa3d5b71-566c-4d29-a137-a44a9f42d573 admin admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 11:03:18.293930 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1c3e6f3-8d8a-4dd6-a53c-d04963e6653b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88dbd6dfd1164994a3e16da553561910', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6de1f3cac02c4bcdb05dd2337176033d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 11:03:18.295335 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1c3e6f3-8d8a-4dd6-a53c-d04963e6653b service cinder] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 11:03:18.295632 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1c3e6f3-8d8a-4dd6-a53c-d04963e6653b service cinder] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 11:03:18.295902 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1c3e6f3-8d8a-4dd6-a53c-d04963e6653b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 11:03:18.305078 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1c3e6f3-8d8a-4dd6-a53c-d04963e6653b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 11:03:18.306507 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1c3e6f3-8d8a-4dd6-a53c-d04963e6653b service cinder] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 11:03:18.316026 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1170/2368] 10.222.0.25 () {60 vars in 1365 bytes} [Sun Aug 7 11:03:18 2022] GET /identity/v3/auth/tokens => generated 3323 bytes in 108 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 07 11:03:18.394458 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fb2e86a-2d07-4583-a266-7b41a8429815 None None] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 11:03:18.405765 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fb2e86a-2d07-4583-a266-7b41a8429815 None None] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 11:03:18.415225 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c9061e3-b206-434b-82e7-e6e2c4ab617f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88dbd6dfd1164994a3e16da553561910', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6de1f3cac02c4bcdb05dd2337176033d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 11:03:18.415935 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c9061e3-b206-434b-82e7-e6e2c4ab617f service cinder] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 11:03:18.416221 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c9061e3-b206-434b-82e7-e6e2c4ab617f service cinder] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 11:03:18.416562 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c9061e3-b206-434b-82e7-e6e2c4ab617f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 11:03:18.478437 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c9061e3-b206-434b-82e7-e6e2c4ab617f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 11:03:18.480223 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c9061e3-b206-434b-82e7-e6e2c4ab617f service cinder] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 11:03:18.491922 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 1199/2369] 10.222.0.25 () {60 vars in 1365 bytes} [Sun Aug 7 11:03:18 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 98 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 07 11:03:20.647764 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1c3e6f3-8d8a-4dd6-a53c-d04963e6653b service cinder] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 11:03:20.649207 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-b1c3e6f3-8d8a-4dd6-a53c-d04963e6653b service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 11:03:20.727127 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1c3e6f3-8d8a-4dd6-a53c-d04963e6653b service cinder] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 11:03:20.731689 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a566392-f250-4ece-809a-476ed2a03732 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5ac47e3b09f43d19a23a0f457fe9e23', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 11:03:20.732909 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a566392-f250-4ece-809a-476ed2a03732 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 11:03:20.733220 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a566392-f250-4ece-809a-476ed2a03732 admin admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 11:03:20.733754 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a566392-f250-4ece-809a-476ed2a03732 admin admin] PATH_INFO: `/v3/users/8cb8ad1bff7f4e439cefdf6f0ef9a2d0` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 11:03:20.734886 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a566392-f250-4ece-809a-476ed2a03732 admin admin] RBAC: Authorizing `identity:delete_user(user_id=8cb8ad1bff7f4e439cefdf6f0ef9a2d0)` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 11:03:20.736835 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a566392-f250-4ece-809a-476ed2a03732 admin admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 11:03:20.777251 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:INSERT=5,keystone:SELECT=90,keystone:DELETE=5 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 11:03:20.786908 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-0a566392-f250-4ece-809a-476ed2a03732 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8cb8ad1bff7f4e439cefdf6f0ef9a2d0'} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 11:03:20.795376 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-0a566392-f250-4ece-809a-476ed2a03732 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8cb8ad1bff7f4e439cefdf6f0ef9a2d0'} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 11:03:20.807642 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-0a566392-f250-4ece-809a-476ed2a03732 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8cb8ad1bff7f4e439cefdf6f0ef9a2d0'} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 11:03:20.808431 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-0a566392-f250-4ece-809a-476ed2a03732 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8cb8ad1bff7f4e439cefdf6f0ef9a2d0'} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 11:03:20.818500 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-0a566392-f250-4ece-809a-476ed2a03732 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8cb8ad1bff7f4e439cefdf6f0ef9a2d0'} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 11:03:20.824536 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1171/2370] 10.222.0.25 () {58 vars in 1224 bytes} [Sun Aug 7 11:03:20 2022] DELETE /identity/v3/users/8cb8ad1bff7f4e439cefdf6f0ef9a2d0 => generated 0 bytes in 177 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 07 11:03:21.002833 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c9061e3-b206-434b-82e7-e6e2c4ab617f service cinder] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 11:03:21.015500 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c9061e3-b206-434b-82e7-e6e2c4ab617f service cinder] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 11:03:21.073669 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a864ffae-601f-4be2-9b95-d6992836f457 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5ac47e3b09f43d19a23a0f457fe9e23', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 11:03:21.074302 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a864ffae-601f-4be2-9b95-d6992836f457 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 11:03:21.074578 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a864ffae-601f-4be2-9b95-d6992836f457 admin admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 11:03:21.074846 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a864ffae-601f-4be2-9b95-d6992836f457 admin admin] PATH_INFO: `/v3/projects/758665157bfd4f7fa6219f898a9c99a2` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 11:03:21.076121 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a864ffae-601f-4be2-9b95-d6992836f457 admin admin] RBAC: Authorizing `identity:delete_project(project_id=758665157bfd4f7fa6219f898a9c99a2)` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 11:03:21.078028 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a864ffae-601f-4be2-9b95-d6992836f457 admin admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 11:03:21.093925 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=100 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 11:03:21.124791 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.notifications [None req-a864ffae-601f-4be2-9b95-d6992836f457 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '758665157bfd4f7fa6219f898a9c99a2', 'request_id': 'req-a864ffae-601f-4be2-9b95-d6992836f457'} {{(pid=80144) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 11:03:21.130458 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.notifications [None req-a864ffae-601f-4be2-9b95-d6992836f457 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '758665157bfd4f7fa6219f898a9c99a2', 'request_id': 'req-a864ffae-601f-4be2-9b95-d6992836f457'} {{(pid=80144) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 11:03:21.138301 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.notifications [None req-a864ffae-601f-4be2-9b95-d6992836f457 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '758665157bfd4f7fa6219f898a9c99a2', 'request_id': 'req-a864ffae-601f-4be2-9b95-d6992836f457'} {{(pid=80144) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 11:03:21.150069 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.notifications [None req-a864ffae-601f-4be2-9b95-d6992836f457 admin admin] The token cache is being invalidate because project 758665157bfd4f7fa6219f898a9c99a2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80144) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 07 11:03:21.150362 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.notifications [None req-a864ffae-601f-4be2-9b95-d6992836f457 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80144) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 11:03:21.151039 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.notifications [None req-a864ffae-601f-4be2-9b95-d6992836f457 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80144) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 11:03:21.152545 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 1200/2371] 10.222.0.25 () {58 vars in 1233 bytes} [Sun Aug 7 11:03:21 2022] DELETE /identity/v3/projects/758665157bfd4f7fa6219f898a9c99a2 => generated 0 bytes in 150 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 07 11:03:21.157139 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a566392-f250-4ece-809a-476ed2a03732 admin admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 11:03:21.158609 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-0a566392-f250-4ece-809a-476ed2a03732 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 11:03:21.172866 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a566392-f250-4ece-809a-476ed2a03732 admin admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 11:03:21.237349 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5789aa4-76fa-49a2-ac3b-bcf109e9e1da admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5ac47e3b09f43d19a23a0f457fe9e23', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 11:03:21.238016 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5789aa4-76fa-49a2-ac3b-bcf109e9e1da admin admin] REQUEST_METHOD: `DELETE` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 11:03:21.238297 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5789aa4-76fa-49a2-ac3b-bcf109e9e1da admin admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 11:03:21.238573 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5789aa4-76fa-49a2-ac3b-bcf109e9e1da admin admin] PATH_INFO: `/v3/users/fca1beb3c14b4c6f8085a09697736f51` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 11:03:21.239797 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5789aa4-76fa-49a2-ac3b-bcf109e9e1da admin admin] RBAC: Authorizing `identity:delete_user(user_id=fca1beb3c14b4c6f8085a09697736f51)` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 11:03:21.241715 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5789aa4-76fa-49a2-ac3b-bcf109e9e1da admin admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 11:03:21.295251 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-b5789aa4-76fa-49a2-ac3b-bcf109e9e1da admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'fca1beb3c14b4c6f8085a09697736f51'} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 11:03:21.303747 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-b5789aa4-76fa-49a2-ac3b-bcf109e9e1da admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'fca1beb3c14b4c6f8085a09697736f51'} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 11:03:21.318271 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-b5789aa4-76fa-49a2-ac3b-bcf109e9e1da admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'fca1beb3c14b4c6f8085a09697736f51'} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 11:03:21.319039 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-b5789aa4-76fa-49a2-ac3b-bcf109e9e1da admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'fca1beb3c14b4c6f8085a09697736f51'} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 11:03:21.326821 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-b5789aa4-76fa-49a2-ac3b-bcf109e9e1da admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'fca1beb3c14b4c6f8085a09697736f51'} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 11:03:21.333207 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1172/2372] 10.222.0.25 () {58 vars in 1224 bytes} [Sun Aug 7 11:03:21 2022] DELETE /identity/v3/users/fca1beb3c14b4c6f8085a09697736f51 => generated 0 bytes in 177 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 07 11:03:21.527387 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a864ffae-601f-4be2-9b95-d6992836f457 admin admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 11:03:21.529796 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.fernet_utils [None req-a864ffae-601f-4be2-9b95-d6992836f457 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80144) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 11:03:21.545858 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a864ffae-601f-4be2-9b95-d6992836f457 admin admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 11:03:21.614957 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce7d5635-2c3b-4147-b111-6e262909fa31 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5ac47e3b09f43d19a23a0f457fe9e23', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 11:03:21.615616 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce7d5635-2c3b-4147-b111-6e262909fa31 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 11:03:21.615897 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce7d5635-2c3b-4147-b111-6e262909fa31 admin admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 11:03:21.616171 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce7d5635-2c3b-4147-b111-6e262909fa31 admin admin] PATH_INFO: `/v3/projects/25a95f83a19142bf817615584a5c5257` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 11:03:21.617909 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce7d5635-2c3b-4147-b111-6e262909fa31 admin admin] RBAC: Authorizing `identity:delete_project(project_id=25a95f83a19142bf817615584a5c5257)` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 11:03:21.619835 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce7d5635-2c3b-4147-b111-6e262909fa31 admin admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 11:03:21.668427 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.notifications [None req-ce7d5635-2c3b-4147-b111-6e262909fa31 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '25a95f83a19142bf817615584a5c5257', 'request_id': 'req-ce7d5635-2c3b-4147-b111-6e262909fa31'} {{(pid=80144) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 11:03:21.674589 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.notifications [None req-ce7d5635-2c3b-4147-b111-6e262909fa31 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '25a95f83a19142bf817615584a5c5257', 'request_id': 'req-ce7d5635-2c3b-4147-b111-6e262909fa31'} {{(pid=80144) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 11:03:21.685799 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.notifications [None req-ce7d5635-2c3b-4147-b111-6e262909fa31 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '25a95f83a19142bf817615584a5c5257', 'request_id': 'req-ce7d5635-2c3b-4147-b111-6e262909fa31'} {{(pid=80144) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 11:03:21.697154 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.notifications [None req-ce7d5635-2c3b-4147-b111-6e262909fa31 admin admin] The token cache is being invalidate because project 25a95f83a19142bf817615584a5c5257 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80144) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 07 11:03:21.697436 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.notifications [None req-ce7d5635-2c3b-4147-b111-6e262909fa31 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80144) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 11:03:21.698086 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.notifications [None req-ce7d5635-2c3b-4147-b111-6e262909fa31 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80144) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 11:03:21.699823 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 1201/2373] 10.222.0.25 () {58 vars in 1233 bytes} [Sun Aug 7 11:03:21 2022] DELETE /identity/v3/projects/25a95f83a19142bf817615584a5c5257 => generated 0 bytes in 173 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 07 11:03:29.535231 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5789aa4-76fa-49a2-ac3b-bcf109e9e1da admin admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 11:03:29.536696 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-b5789aa4-76fa-49a2-ac3b-bcf109e9e1da admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 11:03:29.552083 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5789aa4-76fa-49a2-ac3b-bcf109e9e1da admin admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 11:03:29.588167 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=83,keystone:DELETE=15,keystone:INSERT=2 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 11:03:29.619571 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6043b68a-0751-4c9d-bda2-6feeb1110f0f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5ac47e3b09f43d19a23a0f457fe9e23', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 11:03:29.620124 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6043b68a-0751-4c9d-bda2-6feeb1110f0f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 11:03:29.620409 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6043b68a-0751-4c9d-bda2-6feeb1110f0f admin admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 11:03:29.620685 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6043b68a-0751-4c9d-bda2-6feeb1110f0f admin admin] PATH_INFO: `/v3/users/e274f6fe64a7479bb55a7b9dd1470e85` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 11:03:29.621847 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6043b68a-0751-4c9d-bda2-6feeb1110f0f admin admin] RBAC: Authorizing `identity:delete_user(user_id=e274f6fe64a7479bb55a7b9dd1470e85)` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 11:03:29.624080 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6043b68a-0751-4c9d-bda2-6feeb1110f0f admin admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 11:03:29.683065 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-6043b68a-0751-4c9d-bda2-6feeb1110f0f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e274f6fe64a7479bb55a7b9dd1470e85'} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 11:03:29.692590 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-6043b68a-0751-4c9d-bda2-6feeb1110f0f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e274f6fe64a7479bb55a7b9dd1470e85'} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 11:03:29.707294 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-6043b68a-0751-4c9d-bda2-6feeb1110f0f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e274f6fe64a7479bb55a7b9dd1470e85'} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 11:03:29.708331 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-6043b68a-0751-4c9d-bda2-6feeb1110f0f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e274f6fe64a7479bb55a7b9dd1470e85'} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 11:03:29.719868 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-6043b68a-0751-4c9d-bda2-6feeb1110f0f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e274f6fe64a7479bb55a7b9dd1470e85'} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 11:03:29.726260 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1173/2374] 10.222.0.25 () {58 vars in 1224 bytes} [Sun Aug 7 11:03:29 2022] DELETE /identity/v3/users/e274f6fe64a7479bb55a7b9dd1470e85 => generated 0 bytes in 192 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 07 11:03:29.730857 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2a47631-0007-47e0-b164-5799ab017f6f None None] REQUEST_METHOD: `POST` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 11:03:29.731845 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2a47631-0007-47e0-b164-5799ab017f6f None None] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 11:03:29.731845 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2a47631-0007-47e0-b164-5799ab017f6f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 11:03:29.753917 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.auth.core [None req-b2a47631-0007-47e0-b164-5799ab017f6f None None] MFA Rules not processed for user `bac4e6b76fe64217ac8fa44e71cffe6a`. Rule list: `[]` (Enabled: `True`). {{(pid=80144) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 11:03:29.755226 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.fernet_utils [None req-b2a47631-0007-47e0-b164-5799ab017f6f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80144) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 11:03:29.802496 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 11:03:29.830836 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 1202/2375] 10.222.0.25 () {58 vars in 958 bytes} [Sun Aug 7 11:03:29 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 07 11:03:29.841138 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6043b68a-0751-4c9d-bda2-6feeb1110f0f admin admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 11:03:29.842596 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-6043b68a-0751-4c9d-bda2-6feeb1110f0f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 11:03:29.909845 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6043b68a-0751-4c9d-bda2-6feeb1110f0f admin admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 11:03:29.913548 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83eded7c-0fb4-4ca1-841b-7bf0f8f7d44c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be1283c9bf47424c9aa3ad27cab03027', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6de1f3cac02c4bcdb05dd2337176033d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 11:03:29.914162 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83eded7c-0fb4-4ca1-841b-7bf0f8f7d44c service neutron] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 11:03:29.914445 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83eded7c-0fb4-4ca1-841b-7bf0f8f7d44c service neutron] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 11:03:29.914720 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83eded7c-0fb4-4ca1-841b-7bf0f8f7d44c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 11:03:29.924555 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83eded7c-0fb4-4ca1-841b-7bf0f8f7d44c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 11:03:29.925978 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83eded7c-0fb4-4ca1-841b-7bf0f8f7d44c service neutron] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 11:03:29.936371 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1174/2376] 10.222.0.25 () {60 vars in 1365 bytes} [Sun Aug 7 11:03:29 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 07 11:03:30.220945 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2a47631-0007-47e0-b164-5799ab017f6f None None] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 11:03:30.222496 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.fernet_utils [None req-b2a47631-0007-47e0-b164-5799ab017f6f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80144) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 11:03:30.236150 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2a47631-0007-47e0-b164-5799ab017f6f None None] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 11:03:30.241051 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d1bcf9d-b076-4e42-ba07-8cb607323bc0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5ac47e3b09f43d19a23a0f457fe9e23', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 11:03:30.241690 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d1bcf9d-b076-4e42-ba07-8cb607323bc0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 11:03:30.241967 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d1bcf9d-b076-4e42-ba07-8cb607323bc0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 11:03:30.242236 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d1bcf9d-b076-4e42-ba07-8cb607323bc0 admin admin] PATH_INFO: `/v3/projects/7d2f9bfc71a24e038e555a188b933b23` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 11:03:30.243426 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d1bcf9d-b076-4e42-ba07-8cb607323bc0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=7d2f9bfc71a24e038e555a188b933b23)` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 11:03:30.245314 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d1bcf9d-b076-4e42-ba07-8cb607323bc0 admin admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 11:03:30.293560 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.notifications [None req-2d1bcf9d-b076-4e42-ba07-8cb607323bc0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7d2f9bfc71a24e038e555a188b933b23', 'request_id': 'req-2d1bcf9d-b076-4e42-ba07-8cb607323bc0'} {{(pid=80144) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 11:03:30.299115 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.notifications [None req-2d1bcf9d-b076-4e42-ba07-8cb607323bc0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7d2f9bfc71a24e038e555a188b933b23', 'request_id': 'req-2d1bcf9d-b076-4e42-ba07-8cb607323bc0'} {{(pid=80144) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 11:03:30.307554 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.notifications [None req-2d1bcf9d-b076-4e42-ba07-8cb607323bc0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7d2f9bfc71a24e038e555a188b933b23', 'request_id': 'req-2d1bcf9d-b076-4e42-ba07-8cb607323bc0'} {{(pid=80144) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 11:03:30.319360 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.notifications [None req-2d1bcf9d-b076-4e42-ba07-8cb607323bc0 admin admin] The token cache is being invalidate because project 7d2f9bfc71a24e038e555a188b933b23 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80144) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 07 11:03:30.319648 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.notifications [None req-2d1bcf9d-b076-4e42-ba07-8cb607323bc0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80144) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 11:03:30.320268 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.notifications [None req-2d1bcf9d-b076-4e42-ba07-8cb607323bc0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80144) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 11:03:30.321720 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 1203/2377] 10.222.0.25 () {58 vars in 1233 bytes} [Sun Aug 7 11:03:30 2022] DELETE /identity/v3/projects/7d2f9bfc71a24e038e555a188b933b23 => generated 0 bytes in 101 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 07 11:03:30.325851 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83eded7c-0fb4-4ca1-841b-7bf0f8f7d44c service neutron] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 11:03:30.336084 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83eded7c-0fb4-4ca1-841b-7bf0f8f7d44c service neutron] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 11:03:30.384432 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 11:03:30.396964 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6e74f0f-0bb8-4aca-80b8-af074c4bc6de admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5ac47e3b09f43d19a23a0f457fe9e23', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 11:03:30.397614 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6e74f0f-0bb8-4aca-80b8-af074c4bc6de admin admin] REQUEST_METHOD: `DELETE` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 11:03:30.398236 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6e74f0f-0bb8-4aca-80b8-af074c4bc6de admin admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 11:03:30.398660 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6e74f0f-0bb8-4aca-80b8-af074c4bc6de admin admin] PATH_INFO: `/v3/users/d470f1e77b6a4f898658cb90d72fd0dd` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 11:03:30.399965 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6e74f0f-0bb8-4aca-80b8-af074c4bc6de admin admin] RBAC: Authorizing `identity:delete_user(user_id=d470f1e77b6a4f898658cb90d72fd0dd)` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 11:03:30.402485 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6e74f0f-0bb8-4aca-80b8-af074c4bc6de admin admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 11:03:30.448474 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-b6e74f0f-0bb8-4aca-80b8-af074c4bc6de admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd470f1e77b6a4f898658cb90d72fd0dd'} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 11:03:30.456379 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-b6e74f0f-0bb8-4aca-80b8-af074c4bc6de admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd470f1e77b6a4f898658cb90d72fd0dd'} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 11:03:30.468539 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-b6e74f0f-0bb8-4aca-80b8-af074c4bc6de admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd470f1e77b6a4f898658cb90d72fd0dd'} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 11:03:30.469219 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-b6e74f0f-0bb8-4aca-80b8-af074c4bc6de admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd470f1e77b6a4f898658cb90d72fd0dd'} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 11:03:30.476836 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-b6e74f0f-0bb8-4aca-80b8-af074c4bc6de admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd470f1e77b6a4f898658cb90d72fd0dd'} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 11:03:30.483304 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1175/2378] 10.222.0.25 () {58 vars in 1224 bytes} [Sun Aug 7 11:03:30 2022] DELETE /identity/v3/users/d470f1e77b6a4f898658cb90d72fd0dd => generated 0 bytes in 158 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 07 11:03:30.786616 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d1bcf9d-b076-4e42-ba07-8cb607323bc0 admin admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 11:03:30.788468 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.fernet_utils [None req-2d1bcf9d-b076-4e42-ba07-8cb607323bc0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80144) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 11:03:30.804140 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d1bcf9d-b076-4e42-ba07-8cb607323bc0 admin admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 11:03:30.866787 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f4fa948-a4f1-41ae-a170-691fca8d7c0b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5ac47e3b09f43d19a23a0f457fe9e23', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 11:03:30.867516 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f4fa948-a4f1-41ae-a170-691fca8d7c0b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 11:03:30.867761 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f4fa948-a4f1-41ae-a170-691fca8d7c0b admin admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 11:03:30.868022 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f4fa948-a4f1-41ae-a170-691fca8d7c0b admin admin] PATH_INFO: `/v3/projects/c324497e9ca249b7ab2439d94e6f9e21` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 11:03:30.869150 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f4fa948-a4f1-41ae-a170-691fca8d7c0b admin admin] RBAC: Authorizing `identity:delete_project(project_id=c324497e9ca249b7ab2439d94e6f9e21)` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 11:03:30.871093 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f4fa948-a4f1-41ae-a170-691fca8d7c0b admin admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 11:03:30.924525 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.notifications [None req-0f4fa948-a4f1-41ae-a170-691fca8d7c0b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c324497e9ca249b7ab2439d94e6f9e21', 'request_id': 'req-0f4fa948-a4f1-41ae-a170-691fca8d7c0b'} {{(pid=80144) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 11:03:30.930657 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.notifications [None req-0f4fa948-a4f1-41ae-a170-691fca8d7c0b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c324497e9ca249b7ab2439d94e6f9e21', 'request_id': 'req-0f4fa948-a4f1-41ae-a170-691fca8d7c0b'} {{(pid=80144) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 11:03:30.932318 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=86,keystone:DELETE=13,keystone:INSERT=1 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 11:03:30.939698 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.notifications [None req-0f4fa948-a4f1-41ae-a170-691fca8d7c0b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c324497e9ca249b7ab2439d94e6f9e21', 'request_id': 'req-0f4fa948-a4f1-41ae-a170-691fca8d7c0b'} {{(pid=80144) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 11:03:30.951815 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.notifications [None req-0f4fa948-a4f1-41ae-a170-691fca8d7c0b admin admin] The token cache is being invalidate because project c324497e9ca249b7ab2439d94e6f9e21 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80144) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 07 11:03:30.952101 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.notifications [None req-0f4fa948-a4f1-41ae-a170-691fca8d7c0b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80144) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 11:03:30.952754 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.notifications [None req-0f4fa948-a4f1-41ae-a170-691fca8d7c0b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80144) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 11:03:30.954247 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 1204/2379] 10.222.0.25 () {58 vars in 1233 bytes} [Sun Aug 7 11:03:30 2022] DELETE /identity/v3/projects/c324497e9ca249b7ab2439d94e6f9e21 => generated 0 bytes in 168 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 07 11:03:30.959728 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6caced9-3ac3-4b99-9bc0-677846160e1b None None] REQUEST_METHOD: `POST` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 11:03:30.960202 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6caced9-3ac3-4b99-9bc0-677846160e1b None None] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 11:03:30.960202 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6caced9-3ac3-4b99-9bc0-677846160e1b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 11:03:30.980501 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.auth.core [None req-b6caced9-3ac3-4b99-9bc0-677846160e1b None None] MFA Rules not processed for user `bac4e6b76fe64217ac8fa44e71cffe6a`. Rule list: `[]` (Enabled: `True`). {{(pid=80143) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 11:03:30.982321 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-b6caced9-3ac3-4b99-9bc0-677846160e1b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 11:03:31.047439 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1176/2380] 10.222.0.25 () {58 vars in 958 bytes} [Sun Aug 7 11:03:30 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 07 11:03:31.055837 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6011ad2e-4526-4fbb-b1ce-fd5738d51552 None None] REQUEST_METHOD: `POST` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 11:03:31.056129 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6011ad2e-4526-4fbb-b1ce-fd5738d51552 None None] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 11:03:31.056407 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6011ad2e-4526-4fbb-b1ce-fd5738d51552 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 11:03:31.078012 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.auth.core [None req-6011ad2e-4526-4fbb-b1ce-fd5738d51552 None None] MFA Rules not processed for user `bac4e6b76fe64217ac8fa44e71cffe6a`. Rule list: `[]` (Enabled: `True`). {{(pid=80144) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 11:03:31.079543 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.fernet_utils [None req-6011ad2e-4526-4fbb-b1ce-fd5738d51552 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80144) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 11:03:31.088778 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 1205/2381] 10.222.0.25 () {58 vars in 958 bytes} [Sun Aug 7 11:03:31 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 07 11:03:31.094597 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6caced9-3ac3-4b99-9bc0-677846160e1b None None] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 11:03:31.104895 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6caced9-3ac3-4b99-9bc0-677846160e1b None None] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 11:03:31.110235 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b089ce5b-fb6a-444b-9336-43a0c5a734da admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5ac47e3b09f43d19a23a0f457fe9e23', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 11:03:31.111289 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b089ce5b-fb6a-444b-9336-43a0c5a734da admin admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 11:03:31.111773 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b089ce5b-fb6a-444b-9336-43a0c5a734da admin admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 11:03:31.112191 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b089ce5b-fb6a-444b-9336-43a0c5a734da admin admin] PATH_INFO: `/v3/domains` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 11:03:31.113096 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b089ce5b-fb6a-444b-9336-43a0c5a734da admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80143) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 11:03:31.113439 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b089ce5b-fb6a-444b-9336-43a0c5a734da admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 11:03:31.115638 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b089ce5b-fb6a-444b-9336-43a0c5a734da admin admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 11:03:31.133481 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1177/2382] 10.222.0.25 () {58 vars in 1153 bytes} [Sun Aug 7 11:03:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 11:03:31.134728 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6011ad2e-4526-4fbb-b1ce-fd5738d51552 None None] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 11:03:31.144679 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6011ad2e-4526-4fbb-b1ce-fd5738d51552 None None] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 11:03:31.149717 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07d65bd1-6910-401c-aabc-77b7d1c21ff8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5ac47e3b09f43d19a23a0f457fe9e23', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 11:03:31.150322 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07d65bd1-6910-401c-aabc-77b7d1c21ff8 admin admin] REQUEST_METHOD: `POST` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 11:03:31.150674 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07d65bd1-6910-401c-aabc-77b7d1c21ff8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 11:03:31.151146 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07d65bd1-6910-401c-aabc-77b7d1c21ff8 admin admin] PATH_INFO: `/v3/projects` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 11:03:31.151954 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07d65bd1-6910-401c-aabc-77b7d1c21ff8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 11:03:31.154268 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07d65bd1-6910-401c-aabc-77b7d1c21ff8 admin admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 11:03:31.185593 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 1206/2383] 10.222.0.25 () {60 vars in 1153 bytes} [Sun Aug 7 11:03:31 2022] POST /identity/v3/projects => generated 379 bytes in 51 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 07 11:03:31.189897 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b089ce5b-fb6a-444b-9336-43a0c5a734da admin admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 11:03:31.199894 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b089ce5b-fb6a-444b-9336-43a0c5a734da admin admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 11:03:31.257726 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-795b67c7-cd69-489e-ad22-8d21dda8bde7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5ac47e3b09f43d19a23a0f457fe9e23', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 11:03:31.258350 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-795b67c7-cd69-489e-ad22-8d21dda8bde7 admin admin] REQUEST_METHOD: `POST` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 11:03:31.258628 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-795b67c7-cd69-489e-ad22-8d21dda8bde7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 11:03:31.258901 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-795b67c7-cd69-489e-ad22-8d21dda8bde7 admin admin] PATH_INFO: `/v3/users` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 11:03:31.259561 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-795b67c7-cd69-489e-ad22-8d21dda8bde7 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 11:03:31.267226 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-795b67c7-cd69-489e-ad22-8d21dda8bde7 admin admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 11:03:31.276515 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=88,keystone:DELETE=10,keystone:INSERT=2 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 11:03:31.290902 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1178/2384] 10.222.0.25 () {60 vars in 1144 bytes} [Sun Aug 7 11:03:31 2022] POST /identity/v3/users => generated 290 bytes in 102 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 07 11:03:31.295334 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07d65bd1-6910-401c-aabc-77b7d1c21ff8 admin admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 11:03:31.305789 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07d65bd1-6910-401c-aabc-77b7d1c21ff8 admin admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 11:03:31.311959 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4ef9be2-60b6-4355-9b3e-7dfab1c59132 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5ac47e3b09f43d19a23a0f457fe9e23', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 11:03:31.312560 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4ef9be2-60b6-4355-9b3e-7dfab1c59132 admin admin] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 11:03:31.312844 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4ef9be2-60b6-4355-9b3e-7dfab1c59132 admin admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 11:03:31.313121 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4ef9be2-60b6-4355-9b3e-7dfab1c59132 admin admin] PATH_INFO: `/v3/roles` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 11:03:31.313764 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4ef9be2-60b6-4355-9b3e-7dfab1c59132 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 11:03:31.315396 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4ef9be2-60b6-4355-9b3e-7dfab1c59132 admin admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 11:03:31.326770 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 1207/2385] 10.222.0.25 () {58 vars in 1122 bytes} [Sun Aug 7 11:03:31 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 07 11:03:31.330857 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-795b67c7-cd69-489e-ad22-8d21dda8bde7 admin admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 11:03:31.341139 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-795b67c7-cd69-489e-ad22-8d21dda8bde7 admin admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 11:03:31.345906 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29dbc0d8-4833-40da-9611-0b49291a3d9d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5ac47e3b09f43d19a23a0f457fe9e23', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 11:03:31.346514 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29dbc0d8-4833-40da-9611-0b49291a3d9d admin admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 11:03:31.346789 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29dbc0d8-4833-40da-9611-0b49291a3d9d admin admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 11:03:31.347097 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29dbc0d8-4833-40da-9611-0b49291a3d9d admin admin] PATH_INFO: `/v3/roles` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 11:03:31.347741 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29dbc0d8-4833-40da-9611-0b49291a3d9d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 11:03:31.349334 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29dbc0d8-4833-40da-9611-0b49291a3d9d admin admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 11:03:31.361068 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1179/2386] 10.222.0.25 () {58 vars in 1122 bytes} [Sun Aug 7 11:03:31 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 07 11:03:31.365722 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4ef9be2-60b6-4355-9b3e-7dfab1c59132 admin admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 11:03:31.379588 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4ef9be2-60b6-4355-9b3e-7dfab1c59132 admin admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 11:03:31.386275 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f071886-1acb-4ed1-8cd7-70145dd1ad14 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5ac47e3b09f43d19a23a0f457fe9e23', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 11:03:31.387484 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f071886-1acb-4ed1-8cd7-70145dd1ad14 admin admin] REQUEST_METHOD: `PUT` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 11:03:31.388199 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f071886-1acb-4ed1-8cd7-70145dd1ad14 admin admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 11:03:31.388767 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f071886-1acb-4ed1-8cd7-70145dd1ad14 admin admin] PATH_INFO: `/v3/projects/3ee74081f1a64a409a47f437da5bacec/users/0b89f8f6c4ab49df8fa2bf649233f5ed/roles/b98dfb3b9fee4644af2887a3ac3fd2e6` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 11:03:31.405704 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f071886-1acb-4ed1-8cd7-70145dd1ad14 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3ee74081f1a64a409a47f437da5bacec, user_id=0b89f8f6c4ab49df8fa2bf649233f5ed, role_id=b98dfb3b9fee4644af2887a3ac3fd2e6)` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 11:03:31.408376 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f071886-1acb-4ed1-8cd7-70145dd1ad14 admin admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 11:03:31.423500 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 1208/2387] 10.222.0.25 () {60 vars in 1483 bytes} [Sun Aug 7 11:03:31 2022] PUT /identity/v3/projects/3ee74081f1a64a409a47f437da5bacec/users/0b89f8f6c4ab49df8fa2bf649233f5ed/roles/b98dfb3b9fee4644af2887a3ac3fd2e6 => generated 0 bytes in 59 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 07 11:03:31.434546 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d1d13ee-cf19-4ebb-8b3a-f91ff446bc49 None None] REQUEST_METHOD: `POST` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 11:03:31.435768 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d1d13ee-cf19-4ebb-8b3a-f91ff446bc49 None None] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 11:03:31.435768 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d1d13ee-cf19-4ebb-8b3a-f91ff446bc49 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 11:03:31.462706 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.auth.core [None req-1d1d13ee-cf19-4ebb-8b3a-f91ff446bc49 None None] MFA Rules not processed for user `0b89f8f6c4ab49df8fa2bf649233f5ed`. Rule list: `[]` (Enabled: `True`). {{(pid=80143) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 11:03:31.463968 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-1d1d13ee-cf19-4ebb-8b3a-f91ff446bc49 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 11:03:31.530237 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1180/2388] 10.222.0.25 () {58 vars in 958 bytes} [Sun Aug 7 11:03:31 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 07 11:03:31.539985 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f071886-1acb-4ed1-8cd7-70145dd1ad14 admin admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 11:03:31.603217 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f071886-1acb-4ed1-8cd7-70145dd1ad14 admin admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 11:03:31.608600 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c81a1c06-aeef-4f2f-b72c-4a1ec8efb0ca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5ac47e3b09f43d19a23a0f457fe9e23', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 11:03:31.609204 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c81a1c06-aeef-4f2f-b72c-4a1ec8efb0ca admin admin] REQUEST_METHOD: `POST` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 11:03:31.609487 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c81a1c06-aeef-4f2f-b72c-4a1ec8efb0ca admin admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 11:03:31.609759 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c81a1c06-aeef-4f2f-b72c-4a1ec8efb0ca admin admin] PATH_INFO: `/v3/projects` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 11:03:31.610374 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c81a1c06-aeef-4f2f-b72c-4a1ec8efb0ca admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 11:03:31.612280 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c81a1c06-aeef-4f2f-b72c-4a1ec8efb0ca admin admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 11:03:31.641300 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 1209/2389] 10.222.0.25 () {60 vars in 1153 bytes} [Sun Aug 7 11:03:31 2022] POST /identity/v3/projects => generated 381 bytes in 107 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 07 11:03:31.645572 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d1d13ee-cf19-4ebb-8b3a-f91ff446bc49 None None] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 11:03:31.656449 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d1d13ee-cf19-4ebb-8b3a-f91ff446bc49 None None] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 11:03:31.712174 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6e7ed8e-1c11-43a2-b72f-b11e9892a64e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5ac47e3b09f43d19a23a0f457fe9e23', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 11:03:31.712783 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6e7ed8e-1c11-43a2-b72f-b11e9892a64e admin admin] REQUEST_METHOD: `POST` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 11:03:31.713067 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6e7ed8e-1c11-43a2-b72f-b11e9892a64e admin admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 11:03:31.713345 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6e7ed8e-1c11-43a2-b72f-b11e9892a64e admin admin] PATH_INFO: `/v3/users` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 11:03:31.713974 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6e7ed8e-1c11-43a2-b72f-b11e9892a64e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 11:03:31.716509 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6e7ed8e-1c11-43a2-b72f-b11e9892a64e admin admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 11:03:31.737810 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1181/2390] 10.222.0.25 () {60 vars in 1144 bytes} [Sun Aug 7 11:03:31 2022] POST /identity/v3/users => generated 297 bytes in 93 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 07 11:03:31.741824 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c81a1c06-aeef-4f2f-b72c-4a1ec8efb0ca admin admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 11:03:31.752242 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c81a1c06-aeef-4f2f-b72c-4a1ec8efb0ca admin admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 11:03:31.756905 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10e8a1fa-d38b-4c03-9532-2f4f73256e46 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5ac47e3b09f43d19a23a0f457fe9e23', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 11:03:31.757501 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10e8a1fa-d38b-4c03-9532-2f4f73256e46 admin admin] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 11:03:31.757781 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10e8a1fa-d38b-4c03-9532-2f4f73256e46 admin admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 11:03:31.758057 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10e8a1fa-d38b-4c03-9532-2f4f73256e46 admin admin] PATH_INFO: `/v3/roles` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 11:03:31.758696 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10e8a1fa-d38b-4c03-9532-2f4f73256e46 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 11:03:31.760453 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10e8a1fa-d38b-4c03-9532-2f4f73256e46 admin admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 11:03:31.771094 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 1210/2391] 10.222.0.25 () {58 vars in 1122 bytes} [Sun Aug 7 11:03:31 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 07 11:03:31.775164 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6e7ed8e-1c11-43a2-b72f-b11e9892a64e admin admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 11:03:31.785891 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6e7ed8e-1c11-43a2-b72f-b11e9892a64e admin admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 11:03:31.790503 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8868faf0-1bc2-4bd4-8958-2a8ea338a47a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5ac47e3b09f43d19a23a0f457fe9e23', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 11:03:31.791186 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8868faf0-1bc2-4bd4-8958-2a8ea338a47a admin admin] REQUEST_METHOD: `PUT` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 11:03:31.791470 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8868faf0-1bc2-4bd4-8958-2a8ea338a47a admin admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 11:03:31.791744 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8868faf0-1bc2-4bd4-8958-2a8ea338a47a admin admin] PATH_INFO: `/v3/projects/f1acd5499a88421eb87a071465bac871/users/1af5b0cf121a44f985f447761d35e53c/roles/81b1f1bcee2a473088fbbd327c1c5b23` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 11:03:31.808022 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8868faf0-1bc2-4bd4-8958-2a8ea338a47a admin admin] RBAC: Authorizing `identity:create_grant(project_id=f1acd5499a88421eb87a071465bac871, user_id=1af5b0cf121a44f985f447761d35e53c, role_id=81b1f1bcee2a473088fbbd327c1c5b23)` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 11:03:31.811443 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8868faf0-1bc2-4bd4-8958-2a8ea338a47a admin admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 11:03:31.828564 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1182/2392] 10.222.0.25 () {60 vars in 1483 bytes} [Sun Aug 7 11:03:31 2022] PUT /identity/v3/projects/f1acd5499a88421eb87a071465bac871/users/1af5b0cf121a44f985f447761d35e53c/roles/81b1f1bcee2a473088fbbd327c1c5b23 => generated 0 bytes in 54 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 07 11:03:31.832539 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10e8a1fa-d38b-4c03-9532-2f4f73256e46 admin admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 11:03:31.843132 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10e8a1fa-d38b-4c03-9532-2f4f73256e46 admin admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 11:03:31.892327 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:INSERT=4,keystone:DELETE=1,keystone:SELECT=95 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 11:03:31.898939 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbe86754-d28f-42c7-956d-d42fad0f0193 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5ac47e3b09f43d19a23a0f457fe9e23', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 11:03:31.899809 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbe86754-d28f-42c7-956d-d42fad0f0193 admin admin] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 11:03:31.900249 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbe86754-d28f-42c7-956d-d42fad0f0193 admin admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 11:03:31.900877 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbe86754-d28f-42c7-956d-d42fad0f0193 admin admin] PATH_INFO: `/v3/roles` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 11:03:31.901637 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbe86754-d28f-42c7-956d-d42fad0f0193 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 11:03:31.906107 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbe86754-d28f-42c7-956d-d42fad0f0193 admin admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 11:03:31.914248 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 1211/2393] 10.222.0.25 () {58 vars in 1122 bytes} [Sun Aug 7 11:03:31 2022] GET /identity/v3/roles => generated 1420 bytes in 82 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 07 11:03:31.918346 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8868faf0-1bc2-4bd4-8958-2a8ea338a47a admin admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 11:03:31.929399 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8868faf0-1bc2-4bd4-8958-2a8ea338a47a admin admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 11:03:31.934108 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6906a4d-dcd8-41c8-b8b0-c1670492527f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5ac47e3b09f43d19a23a0f457fe9e23', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 11:03:31.934910 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6906a4d-dcd8-41c8-b8b0-c1670492527f admin admin] REQUEST_METHOD: `PUT` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 11:03:31.935216 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6906a4d-dcd8-41c8-b8b0-c1670492527f admin admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 11:03:31.935490 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6906a4d-dcd8-41c8-b8b0-c1670492527f admin admin] PATH_INFO: `/v3/projects/f1acd5499a88421eb87a071465bac871/users/1af5b0cf121a44f985f447761d35e53c/roles/81b1f1bcee2a473088fbbd327c1c5b23` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 11:03:31.937423 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6906a4d-dcd8-41c8-b8b0-c1670492527f admin admin] RBAC: Authorizing `identity:create_grant(project_id=f1acd5499a88421eb87a071465bac871, user_id=1af5b0cf121a44f985f447761d35e53c, role_id=81b1f1bcee2a473088fbbd327c1c5b23)` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 11:03:31.940160 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6906a4d-dcd8-41c8-b8b0-c1670492527f admin admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 11:03:31.952259 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1183/2394] 10.222.0.25 () {60 vars in 1483 bytes} [Sun Aug 7 11:03:31 2022] PUT /identity/v3/projects/f1acd5499a88421eb87a071465bac871/users/1af5b0cf121a44f985f447761d35e53c/roles/81b1f1bcee2a473088fbbd327c1c5b23 => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 07 11:03:31.964194 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf4949fd-b2ee-421f-93d9-3957c029a262 None None] REQUEST_METHOD: `POST` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 11:03:31.964489 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf4949fd-b2ee-421f-93d9-3957c029a262 None None] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 11:03:31.964863 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf4949fd-b2ee-421f-93d9-3957c029a262 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 11:03:31.995391 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.auth.core [None req-cf4949fd-b2ee-421f-93d9-3957c029a262 None None] MFA Rules not processed for user `1af5b0cf121a44f985f447761d35e53c`. Rule list: `[]` (Enabled: `True`). {{(pid=80144) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 11:03:31.995602 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.fernet_utils [None req-cf4949fd-b2ee-421f-93d9-3957c029a262 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80144) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 11:03:32.081163 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 1212/2395] 10.222.0.25 () {58 vars in 958 bytes} [Sun Aug 7 11:03:31 2022] POST /identity/v3/auth/tokens => generated 3325 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 07 11:03:32.094415 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6906a4d-dcd8-41c8-b8b0-c1670492527f admin admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 11:03:32.096336 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-e6906a4d-dcd8-41c8-b8b0-c1670492527f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 11:03:32.124658 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:INSERT=7,keystone:SELECT=93 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 11:03:32.160364 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6906a4d-dcd8-41c8-b8b0-c1670492527f admin admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 11:03:32.164036 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16c76063-75a8-43a2-a6e8-96d7bba1ea21 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88dbd6dfd1164994a3e16da553561910', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6de1f3cac02c4bcdb05dd2337176033d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 11:03:32.164648 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16c76063-75a8-43a2-a6e8-96d7bba1ea21 service cinder] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 11:03:32.164932 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16c76063-75a8-43a2-a6e8-96d7bba1ea21 service cinder] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 11:03:32.165204 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16c76063-75a8-43a2-a6e8-96d7bba1ea21 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 11:03:32.175843 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16c76063-75a8-43a2-a6e8-96d7bba1ea21 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 11:03:32.177267 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16c76063-75a8-43a2-a6e8-96d7bba1ea21 service cinder] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 11:03:32.189479 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1184/2396] 10.222.0.25 () {60 vars in 1365 bytes} [Sun Aug 7 11:03:32 2022] GET /identity/v3/auth/tokens => generated 3325 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 07 11:03:32.557459 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf4949fd-b2ee-421f-93d9-3957c029a262 None None] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 11:03:32.569167 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf4949fd-b2ee-421f-93d9-3957c029a262 None None] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 11:03:32.574008 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47e3fa78-b04c-4233-88b3-c87a7504f660 tempest-VolumeQuotaClassesTest-1118615046 tempest-VolumeQuotaClassesTest-1118615046-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1af5b0cf121a44f985f447761d35e53c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f1acd5499a88421eb87a071465bac871', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 11:03:32.574908 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47e3fa78-b04c-4233-88b3-c87a7504f660 tempest-VolumeQuotaClassesTest-1118615046 tempest-VolumeQuotaClassesTest-1118615046-project-admin] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 11:03:32.576000 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47e3fa78-b04c-4233-88b3-c87a7504f660 tempest-VolumeQuotaClassesTest-1118615046 tempest-VolumeQuotaClassesTest-1118615046-project-admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 11:03:32.576275 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47e3fa78-b04c-4233-88b3-c87a7504f660 tempest-VolumeQuotaClassesTest-1118615046 tempest-VolumeQuotaClassesTest-1118615046-project-admin] PATH_INFO: `/v3/domains` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 11:03:32.577096 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47e3fa78-b04c-4233-88b3-c87a7504f660 tempest-VolumeQuotaClassesTest-1118615046 tempest-VolumeQuotaClassesTest-1118615046-project-admin] RBAC: Adding query filter params (name=Default) {{(pid=80144) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 11:03:32.577637 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47e3fa78-b04c-4233-88b3-c87a7504f660 tempest-VolumeQuotaClassesTest-1118615046 tempest-VolumeQuotaClassesTest-1118615046-project-admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 11:03:32.579752 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47e3fa78-b04c-4233-88b3-c87a7504f660 tempest-VolumeQuotaClassesTest-1118615046 tempest-VolumeQuotaClassesTest-1118615046-project-admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 11:03:32.594596 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 1213/2397] 10.222.0.25 () {58 vars in 1153 bytes} [Sun Aug 7 11:03:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 11:03:32.598806 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16c76063-75a8-43a2-a6e8-96d7bba1ea21 service cinder] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 11:03:32.607988 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16c76063-75a8-43a2-a6e8-96d7bba1ea21 service cinder] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 11:03:32.612899 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e086b733-f002-4d04-9a09-ccea5fffe276 tempest-VolumeQuotaClassesTest-1118615046 tempest-VolumeQuotaClassesTest-1118615046-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1af5b0cf121a44f985f447761d35e53c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f1acd5499a88421eb87a071465bac871', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 11:03:32.613543 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e086b733-f002-4d04-9a09-ccea5fffe276 tempest-VolumeQuotaClassesTest-1118615046 tempest-VolumeQuotaClassesTest-1118615046-project-admin] REQUEST_METHOD: `POST` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 11:03:32.613995 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e086b733-f002-4d04-9a09-ccea5fffe276 tempest-VolumeQuotaClassesTest-1118615046 tempest-VolumeQuotaClassesTest-1118615046-project-admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 11:03:32.614362 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e086b733-f002-4d04-9a09-ccea5fffe276 tempest-VolumeQuotaClassesTest-1118615046 tempest-VolumeQuotaClassesTest-1118615046-project-admin] PATH_INFO: `/v3/projects` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 11:03:32.615218 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e086b733-f002-4d04-9a09-ccea5fffe276 tempest-VolumeQuotaClassesTest-1118615046 tempest-VolumeQuotaClassesTest-1118615046-project-admin] RBAC: Authorizing `identity:create_project()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 11:03:32.617812 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e086b733-f002-4d04-9a09-ccea5fffe276 tempest-VolumeQuotaClassesTest-1118615046 tempest-VolumeQuotaClassesTest-1118615046-project-admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 11:03:32.648772 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1185/2398] 10.222.0.25 () {60 vars in 1153 bytes} [Sun Aug 7 11:03:32 2022] POST /identity/v3/projects => generated 354 bytes in 50 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 07 11:03:32.652849 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47e3fa78-b04c-4233-88b3-c87a7504f660 tempest-VolumeQuotaClassesTest-1118615046 tempest-VolumeQuotaClassesTest-1118615046-project-admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 11:03:32.663557 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47e3fa78-b04c-4233-88b3-c87a7504f660 tempest-VolumeQuotaClassesTest-1118615046 tempest-VolumeQuotaClassesTest-1118615046-project-admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 11:03:32.722329 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3538d91e-4a6c-481f-99e2-25ab43c263f3 tempest-VolumeQuotaClassesTest-1118615046 tempest-VolumeQuotaClassesTest-1118615046-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1af5b0cf121a44f985f447761d35e53c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f1acd5499a88421eb87a071465bac871', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 11:03:32.722973 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3538d91e-4a6c-481f-99e2-25ab43c263f3 tempest-VolumeQuotaClassesTest-1118615046 tempest-VolumeQuotaClassesTest-1118615046-project-admin] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 11:03:32.723266 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3538d91e-4a6c-481f-99e2-25ab43c263f3 tempest-VolumeQuotaClassesTest-1118615046 tempest-VolumeQuotaClassesTest-1118615046-project-admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 11:03:32.723547 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3538d91e-4a6c-481f-99e2-25ab43c263f3 tempest-VolumeQuotaClassesTest-1118615046 tempest-VolumeQuotaClassesTest-1118615046-project-admin] PATH_INFO: `/v3/domains` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 11:03:32.724198 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3538d91e-4a6c-481f-99e2-25ab43c263f3 tempest-VolumeQuotaClassesTest-1118615046 tempest-VolumeQuotaClassesTest-1118615046-project-admin] RBAC: Adding query filter params (name=Default) {{(pid=80144) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 11:03:32.724511 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3538d91e-4a6c-481f-99e2-25ab43c263f3 tempest-VolumeQuotaClassesTest-1118615046 tempest-VolumeQuotaClassesTest-1118615046-project-admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 11:03:32.726111 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3538d91e-4a6c-481f-99e2-25ab43c263f3 tempest-VolumeQuotaClassesTest-1118615046 tempest-VolumeQuotaClassesTest-1118615046-project-admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 11:03:32.738886 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 1214/2399] 10.222.0.25 () {58 vars in 1153 bytes} [Sun Aug 7 11:03:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 86 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 11:03:32.776085 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e086b733-f002-4d04-9a09-ccea5fffe276 tempest-VolumeQuotaClassesTest-1118615046 tempest-VolumeQuotaClassesTest-1118615046-project-admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 11:03:32.786982 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e086b733-f002-4d04-9a09-ccea5fffe276 tempest-VolumeQuotaClassesTest-1118615046 tempest-VolumeQuotaClassesTest-1118615046-project-admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 11:03:32.791457 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46dae4ec-e9eb-42b7-bf47-1016a4c48d16 tempest-VolumeQuotaClassesTest-1118615046 tempest-VolumeQuotaClassesTest-1118615046-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1af5b0cf121a44f985f447761d35e53c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f1acd5499a88421eb87a071465bac871', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 11:03:32.792078 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46dae4ec-e9eb-42b7-bf47-1016a4c48d16 tempest-VolumeQuotaClassesTest-1118615046 tempest-VolumeQuotaClassesTest-1118615046-project-admin] REQUEST_METHOD: `DELETE` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 11:03:32.792362 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46dae4ec-e9eb-42b7-bf47-1016a4c48d16 tempest-VolumeQuotaClassesTest-1118615046 tempest-VolumeQuotaClassesTest-1118615046-project-admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 11:03:32.792637 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46dae4ec-e9eb-42b7-bf47-1016a4c48d16 tempest-VolumeQuotaClassesTest-1118615046 tempest-VolumeQuotaClassesTest-1118615046-project-admin] PATH_INFO: `/v3/projects/eabb752e218048238b5b9bfcdcb53ccb` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 11:03:32.793820 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46dae4ec-e9eb-42b7-bf47-1016a4c48d16 tempest-VolumeQuotaClassesTest-1118615046 tempest-VolumeQuotaClassesTest-1118615046-project-admin] RBAC: Authorizing `identity:delete_project(project_id=eabb752e218048238b5b9bfcdcb53ccb)` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 11:03:32.796234 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46dae4ec-e9eb-42b7-bf47-1016a4c48d16 tempest-VolumeQuotaClassesTest-1118615046 tempest-VolumeQuotaClassesTest-1118615046-project-admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 11:03:32.844740 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-46dae4ec-e9eb-42b7-bf47-1016a4c48d16 tempest-VolumeQuotaClassesTest-1118615046 tempest-VolumeQuotaClassesTest-1118615046-project-admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'eabb752e218048238b5b9bfcdcb53ccb', 'request_id': 'req-46dae4ec-e9eb-42b7-bf47-1016a4c48d16'} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 11:03:32.850544 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-46dae4ec-e9eb-42b7-bf47-1016a4c48d16 tempest-VolumeQuotaClassesTest-1118615046 tempest-VolumeQuotaClassesTest-1118615046-project-admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'eabb752e218048238b5b9bfcdcb53ccb', 'request_id': 'req-46dae4ec-e9eb-42b7-bf47-1016a4c48d16'} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 11:03:32.858243 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-46dae4ec-e9eb-42b7-bf47-1016a4c48d16 tempest-VolumeQuotaClassesTest-1118615046 tempest-VolumeQuotaClassesTest-1118615046-project-admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'eabb752e218048238b5b9bfcdcb53ccb', 'request_id': 'req-46dae4ec-e9eb-42b7-bf47-1016a4c48d16'} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 11:03:32.870600 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-46dae4ec-e9eb-42b7-bf47-1016a4c48d16 tempest-VolumeQuotaClassesTest-1118615046 tempest-VolumeQuotaClassesTest-1118615046-project-admin] The token cache is being invalidate because project eabb752e218048238b5b9bfcdcb53ccb was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80143) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 07 11:03:32.870896 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-46dae4ec-e9eb-42b7-bf47-1016a4c48d16 tempest-VolumeQuotaClassesTest-1118615046 tempest-VolumeQuotaClassesTest-1118615046-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 11:03:32.871570 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-46dae4ec-e9eb-42b7-bf47-1016a4c48d16 tempest-VolumeQuotaClassesTest-1118615046 tempest-VolumeQuotaClassesTest-1118615046-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 11:03:32.873819 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1186/2400] 10.222.0.25 () {58 vars in 1233 bytes} [Sun Aug 7 11:03:32 2022] DELETE /identity/v3/projects/eabb752e218048238b5b9bfcdcb53ccb => generated 0 bytes in 98 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 07 11:03:33.072523 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3538d91e-4a6c-481f-99e2-25ab43c263f3 tempest-VolumeQuotaClassesTest-1118615046 tempest-VolumeQuotaClassesTest-1118615046-project-admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 11:03:33.074368 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.fernet_utils [None req-3538d91e-4a6c-481f-99e2-25ab43c263f3 tempest-VolumeQuotaClassesTest-1118615046 tempest-VolumeQuotaClassesTest-1118615046-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80144) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 11:03:33.139632 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3538d91e-4a6c-481f-99e2-25ab43c263f3 tempest-VolumeQuotaClassesTest-1118615046 tempest-VolumeQuotaClassesTest-1118615046-project-admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 11:03:33.143079 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=100 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 11:03:33.195471 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-915db728-3424-477e-aab0-9794d0c4cfc8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5ac47e3b09f43d19a23a0f457fe9e23', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 11:03:33.196097 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-915db728-3424-477e-aab0-9794d0c4cfc8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 11:03:33.196376 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-915db728-3424-477e-aab0-9794d0c4cfc8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 11:03:33.196650 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-915db728-3424-477e-aab0-9794d0c4cfc8 admin admin] PATH_INFO: `/v3/users/0b89f8f6c4ab49df8fa2bf649233f5ed` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 11:03:33.197760 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-915db728-3424-477e-aab0-9794d0c4cfc8 admin admin] RBAC: Authorizing `identity:delete_user(user_id=0b89f8f6c4ab49df8fa2bf649233f5ed)` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 11:03:33.199687 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-915db728-3424-477e-aab0-9794d0c4cfc8 admin admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 11:03:33.246782 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.notifications [None req-915db728-3424-477e-aab0-9794d0c4cfc8 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0b89f8f6c4ab49df8fa2bf649233f5ed'} {{(pid=80144) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 11:03:33.253709 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.notifications [None req-915db728-3424-477e-aab0-9794d0c4cfc8 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0b89f8f6c4ab49df8fa2bf649233f5ed'} {{(pid=80144) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 11:03:33.260588 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.notifications [None req-915db728-3424-477e-aab0-9794d0c4cfc8 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0b89f8f6c4ab49df8fa2bf649233f5ed'} {{(pid=80144) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 11:03:33.261334 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.notifications [None req-915db728-3424-477e-aab0-9794d0c4cfc8 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0b89f8f6c4ab49df8fa2bf649233f5ed'} {{(pid=80144) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 11:03:33.261949 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.notifications [None req-915db728-3424-477e-aab0-9794d0c4cfc8 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0b89f8f6c4ab49df8fa2bf649233f5ed'} {{(pid=80144) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 11:03:33.279904 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 1215/2401] 10.222.0.25 () {58 vars in 1224 bytes} [Sun Aug 7 11:03:33 2022] DELETE /identity/v3/users/0b89f8f6c4ab49df8fa2bf649233f5ed => generated 0 bytes in 208 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 07 11:03:33.284505 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd511a9b-d465-4f69-a33a-cdd258a674a2 None None] REQUEST_METHOD: `POST` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 11:03:33.284969 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd511a9b-d465-4f69-a33a-cdd258a674a2 None None] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 11:03:33.285090 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd511a9b-d465-4f69-a33a-cdd258a674a2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 11:03:33.305933 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.auth.core [None req-fd511a9b-d465-4f69-a33a-cdd258a674a2 None None] MFA Rules not processed for user `bac4e6b76fe64217ac8fa44e71cffe6a`. Rule list: `[]` (Enabled: `True`). {{(pid=80143) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 11:03:33.307366 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-fd511a9b-d465-4f69-a33a-cdd258a674a2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 11:03:33.369398 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1187/2402] 10.222.0.25 () {58 vars in 958 bytes} [Sun Aug 7 11:03:33 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 07 11:03:33.379644 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-915db728-3424-477e-aab0-9794d0c4cfc8 admin admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 11:03:33.381037 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.fernet_utils [None req-915db728-3424-477e-aab0-9794d0c4cfc8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80144) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 11:03:33.441940 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-915db728-3424-477e-aab0-9794d0c4cfc8 admin admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 11:03:33.451308 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-161cf6b0-aecd-4cd4-a323-703e145c4524 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be1283c9bf47424c9aa3ad27cab03027', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6de1f3cac02c4bcdb05dd2337176033d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 11:03:33.451932 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-161cf6b0-aecd-4cd4-a323-703e145c4524 service neutron] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 11:03:33.452211 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-161cf6b0-aecd-4cd4-a323-703e145c4524 service neutron] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 11:03:33.452484 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-161cf6b0-aecd-4cd4-a323-703e145c4524 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 11:03:33.462322 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-161cf6b0-aecd-4cd4-a323-703e145c4524 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 11:03:33.463785 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-161cf6b0-aecd-4cd4-a323-703e145c4524 service neutron] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 11:03:33.474676 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 1216/2403] 10.222.0.25 () {60 vars in 1365 bytes} [Sun Aug 7 11:03:33 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 07 11:03:33.759076 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd511a9b-d465-4f69-a33a-cdd258a674a2 None None] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 11:03:33.760782 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-fd511a9b-d465-4f69-a33a-cdd258a674a2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 11:03:33.774951 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd511a9b-d465-4f69-a33a-cdd258a674a2 None None] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 11:03:33.779946 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22bc7299-ab58-4680-8909-9fda2f893a73 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5ac47e3b09f43d19a23a0f457fe9e23', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 11:03:33.780859 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22bc7299-ab58-4680-8909-9fda2f893a73 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 11:03:33.781190 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22bc7299-ab58-4680-8909-9fda2f893a73 admin admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 11:03:33.781470 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22bc7299-ab58-4680-8909-9fda2f893a73 admin admin] PATH_INFO: `/v3/projects/3ee74081f1a64a409a47f437da5bacec` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 11:03:33.782777 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22bc7299-ab58-4680-8909-9fda2f893a73 admin admin] RBAC: Authorizing `identity:delete_project(project_id=3ee74081f1a64a409a47f437da5bacec)` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 11:03:33.786016 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22bc7299-ab58-4680-8909-9fda2f893a73 admin admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 11:03:33.811971 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 11:03:33.836435 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-22bc7299-ab58-4680-8909-9fda2f893a73 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3ee74081f1a64a409a47f437da5bacec', 'request_id': 'req-22bc7299-ab58-4680-8909-9fda2f893a73'} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 11:03:33.842477 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-22bc7299-ab58-4680-8909-9fda2f893a73 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3ee74081f1a64a409a47f437da5bacec', 'request_id': 'req-22bc7299-ab58-4680-8909-9fda2f893a73'} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 11:03:33.848870 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-22bc7299-ab58-4680-8909-9fda2f893a73 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3ee74081f1a64a409a47f437da5bacec', 'request_id': 'req-22bc7299-ab58-4680-8909-9fda2f893a73'} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 11:03:33.861890 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-22bc7299-ab58-4680-8909-9fda2f893a73 admin admin] The token cache is being invalidate because project 3ee74081f1a64a409a47f437da5bacec was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80143) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 07 11:03:33.862181 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-22bc7299-ab58-4680-8909-9fda2f893a73 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 11:03:33.862806 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-22bc7299-ab58-4680-8909-9fda2f893a73 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 11:03:33.866352 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1188/2404] 10.222.0.25 () {58 vars in 1233 bytes} [Sun Aug 7 11:03:33 2022] DELETE /identity/v3/projects/3ee74081f1a64a409a47f437da5bacec => generated 0 bytes in 106 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 07 11:03:33.868495 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-161cf6b0-aecd-4cd4-a323-703e145c4524 service neutron] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 11:03:33.879692 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-161cf6b0-aecd-4cd4-a323-703e145c4524 service neutron] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 11:03:33.907014 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 11:03:33.939280 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3386cd97-ef97-44af-a4ad-6aeb3b443c8c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5ac47e3b09f43d19a23a0f457fe9e23', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 11:03:33.939838 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3386cd97-ef97-44af-a4ad-6aeb3b443c8c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 11:03:33.940121 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3386cd97-ef97-44af-a4ad-6aeb3b443c8c admin admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 11:03:33.940396 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3386cd97-ef97-44af-a4ad-6aeb3b443c8c admin admin] PATH_INFO: `/v3/users/1af5b0cf121a44f985f447761d35e53c` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 11:03:33.941527 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3386cd97-ef97-44af-a4ad-6aeb3b443c8c admin admin] RBAC: Authorizing `identity:delete_user(user_id=1af5b0cf121a44f985f447761d35e53c)` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 11:03:33.943467 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3386cd97-ef97-44af-a4ad-6aeb3b443c8c admin admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 11:03:33.991999 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.notifications [None req-3386cd97-ef97-44af-a4ad-6aeb3b443c8c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1af5b0cf121a44f985f447761d35e53c'} {{(pid=80144) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 11:03:33.999344 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.notifications [None req-3386cd97-ef97-44af-a4ad-6aeb3b443c8c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1af5b0cf121a44f985f447761d35e53c'} {{(pid=80144) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 11:03:34.007342 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.notifications [None req-3386cd97-ef97-44af-a4ad-6aeb3b443c8c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1af5b0cf121a44f985f447761d35e53c'} {{(pid=80144) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 11:03:34.007978 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.notifications [None req-3386cd97-ef97-44af-a4ad-6aeb3b443c8c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1af5b0cf121a44f985f447761d35e53c'} {{(pid=80144) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 11:03:34.008583 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.notifications [None req-3386cd97-ef97-44af-a4ad-6aeb3b443c8c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1af5b0cf121a44f985f447761d35e53c'} {{(pid=80144) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 11:03:34.027976 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 1217/2405] 10.222.0.25 () {58 vars in 1224 bytes} [Sun Aug 7 11:03:33 2022] DELETE /identity/v3/users/1af5b0cf121a44f985f447761d35e53c => generated 0 bytes in 160 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 07 11:03:34.307239 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22bc7299-ab58-4680-8909-9fda2f893a73 admin admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 11:03:34.308696 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-22bc7299-ab58-4680-8909-9fda2f893a73 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 11:03:34.321959 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22bc7299-ab58-4680-8909-9fda2f893a73 admin admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 11:03:34.391593 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e02cd677-30ac-4bbc-9995-9b2b002bd040 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5ac47e3b09f43d19a23a0f457fe9e23', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 11:03:34.392235 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e02cd677-30ac-4bbc-9995-9b2b002bd040 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 11:03:34.392518 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e02cd677-30ac-4bbc-9995-9b2b002bd040 admin admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 11:03:34.392836 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e02cd677-30ac-4bbc-9995-9b2b002bd040 admin admin] PATH_INFO: `/v3/projects/f1acd5499a88421eb87a071465bac871` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 11:03:34.394136 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e02cd677-30ac-4bbc-9995-9b2b002bd040 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f1acd5499a88421eb87a071465bac871)` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 11:03:34.396264 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e02cd677-30ac-4bbc-9995-9b2b002bd040 admin admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 11:03:34.448076 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-e02cd677-30ac-4bbc-9995-9b2b002bd040 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f1acd5499a88421eb87a071465bac871', 'request_id': 'req-e02cd677-30ac-4bbc-9995-9b2b002bd040'} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 11:03:34.453596 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-e02cd677-30ac-4bbc-9995-9b2b002bd040 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f1acd5499a88421eb87a071465bac871', 'request_id': 'req-e02cd677-30ac-4bbc-9995-9b2b002bd040'} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 11:03:34.459599 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-e02cd677-30ac-4bbc-9995-9b2b002bd040 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f1acd5499a88421eb87a071465bac871', 'request_id': 'req-e02cd677-30ac-4bbc-9995-9b2b002bd040'} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 11:03:34.473693 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-e02cd677-30ac-4bbc-9995-9b2b002bd040 admin admin] The token cache is being invalidate because project f1acd5499a88421eb87a071465bac871 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80143) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 07 11:03:34.474080 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-e02cd677-30ac-4bbc-9995-9b2b002bd040 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 11:03:34.474730 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-e02cd677-30ac-4bbc-9995-9b2b002bd040 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 11:03:34.476783 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1189/2406] 10.222.0.25 () {58 vars in 1233 bytes} [Sun Aug 7 11:03:34 2022] DELETE /identity/v3/projects/f1acd5499a88421eb87a071465bac871 => generated 0 bytes in 170 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 07 11:03:34.482844 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4ec0158-6a5c-4da5-8ea8-aa0c2d0528c6 None None] REQUEST_METHOD: `POST` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 11:03:34.483620 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4ec0158-6a5c-4da5-8ea8-aa0c2d0528c6 None None] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 11:03:34.483986 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4ec0158-6a5c-4da5-8ea8-aa0c2d0528c6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 11:03:34.504996 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.auth.core [None req-a4ec0158-6a5c-4da5-8ea8-aa0c2d0528c6 None None] MFA Rules not processed for user `bac4e6b76fe64217ac8fa44e71cffe6a`. Rule list: `[]` (Enabled: `True`). {{(pid=80144) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 11:03:34.506205 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.fernet_utils [None req-a4ec0158-6a5c-4da5-8ea8-aa0c2d0528c6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80144) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 11:03:34.568409 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 1218/2407] 10.222.0.25 () {58 vars in 958 bytes} [Sun Aug 7 11:03:34 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 07 11:03:34.576541 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aae444a3-87fb-449a-b378-0a0a7e27f841 None None] REQUEST_METHOD: `POST` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 11:03:34.576864 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aae444a3-87fb-449a-b378-0a0a7e27f841 None None] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 11:03:34.577141 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aae444a3-87fb-449a-b378-0a0a7e27f841 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 11:03:34.598209 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.auth.core [None req-aae444a3-87fb-449a-b378-0a0a7e27f841 None None] MFA Rules not processed for user `bac4e6b76fe64217ac8fa44e71cffe6a`. Rule list: `[]` (Enabled: `True`). {{(pid=80143) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 11:03:34.599455 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-aae444a3-87fb-449a-b378-0a0a7e27f841 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 11:03:34.608089 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1190/2408] 10.222.0.25 () {58 vars in 958 bytes} [Sun Aug 7 11:03:34 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 07 11:03:34.612706 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4ec0158-6a5c-4da5-8ea8-aa0c2d0528c6 None None] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 11:03:34.623541 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4ec0158-6a5c-4da5-8ea8-aa0c2d0528c6 None None] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 11:03:34.628135 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf5ac3ce-fd21-486b-b746-5de20b748af7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5ac47e3b09f43d19a23a0f457fe9e23', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 11:03:34.628738 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf5ac3ce-fd21-486b-b746-5de20b748af7 admin admin] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 11:03:34.629019 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf5ac3ce-fd21-486b-b746-5de20b748af7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 11:03:34.629293 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf5ac3ce-fd21-486b-b746-5de20b748af7 admin admin] PATH_INFO: `/v3/domains` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 11:03:34.629942 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf5ac3ce-fd21-486b-b746-5de20b748af7 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80144) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 11:03:34.630251 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf5ac3ce-fd21-486b-b746-5de20b748af7 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 11:03:34.631861 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf5ac3ce-fd21-486b-b746-5de20b748af7 admin admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 11:03:34.648420 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 1219/2409] 10.222.0.25 () {58 vars in 1153 bytes} [Sun Aug 7 11:03:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 11:03:34.649647 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aae444a3-87fb-449a-b378-0a0a7e27f841 None None] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 11:03:34.660412 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aae444a3-87fb-449a-b378-0a0a7e27f841 None None] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 11:03:34.666051 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66d71450-b365-4292-bd29-f322a0eb30cc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5ac47e3b09f43d19a23a0f457fe9e23', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 11:03:34.666831 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66d71450-b365-4292-bd29-f322a0eb30cc admin admin] REQUEST_METHOD: `POST` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 11:03:34.667167 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66d71450-b365-4292-bd29-f322a0eb30cc admin admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 11:03:34.667451 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66d71450-b365-4292-bd29-f322a0eb30cc admin admin] PATH_INFO: `/v3/projects` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 11:03:34.668081 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66d71450-b365-4292-bd29-f322a0eb30cc admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 11:03:34.670629 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66d71450-b365-4292-bd29-f322a0eb30cc admin admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 11:03:34.700299 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1191/2410] 10.222.0.25 () {60 vars in 1153 bytes} [Sun Aug 7 11:03:34 2022] POST /identity/v3/projects => generated 409 bytes in 51 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 07 11:03:34.704762 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf5ac3ce-fd21-486b-b746-5de20b748af7 admin admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 11:03:34.715065 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf5ac3ce-fd21-486b-b746-5de20b748af7 admin admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 11:03:34.749805 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 11:03:34.778159 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-072debc3-df08-4769-baad-919f461a1019 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5ac47e3b09f43d19a23a0f457fe9e23', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 11:03:34.778761 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-072debc3-df08-4769-baad-919f461a1019 admin admin] REQUEST_METHOD: `POST` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 11:03:34.779077 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-072debc3-df08-4769-baad-919f461a1019 admin admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 11:03:34.779353 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-072debc3-df08-4769-baad-919f461a1019 admin admin] PATH_INFO: `/v3/users` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 11:03:34.779978 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-072debc3-df08-4769-baad-919f461a1019 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 11:03:34.781880 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-072debc3-df08-4769-baad-919f461a1019 admin admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 11:03:34.802741 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 1220/2411] 10.222.0.25 () {60 vars in 1144 bytes} [Sun Aug 7 11:03:34 2022] POST /identity/v3/users => generated 305 bytes in 98 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 07 11:03:34.807030 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66d71450-b365-4292-bd29-f322a0eb30cc admin admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 11:03:34.818734 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66d71450-b365-4292-bd29-f322a0eb30cc admin admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 11:03:34.822483 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98298df9-aabf-44af-88fd-88ee8bac11c0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5ac47e3b09f43d19a23a0f457fe9e23', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 11:03:34.823113 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98298df9-aabf-44af-88fd-88ee8bac11c0 admin admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 11:03:34.824648 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98298df9-aabf-44af-88fd-88ee8bac11c0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 11:03:34.824648 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98298df9-aabf-44af-88fd-88ee8bac11c0 admin admin] PATH_INFO: `/v3/roles` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 11:03:34.824648 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98298df9-aabf-44af-88fd-88ee8bac11c0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 11:03:34.834212 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98298df9-aabf-44af-88fd-88ee8bac11c0 admin admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 11:03:34.834212 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:DELETE=14,keystone:SELECT=83,keystone:INSERT=3 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 11:03:34.839462 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1192/2412] 10.222.0.25 () {58 vars in 1122 bytes} [Sun Aug 7 11:03:34 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 07 11:03:34.842193 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-072debc3-df08-4769-baad-919f461a1019 admin admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 11:03:34.852854 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-072debc3-df08-4769-baad-919f461a1019 admin admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 11:03:34.858313 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53e65ffd-58b6-48fd-a1ae-c4d792b155c3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5ac47e3b09f43d19a23a0f457fe9e23', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 11:03:34.859221 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53e65ffd-58b6-48fd-a1ae-c4d792b155c3 admin admin] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 11:03:34.859538 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53e65ffd-58b6-48fd-a1ae-c4d792b155c3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 11:03:34.859814 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53e65ffd-58b6-48fd-a1ae-c4d792b155c3 admin admin] PATH_INFO: `/v3/roles` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 11:03:34.860459 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53e65ffd-58b6-48fd-a1ae-c4d792b155c3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 11:03:34.862147 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53e65ffd-58b6-48fd-a1ae-c4d792b155c3 admin admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 11:03:34.872290 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 1221/2413] 10.222.0.25 () {58 vars in 1122 bytes} [Sun Aug 7 11:03:34 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 07 11:03:34.880149 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98298df9-aabf-44af-88fd-88ee8bac11c0 admin admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 11:03:34.890563 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98298df9-aabf-44af-88fd-88ee8bac11c0 admin admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 11:03:34.895159 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27fd096f-4037-491f-a221-12f9e73531f4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5ac47e3b09f43d19a23a0f457fe9e23', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 11:03:34.895811 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27fd096f-4037-491f-a221-12f9e73531f4 admin admin] REQUEST_METHOD: `PUT` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 11:03:34.896092 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27fd096f-4037-491f-a221-12f9e73531f4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 11:03:34.896367 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27fd096f-4037-491f-a221-12f9e73531f4 admin admin] PATH_INFO: `/v3/projects/d8a2541c35664cfaa81ed159a999a5b0/users/54a3e9ee80f64c2fbd685f6b7115bca8/roles/b98dfb3b9fee4644af2887a3ac3fd2e6` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 11:03:34.911589 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27fd096f-4037-491f-a221-12f9e73531f4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d8a2541c35664cfaa81ed159a999a5b0, user_id=54a3e9ee80f64c2fbd685f6b7115bca8, role_id=b98dfb3b9fee4644af2887a3ac3fd2e6)` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 11:03:34.914391 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27fd096f-4037-491f-a221-12f9e73531f4 admin admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 11:03:34.929559 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1193/2414] 10.222.0.25 () {60 vars in 1483 bytes} [Sun Aug 7 11:03:34 2022] PUT /identity/v3/projects/d8a2541c35664cfaa81ed159a999a5b0/users/54a3e9ee80f64c2fbd685f6b7115bca8/roles/b98dfb3b9fee4644af2887a3ac3fd2e6 => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 07 11:03:34.941981 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2afda425-12dd-44c8-9383-359a879f9388 None None] REQUEST_METHOD: `POST` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 11:03:34.942267 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2afda425-12dd-44c8-9383-359a879f9388 None None] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 11:03:34.942540 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2afda425-12dd-44c8-9383-359a879f9388 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 11:03:34.974359 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.auth.core [None req-2afda425-12dd-44c8-9383-359a879f9388 None None] MFA Rules not processed for user `54a3e9ee80f64c2fbd685f6b7115bca8`. Rule list: `[]` (Enabled: `True`). {{(pid=80144) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 11:03:34.975522 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.fernet_utils [None req-2afda425-12dd-44c8-9383-359a879f9388 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80144) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 11:03:35.043075 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 1222/2415] 10.222.0.25 () {58 vars in 958 bytes} [Sun Aug 7 11:03:34 2022] POST /identity/v3/auth/tokens => generated 3286 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 07 11:03:35.048529 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27fd096f-4037-491f-a221-12f9e73531f4 admin admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 11:03:35.109713 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27fd096f-4037-491f-a221-12f9e73531f4 admin admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 11:03:35.114459 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cfeddaf-85fa-4f23-a4f9-fdbe92a04866 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5ac47e3b09f43d19a23a0f457fe9e23', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 11:03:35.115104 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cfeddaf-85fa-4f23-a4f9-fdbe92a04866 admin admin] REQUEST_METHOD: `POST` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 11:03:35.115420 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cfeddaf-85fa-4f23-a4f9-fdbe92a04866 admin admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 11:03:35.115700 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cfeddaf-85fa-4f23-a4f9-fdbe92a04866 admin admin] PATH_INFO: `/v3/projects` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 11:03:35.116485 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cfeddaf-85fa-4f23-a4f9-fdbe92a04866 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 11:03:35.118426 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cfeddaf-85fa-4f23-a4f9-fdbe92a04866 admin admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 11:03:35.146285 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1194/2416] 10.222.0.25 () {60 vars in 1153 bytes} [Sun Aug 7 11:03:35 2022] POST /identity/v3/projects => generated 409 bytes in 98 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 07 11:03:35.150510 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2afda425-12dd-44c8-9383-359a879f9388 None None] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 11:03:35.161801 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2afda425-12dd-44c8-9383-359a879f9388 None None] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 11:03:35.219922 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd627ed8-cf80-4d17-8d85-ea85bf384c36 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5ac47e3b09f43d19a23a0f457fe9e23', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 11:03:35.220529 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd627ed8-cf80-4d17-8d85-ea85bf384c36 admin admin] REQUEST_METHOD: `POST` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 11:03:35.220815 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd627ed8-cf80-4d17-8d85-ea85bf384c36 admin admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 11:03:35.221090 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd627ed8-cf80-4d17-8d85-ea85bf384c36 admin admin] PATH_INFO: `/v3/users` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 11:03:35.221716 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd627ed8-cf80-4d17-8d85-ea85bf384c36 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 11:03:35.223648 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd627ed8-cf80-4d17-8d85-ea85bf384c36 admin admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 11:03:35.244467 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 1223/2417] 10.222.0.25 () {60 vars in 1144 bytes} [Sun Aug 7 11:03:35 2022] POST /identity/v3/users => generated 311 bytes in 94 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 07 11:03:35.248655 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cfeddaf-85fa-4f23-a4f9-fdbe92a04866 admin admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 11:03:35.259013 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cfeddaf-85fa-4f23-a4f9-fdbe92a04866 admin admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 11:03:35.264627 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35a8c360-b1ab-4f58-ab8e-68e11735537f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5ac47e3b09f43d19a23a0f457fe9e23', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 11:03:35.265373 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35a8c360-b1ab-4f58-ab8e-68e11735537f admin admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 11:03:35.265657 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35a8c360-b1ab-4f58-ab8e-68e11735537f admin admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 11:03:35.265936 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35a8c360-b1ab-4f58-ab8e-68e11735537f admin admin] PATH_INFO: `/v3/roles` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 11:03:35.266586 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35a8c360-b1ab-4f58-ab8e-68e11735537f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 11:03:35.268242 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35a8c360-b1ab-4f58-ab8e-68e11735537f admin admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 11:03:35.279834 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1195/2418] 10.222.0.25 () {58 vars in 1122 bytes} [Sun Aug 7 11:03:35 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 07 11:03:35.283998 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd627ed8-cf80-4d17-8d85-ea85bf384c36 admin admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 11:03:35.294074 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd627ed8-cf80-4d17-8d85-ea85bf384c36 admin admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 11:03:35.298831 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0edd02b2-d6c0-4140-bada-9b23d6fddba2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5ac47e3b09f43d19a23a0f457fe9e23', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 11:03:35.299523 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0edd02b2-d6c0-4140-bada-9b23d6fddba2 admin admin] REQUEST_METHOD: `PUT` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 11:03:35.299809 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0edd02b2-d6c0-4140-bada-9b23d6fddba2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 11:03:35.300091 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0edd02b2-d6c0-4140-bada-9b23d6fddba2 admin admin] PATH_INFO: `/v3/projects/dcf0a9bc0b634e55b7125ff2e399ff84/users/2cd0658c315841199306d7056bce829f/roles/81b1f1bcee2a473088fbbd327c1c5b23` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 11:03:35.314411 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0edd02b2-d6c0-4140-bada-9b23d6fddba2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=dcf0a9bc0b634e55b7125ff2e399ff84, user_id=2cd0658c315841199306d7056bce829f, role_id=81b1f1bcee2a473088fbbd327c1c5b23)` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 11:03:35.317111 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0edd02b2-d6c0-4140-bada-9b23d6fddba2 admin admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 11:03:35.335921 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 1224/2419] 10.222.0.25 () {60 vars in 1483 bytes} [Sun Aug 7 11:03:35 2022] PUT /identity/v3/projects/dcf0a9bc0b634e55b7125ff2e399ff84/users/2cd0658c315841199306d7056bce829f/roles/81b1f1bcee2a473088fbbd327c1c5b23 => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 07 11:03:35.340015 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35a8c360-b1ab-4f58-ab8e-68e11735537f admin admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 11:03:35.350872 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35a8c360-b1ab-4f58-ab8e-68e11735537f admin admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 11:03:35.409938 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d57f80cd-9222-436c-b4b5-e910e26eac62 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5ac47e3b09f43d19a23a0f457fe9e23', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 11:03:35.410553 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d57f80cd-9222-436c-b4b5-e910e26eac62 admin admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 11:03:35.410837 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d57f80cd-9222-436c-b4b5-e910e26eac62 admin admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 11:03:35.411143 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d57f80cd-9222-436c-b4b5-e910e26eac62 admin admin] PATH_INFO: `/v3/roles` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 11:03:35.411792 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d57f80cd-9222-436c-b4b5-e910e26eac62 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 11:03:35.413384 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d57f80cd-9222-436c-b4b5-e910e26eac62 admin admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 11:03:35.423882 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1196/2420] 10.222.0.25 () {58 vars in 1122 bytes} [Sun Aug 7 11:03:35 2022] GET /identity/v3/roles => generated 1420 bytes in 84 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 07 11:03:35.427998 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0edd02b2-d6c0-4140-bada-9b23d6fddba2 admin admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 11:03:35.438963 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0edd02b2-d6c0-4140-bada-9b23d6fddba2 admin admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 11:03:35.443669 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21861fcb-3c07-4384-8299-e7aa45b96f12 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5ac47e3b09f43d19a23a0f457fe9e23', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 11:03:35.444326 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21861fcb-3c07-4384-8299-e7aa45b96f12 admin admin] REQUEST_METHOD: `PUT` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 11:03:35.444611 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21861fcb-3c07-4384-8299-e7aa45b96f12 admin admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 11:03:35.444888 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21861fcb-3c07-4384-8299-e7aa45b96f12 admin admin] PATH_INFO: `/v3/projects/dcf0a9bc0b634e55b7125ff2e399ff84/users/2cd0658c315841199306d7056bce829f/roles/81b1f1bcee2a473088fbbd327c1c5b23` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 11:03:35.446822 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21861fcb-3c07-4384-8299-e7aa45b96f12 admin admin] RBAC: Authorizing `identity:create_grant(project_id=dcf0a9bc0b634e55b7125ff2e399ff84, user_id=2cd0658c315841199306d7056bce829f, role_id=81b1f1bcee2a473088fbbd327c1c5b23)` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 11:03:35.449497 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21861fcb-3c07-4384-8299-e7aa45b96f12 admin admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 11:03:35.453498 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=92,keystone:INSERT=8 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 11:03:35.462341 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 1225/2421] 10.222.0.25 () {60 vars in 1483 bytes} [Sun Aug 7 11:03:35 2022] PUT /identity/v3/projects/dcf0a9bc0b634e55b7125ff2e399ff84/users/2cd0658c315841199306d7056bce829f/roles/81b1f1bcee2a473088fbbd327c1c5b23 => generated 0 bytes in 35 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 07 11:03:35.527662 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bd32239-98de-4c6a-9fe4-94b35f49e7c5 None None] REQUEST_METHOD: `POST` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 11:03:35.527944 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bd32239-98de-4c6a-9fe4-94b35f49e7c5 None None] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 11:03:35.528475 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bd32239-98de-4c6a-9fe4-94b35f49e7c5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 11:03:35.558104 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.auth.core [None req-0bd32239-98de-4c6a-9fe4-94b35f49e7c5 None None] MFA Rules not processed for user `2cd0658c315841199306d7056bce829f`. Rule list: `[]` (Enabled: `True`). {{(pid=80143) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 11:03:35.559881 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-0bd32239-98de-4c6a-9fe4-94b35f49e7c5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 11:03:35.594752 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 11:03:35.632420 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1197/2422] 10.222.0.25 () {58 vars in 958 bytes} [Sun Aug 7 11:03:35 2022] POST /identity/v3/auth/tokens => generated 3353 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 07 11:03:35.644190 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21861fcb-3c07-4384-8299-e7aa45b96f12 admin admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 11:03:35.645952 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.fernet_utils [None req-21861fcb-3c07-4384-8299-e7aa45b96f12 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80144) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 11:03:35.705058 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21861fcb-3c07-4384-8299-e7aa45b96f12 admin admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 11:03:35.708694 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83bc11ba-d6da-4f00-8a54-67f44322feb9 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88dbd6dfd1164994a3e16da553561910', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6de1f3cac02c4bcdb05dd2337176033d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 11:03:35.709313 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83bc11ba-d6da-4f00-8a54-67f44322feb9 service cinder] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 11:03:35.709577 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83bc11ba-d6da-4f00-8a54-67f44322feb9 service cinder] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 11:03:35.709874 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83bc11ba-d6da-4f00-8a54-67f44322feb9 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 11:03:35.719545 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83bc11ba-d6da-4f00-8a54-67f44322feb9 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 11:03:35.720969 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83bc11ba-d6da-4f00-8a54-67f44322feb9 service cinder] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 11:03:35.730343 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 1226/2423] 10.222.0.25 () {60 vars in 1365 bytes} [Sun Aug 7 11:03:35 2022] GET /identity/v3/auth/tokens => generated 3353 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 07 11:03:35.895740 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bd32239-98de-4c6a-9fe4-94b35f49e7c5 None None] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 11:03:35.906114 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bd32239-98de-4c6a-9fe4-94b35f49e7c5 None None] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 11:03:35.909664 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c61a0a7-2d45-4c77-8f44-fed87ef4bcf4 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88dbd6dfd1164994a3e16da553561910', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6de1f3cac02c4bcdb05dd2337176033d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 11:03:35.910270 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c61a0a7-2d45-4c77-8f44-fed87ef4bcf4 service cinder] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 11:03:35.910550 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c61a0a7-2d45-4c77-8f44-fed87ef4bcf4 service cinder] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 11:03:35.910822 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c61a0a7-2d45-4c77-8f44-fed87ef4bcf4 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 11:03:35.969620 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c61a0a7-2d45-4c77-8f44-fed87ef4bcf4 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 11:03:35.971139 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c61a0a7-2d45-4c77-8f44-fed87ef4bcf4 service cinder] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 11:03:35.979956 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1198/2424] 10.222.0.25 () {60 vars in 1365 bytes} [Sun Aug 7 11:03:35 2022] GET /identity/v3/auth/tokens => generated 3286 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 07 11:03:37.788045 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3b5f828-a018-4f57-b1b1-4ce7caecf345 None None] REQUEST_METHOD: `POST` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 11:03:37.788538 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3b5f828-a018-4f57-b1b1-4ce7caecf345 None None] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 11:03:37.788684 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3b5f828-a018-4f57-b1b1-4ce7caecf345 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 11:03:37.820075 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.auth.core [None req-e3b5f828-a018-4f57-b1b1-4ce7caecf345 None None] MFA Rules not processed for user `de1d6c24b1a14d10a20d82a4c65793d8`. Rule list: `[]` (Enabled: `True`). {{(pid=80144) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 11:03:37.821233 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.fernet_utils [None req-e3b5f828-a018-4f57-b1b1-4ce7caecf345 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80144) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 11:03:37.884041 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 1227/2425] 10.222.0.25 () {58 vars in 1009 bytes} [Sun Aug 7 11:03:37 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 07 11:03:37.932028 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c61a0a7-2d45-4c77-8f44-fed87ef4bcf4 service cinder] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 11:03:37.933581 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-2c61a0a7-2d45-4c77-8f44-fed87ef4bcf4 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 11:03:38.006184 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c61a0a7-2d45-4c77-8f44-fed87ef4bcf4 service cinder] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 11:03:38.012145 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8912bd8-1535-4695-8646-3272814e56d5 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69490ea2a9a340bf91aca38bf5655b45', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6de1f3cac02c4bcdb05dd2337176033d', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 11:03:38.013583 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8912bd8-1535-4695-8646-3272814e56d5 service swift] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 11:03:38.013789 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8912bd8-1535-4695-8646-3272814e56d5 service swift] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 11:03:38.014071 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8912bd8-1535-4695-8646-3272814e56d5 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 11:03:38.022567 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8912bd8-1535-4695-8646-3272814e56d5 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 11:03:38.024188 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8912bd8-1535-4695-8646-3272814e56d5 service swift] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 11:03:38.032240 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1199/2426] 10.222.0.25 () {60 vars in 1384 bytes} [Sun Aug 7 11:03:37 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 101 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 07 11:03:45.654981 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3b5f828-a018-4f57-b1b1-4ce7caecf345 None None] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 11:03:45.743056 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3b5f828-a018-4f57-b1b1-4ce7caecf345 None None] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 11:03:45.748630 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1def883d-c4bb-4aca-8370-05a6a85abdb3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5ac47e3b09f43d19a23a0f457fe9e23', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 11:03:45.749444 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1def883d-c4bb-4aca-8370-05a6a85abdb3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 11:03:45.749800 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1def883d-c4bb-4aca-8370-05a6a85abdb3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 11:03:45.750143 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1def883d-c4bb-4aca-8370-05a6a85abdb3 admin admin] PATH_INFO: `/v3/users/54a3e9ee80f64c2fbd685f6b7115bca8` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 11:03:45.751401 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1def883d-c4bb-4aca-8370-05a6a85abdb3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=54a3e9ee80f64c2fbd685f6b7115bca8)` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 11:03:45.753461 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1def883d-c4bb-4aca-8370-05a6a85abdb3 admin admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 11:03:45.795292 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=95,keystone:DELETE=5 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 11:03:45.812548 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.notifications [None req-1def883d-c4bb-4aca-8370-05a6a85abdb3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '54a3e9ee80f64c2fbd685f6b7115bca8'} {{(pid=80144) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 11:03:45.820555 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.notifications [None req-1def883d-c4bb-4aca-8370-05a6a85abdb3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '54a3e9ee80f64c2fbd685f6b7115bca8'} {{(pid=80144) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 11:03:45.830463 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.notifications [None req-1def883d-c4bb-4aca-8370-05a6a85abdb3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '54a3e9ee80f64c2fbd685f6b7115bca8'} {{(pid=80144) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 11:03:45.831276 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.notifications [None req-1def883d-c4bb-4aca-8370-05a6a85abdb3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '54a3e9ee80f64c2fbd685f6b7115bca8'} {{(pid=80144) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 11:03:45.832241 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.notifications [None req-1def883d-c4bb-4aca-8370-05a6a85abdb3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '54a3e9ee80f64c2fbd685f6b7115bca8'} {{(pid=80144) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 11:03:45.854954 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 1228/2427] 10.222.0.25 () {58 vars in 1224 bytes} [Sun Aug 7 11:03:45 2022] DELETE /identity/v3/users/54a3e9ee80f64c2fbd685f6b7115bca8 => generated 0 bytes in 200 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 07 11:03:45.860159 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab101e0a-5226-4fa9-b7ad-05087a5c4a77 None None] REQUEST_METHOD: `POST` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 11:03:45.860674 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab101e0a-5226-4fa9-b7ad-05087a5c4a77 None None] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 11:03:45.860768 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab101e0a-5226-4fa9-b7ad-05087a5c4a77 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 11:03:45.882694 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.auth.core [None req-ab101e0a-5226-4fa9-b7ad-05087a5c4a77 None None] MFA Rules not processed for user `bac4e6b76fe64217ac8fa44e71cffe6a`. Rule list: `[]` (Enabled: `True`). {{(pid=80143) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 11:03:45.884666 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-ab101e0a-5226-4fa9-b7ad-05087a5c4a77 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 11:03:45.948426 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=100 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 11:03:45.960753 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1200/2428] 10.222.0.25 () {58 vars in 958 bytes} [Sun Aug 7 11:03:45 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 07 11:03:45.971631 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1def883d-c4bb-4aca-8370-05a6a85abdb3 admin admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 11:03:45.973741 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.fernet_utils [None req-1def883d-c4bb-4aca-8370-05a6a85abdb3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80144) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 11:03:46.045957 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1def883d-c4bb-4aca-8370-05a6a85abdb3 admin admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 11:03:46.050290 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-920fe133-c155-43ed-a509-d7df5ef32d71 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be1283c9bf47424c9aa3ad27cab03027', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6de1f3cac02c4bcdb05dd2337176033d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 11:03:46.051121 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-920fe133-c155-43ed-a509-d7df5ef32d71 service neutron] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 11:03:46.051600 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-920fe133-c155-43ed-a509-d7df5ef32d71 service neutron] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 11:03:46.051874 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-920fe133-c155-43ed-a509-d7df5ef32d71 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 11:03:46.063288 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-920fe133-c155-43ed-a509-d7df5ef32d71 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 11:03:46.064722 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-920fe133-c155-43ed-a509-d7df5ef32d71 service neutron] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 11:03:46.074494 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 1229/2429] 10.222.0.25 () {60 vars in 1365 bytes} [Sun Aug 7 11:03:45 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 103 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 07 11:03:46.414325 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab101e0a-5226-4fa9-b7ad-05087a5c4a77 None None] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 11:03:46.421792 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-ab101e0a-5226-4fa9-b7ad-05087a5c4a77 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 11:03:46.441212 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab101e0a-5226-4fa9-b7ad-05087a5c4a77 None None] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 11:03:46.446537 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b044008c-f4da-45f9-ae77-d95d1b4bba5e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5ac47e3b09f43d19a23a0f457fe9e23', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 11:03:46.447219 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b044008c-f4da-45f9-ae77-d95d1b4bba5e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 11:03:46.447525 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b044008c-f4da-45f9-ae77-d95d1b4bba5e admin admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 11:03:46.447772 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b044008c-f4da-45f9-ae77-d95d1b4bba5e admin admin] PATH_INFO: `/v3/projects/d8a2541c35664cfaa81ed159a999a5b0` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 11:03:46.448911 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b044008c-f4da-45f9-ae77-d95d1b4bba5e admin admin] RBAC: Authorizing `identity:delete_project(project_id=d8a2541c35664cfaa81ed159a999a5b0)` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 11:03:46.450862 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b044008c-f4da-45f9-ae77-d95d1b4bba5e admin admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 11:03:46.512212 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-b044008c-f4da-45f9-ae77-d95d1b4bba5e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd8a2541c35664cfaa81ed159a999a5b0', 'request_id': 'req-b044008c-f4da-45f9-ae77-d95d1b4bba5e'} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 11:03:46.519542 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-b044008c-f4da-45f9-ae77-d95d1b4bba5e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd8a2541c35664cfaa81ed159a999a5b0', 'request_id': 'req-b044008c-f4da-45f9-ae77-d95d1b4bba5e'} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 11:03:46.526122 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-b044008c-f4da-45f9-ae77-d95d1b4bba5e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd8a2541c35664cfaa81ed159a999a5b0', 'request_id': 'req-b044008c-f4da-45f9-ae77-d95d1b4bba5e'} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 11:03:46.545357 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-b044008c-f4da-45f9-ae77-d95d1b4bba5e admin admin] The token cache is being invalidate because project d8a2541c35664cfaa81ed159a999a5b0 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80143) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 07 11:03:46.545655 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-b044008c-f4da-45f9-ae77-d95d1b4bba5e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 11:03:46.546331 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-b044008c-f4da-45f9-ae77-d95d1b4bba5e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 11:03:46.548091 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1201/2430] 10.222.0.25 () {58 vars in 1233 bytes} [Sun Aug 7 11:03:46 2022] DELETE /identity/v3/projects/d8a2541c35664cfaa81ed159a999a5b0 => generated 0 bytes in 135 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 07 11:03:46.553011 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-920fe133-c155-43ed-a509-d7df5ef32d71 service neutron] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 11:03:46.564470 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-920fe133-c155-43ed-a509-d7df5ef32d71 service neutron] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 11:03:46.629362 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cacbdae2-d8f6-4754-acb6-a8ef92f307a5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5ac47e3b09f43d19a23a0f457fe9e23', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 11:03:46.630037 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cacbdae2-d8f6-4754-acb6-a8ef92f307a5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 11:03:46.630413 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cacbdae2-d8f6-4754-acb6-a8ef92f307a5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 11:03:46.630673 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cacbdae2-d8f6-4754-acb6-a8ef92f307a5 admin admin] PATH_INFO: `/v3/users/2cd0658c315841199306d7056bce829f` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 11:03:46.633651 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cacbdae2-d8f6-4754-acb6-a8ef92f307a5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=2cd0658c315841199306d7056bce829f)` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 11:03:46.636591 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cacbdae2-d8f6-4754-acb6-a8ef92f307a5 admin admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 11:03:46.694215 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.notifications [None req-cacbdae2-d8f6-4754-acb6-a8ef92f307a5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2cd0658c315841199306d7056bce829f'} {{(pid=80144) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 11:03:46.699869 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 11:03:46.705162 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.notifications [None req-cacbdae2-d8f6-4754-acb6-a8ef92f307a5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2cd0658c315841199306d7056bce829f'} {{(pid=80144) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 11:03:46.719618 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.notifications [None req-cacbdae2-d8f6-4754-acb6-a8ef92f307a5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2cd0658c315841199306d7056bce829f'} {{(pid=80144) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 11:03:46.720337 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.notifications [None req-cacbdae2-d8f6-4754-acb6-a8ef92f307a5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2cd0658c315841199306d7056bce829f'} {{(pid=80144) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 11:03:46.721059 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.notifications [None req-cacbdae2-d8f6-4754-acb6-a8ef92f307a5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2cd0658c315841199306d7056bce829f'} {{(pid=80144) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 11:03:46.742472 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 1230/2431] 10.222.0.25 () {58 vars in 1224 bytes} [Sun Aug 7 11:03:46 2022] DELETE /identity/v3/users/2cd0658c315841199306d7056bce829f => generated 0 bytes in 190 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 07 11:03:47.056659 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b044008c-f4da-45f9-ae77-d95d1b4bba5e admin admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 11:03:47.059616 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-b044008c-f4da-45f9-ae77-d95d1b4bba5e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 11:03:47.074188 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b044008c-f4da-45f9-ae77-d95d1b4bba5e admin admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 11:03:47.144242 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bf373db-6f98-4675-baa8-a2df1aca26fc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5ac47e3b09f43d19a23a0f457fe9e23', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 11:03:47.145083 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bf373db-6f98-4675-baa8-a2df1aca26fc admin admin] REQUEST_METHOD: `DELETE` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 11:03:47.145427 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bf373db-6f98-4675-baa8-a2df1aca26fc admin admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 11:03:47.146261 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bf373db-6f98-4675-baa8-a2df1aca26fc admin admin] PATH_INFO: `/v3/projects/dcf0a9bc0b634e55b7125ff2e399ff84` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 11:03:47.147096 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bf373db-6f98-4675-baa8-a2df1aca26fc admin admin] RBAC: Authorizing `identity:delete_project(project_id=dcf0a9bc0b634e55b7125ff2e399ff84)` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 11:03:47.149228 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bf373db-6f98-4675-baa8-a2df1aca26fc admin admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 11:03:47.199228 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-4bf373db-6f98-4675-baa8-a2df1aca26fc admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'dcf0a9bc0b634e55b7125ff2e399ff84', 'request_id': 'req-4bf373db-6f98-4675-baa8-a2df1aca26fc'} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 11:03:47.205629 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-4bf373db-6f98-4675-baa8-a2df1aca26fc admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'dcf0a9bc0b634e55b7125ff2e399ff84', 'request_id': 'req-4bf373db-6f98-4675-baa8-a2df1aca26fc'} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 11:03:47.211835 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-4bf373db-6f98-4675-baa8-a2df1aca26fc admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'dcf0a9bc0b634e55b7125ff2e399ff84', 'request_id': 'req-4bf373db-6f98-4675-baa8-a2df1aca26fc'} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 11:03:47.225038 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-4bf373db-6f98-4675-baa8-a2df1aca26fc admin admin] The token cache is being invalidate because project dcf0a9bc0b634e55b7125ff2e399ff84 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80143) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 07 11:03:47.225331 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-4bf373db-6f98-4675-baa8-a2df1aca26fc admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 11:03:47.225985 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-4bf373db-6f98-4675-baa8-a2df1aca26fc admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 11:03:47.227542 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1202/2432] 10.222.0.25 () {58 vars in 1233 bytes} [Sun Aug 7 11:03:47 2022] DELETE /identity/v3/projects/dcf0a9bc0b634e55b7125ff2e399ff84 => generated 0 bytes in 171 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 07 11:03:47.233264 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b0a4eec-e81d-45d2-8169-237be4dc91d0 None None] REQUEST_METHOD: `POST` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 11:03:47.233805 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b0a4eec-e81d-45d2-8169-237be4dc91d0 None None] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 11:03:47.234088 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b0a4eec-e81d-45d2-8169-237be4dc91d0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 11:03:47.255394 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.auth.core [None req-4b0a4eec-e81d-45d2-8169-237be4dc91d0 None None] MFA Rules not processed for user `bac4e6b76fe64217ac8fa44e71cffe6a`. Rule list: `[]` (Enabled: `True`). {{(pid=80144) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 11:03:47.256680 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.fernet_utils [None req-4b0a4eec-e81d-45d2-8169-237be4dc91d0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80144) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 11:03:47.321853 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 1231/2433] 10.222.0.25 () {58 vars in 958 bytes} [Sun Aug 7 11:03:47 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 07 11:03:47.329543 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58588ddd-a11f-4e9c-8754-95a063925a21 None None] REQUEST_METHOD: `POST` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 11:03:47.329829 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58588ddd-a11f-4e9c-8754-95a063925a21 None None] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 11:03:47.330104 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58588ddd-a11f-4e9c-8754-95a063925a21 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 11:03:47.349935 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.auth.core [None req-58588ddd-a11f-4e9c-8754-95a063925a21 None None] MFA Rules not processed for user `bac4e6b76fe64217ac8fa44e71cffe6a`. Rule list: `[]` (Enabled: `True`). {{(pid=80143) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 11:03:47.351180 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-58588ddd-a11f-4e9c-8754-95a063925a21 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 11:03:47.360056 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1203/2434] 10.222.0.25 () {58 vars in 958 bytes} [Sun Aug 7 11:03:47 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 07 11:03:47.364351 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b0a4eec-e81d-45d2-8169-237be4dc91d0 None None] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 11:03:47.375489 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b0a4eec-e81d-45d2-8169-237be4dc91d0 None None] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 11:03:47.380271 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb4f25bc-c706-44c6-9439-0944b0e16a8f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5ac47e3b09f43d19a23a0f457fe9e23', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 11:03:47.380904 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb4f25bc-c706-44c6-9439-0944b0e16a8f admin admin] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 11:03:47.381186 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb4f25bc-c706-44c6-9439-0944b0e16a8f admin admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 11:03:47.381462 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb4f25bc-c706-44c6-9439-0944b0e16a8f admin admin] PATH_INFO: `/v3/domains` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 11:03:47.382108 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb4f25bc-c706-44c6-9439-0944b0e16a8f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80144) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 11:03:47.382414 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb4f25bc-c706-44c6-9439-0944b0e16a8f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 11:03:47.384065 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb4f25bc-c706-44c6-9439-0944b0e16a8f admin admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 11:03:47.395698 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 1232/2435] 10.222.0.25 () {58 vars in 1153 bytes} [Sun Aug 7 11:03:47 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 11:03:47.399978 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58588ddd-a11f-4e9c-8754-95a063925a21 None None] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 11:03:47.409332 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58588ddd-a11f-4e9c-8754-95a063925a21 None None] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 11:03:47.414065 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a0c8419-5830-4af1-952d-ed6837643c41 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5ac47e3b09f43d19a23a0f457fe9e23', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 11:03:47.414675 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a0c8419-5830-4af1-952d-ed6837643c41 admin admin] REQUEST_METHOD: `POST` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 11:03:47.414986 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a0c8419-5830-4af1-952d-ed6837643c41 admin admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 11:03:47.415262 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a0c8419-5830-4af1-952d-ed6837643c41 admin admin] PATH_INFO: `/v3/projects` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 11:03:47.415904 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a0c8419-5830-4af1-952d-ed6837643c41 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 11:03:47.417790 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a0c8419-5830-4af1-952d-ed6837643c41 admin admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 11:03:47.423163 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 11:03:47.447740 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1204/2436] 10.222.0.25 () {60 vars in 1153 bytes} [Sun Aug 7 11:03:47 2022] POST /identity/v3/projects => generated 387 bytes in 48 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 07 11:03:47.452070 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb4f25bc-c706-44c6-9439-0944b0e16a8f admin admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 11:03:47.463953 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb4f25bc-c706-44c6-9439-0944b0e16a8f admin admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 11:03:47.521041 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1542aed7-09ac-493d-b71c-1bd489ea7f6d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5ac47e3b09f43d19a23a0f457fe9e23', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 11:03:47.521647 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1542aed7-09ac-493d-b71c-1bd489ea7f6d admin admin] REQUEST_METHOD: `POST` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 11:03:47.521934 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1542aed7-09ac-493d-b71c-1bd489ea7f6d admin admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 11:03:47.522214 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1542aed7-09ac-493d-b71c-1bd489ea7f6d admin admin] PATH_INFO: `/v3/users` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 11:03:47.522845 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1542aed7-09ac-493d-b71c-1bd489ea7f6d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 11:03:47.524783 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1542aed7-09ac-493d-b71c-1bd489ea7f6d admin admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 11:03:47.546375 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 1233/2437] 10.222.0.25 () {60 vars in 1144 bytes} [Sun Aug 7 11:03:47 2022] POST /identity/v3/users => generated 294 bytes in 95 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 07 11:03:47.550506 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a0c8419-5830-4af1-952d-ed6837643c41 admin admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 11:03:47.560870 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a0c8419-5830-4af1-952d-ed6837643c41 admin admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 11:03:47.565442 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6383879b-d8da-4378-b2e1-d58fb8806e35 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5ac47e3b09f43d19a23a0f457fe9e23', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 11:03:47.566040 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6383879b-d8da-4378-b2e1-d58fb8806e35 admin admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 11:03:47.566322 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6383879b-d8da-4378-b2e1-d58fb8806e35 admin admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 11:03:47.566598 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6383879b-d8da-4378-b2e1-d58fb8806e35 admin admin] PATH_INFO: `/v3/roles` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 11:03:47.567280 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6383879b-d8da-4378-b2e1-d58fb8806e35 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 11:03:47.568864 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6383879b-d8da-4378-b2e1-d58fb8806e35 admin admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 11:03:47.578880 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1205/2438] 10.222.0.25 () {58 vars in 1122 bytes} [Sun Aug 7 11:03:47 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 07 11:03:47.582666 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1542aed7-09ac-493d-b71c-1bd489ea7f6d admin admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 11:03:47.593077 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1542aed7-09ac-493d-b71c-1bd489ea7f6d admin admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 11:03:47.597771 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84b166bb-6cf9-495a-8f91-4acfc0e291bf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5ac47e3b09f43d19a23a0f457fe9e23', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 11:03:47.598387 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84b166bb-6cf9-495a-8f91-4acfc0e291bf admin admin] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 11:03:47.598665 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84b166bb-6cf9-495a-8f91-4acfc0e291bf admin admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 11:03:47.598959 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84b166bb-6cf9-495a-8f91-4acfc0e291bf admin admin] PATH_INFO: `/v3/roles` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 11:03:47.599600 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84b166bb-6cf9-495a-8f91-4acfc0e291bf admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 11:03:47.601159 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84b166bb-6cf9-495a-8f91-4acfc0e291bf admin admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 11:03:47.612480 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 1234/2439] 10.222.0.25 () {58 vars in 1122 bytes} [Sun Aug 7 11:03:47 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 07 11:03:47.621604 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6383879b-d8da-4378-b2e1-d58fb8806e35 admin admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 11:03:47.632072 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6383879b-d8da-4378-b2e1-d58fb8806e35 admin admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 11:03:47.637066 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-553e2139-c981-43cd-9d70-6fed89de607e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5ac47e3b09f43d19a23a0f457fe9e23', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 11:03:47.638056 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-553e2139-c981-43cd-9d70-6fed89de607e admin admin] REQUEST_METHOD: `PUT` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 11:03:47.638337 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-553e2139-c981-43cd-9d70-6fed89de607e admin admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 11:03:47.638615 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-553e2139-c981-43cd-9d70-6fed89de607e admin admin] PATH_INFO: `/v3/projects/0755d5c9e2a1402bbf6a5ec53b79aca2/users/0b609ab66a5b47f892135d5cb24dd120/roles/b98dfb3b9fee4644af2887a3ac3fd2e6` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 11:03:47.653514 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-553e2139-c981-43cd-9d70-6fed89de607e admin admin] RBAC: Authorizing `identity:create_grant(project_id=0755d5c9e2a1402bbf6a5ec53b79aca2, user_id=0b609ab66a5b47f892135d5cb24dd120, role_id=b98dfb3b9fee4644af2887a3ac3fd2e6)` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 11:03:47.656356 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-553e2139-c981-43cd-9d70-6fed89de607e admin admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 11:03:47.669917 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1206/2440] 10.222.0.25 () {60 vars in 1483 bytes} [Sun Aug 7 11:03:47 2022] PUT /identity/v3/projects/0755d5c9e2a1402bbf6a5ec53b79aca2/users/0b609ab66a5b47f892135d5cb24dd120/roles/b98dfb3b9fee4644af2887a3ac3fd2e6 => generated 0 bytes in 54 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 07 11:03:47.681376 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb21405c-7248-40b0-a47f-81d7eefac344 None None] REQUEST_METHOD: `POST` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 11:03:47.681666 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb21405c-7248-40b0-a47f-81d7eefac344 None None] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 11:03:47.681942 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb21405c-7248-40b0-a47f-81d7eefac344 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 11:03:47.709277 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.auth.core [None req-cb21405c-7248-40b0-a47f-81d7eefac344 None None] MFA Rules not processed for user `0b609ab66a5b47f892135d5cb24dd120`. Rule list: `[]` (Enabled: `True`). {{(pid=80144) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 11:03:47.710417 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.fernet_utils [None req-cb21405c-7248-40b0-a47f-81d7eefac344 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80144) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 11:03:47.736563 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=93,keystone:INSERT=4,keystone:DELETE=3 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 11:03:47.784627 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 1235/2441] 10.222.0.25 () {58 vars in 958 bytes} [Sun Aug 7 11:03:47 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 07 11:03:47.789074 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-553e2139-c981-43cd-9d70-6fed89de607e admin admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 11:03:47.850506 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-553e2139-c981-43cd-9d70-6fed89de607e admin admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 11:03:47.855264 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcc0c8c1-9927-41b8-b345-38fa76ba3972 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5ac47e3b09f43d19a23a0f457fe9e23', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 11:03:47.855870 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcc0c8c1-9927-41b8-b345-38fa76ba3972 admin admin] REQUEST_METHOD: `POST` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 11:03:47.856150 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcc0c8c1-9927-41b8-b345-38fa76ba3972 admin admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 11:03:47.856423 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcc0c8c1-9927-41b8-b345-38fa76ba3972 admin admin] PATH_INFO: `/v3/projects` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 11:03:47.857048 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcc0c8c1-9927-41b8-b345-38fa76ba3972 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 11:03:47.859224 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcc0c8c1-9927-41b8-b345-38fa76ba3972 admin admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 11:03:47.884957 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1207/2442] 10.222.0.25 () {60 vars in 1153 bytes} [Sun Aug 7 11:03:47 2022] POST /identity/v3/projects => generated 383 bytes in 96 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 07 11:03:47.889607 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb21405c-7248-40b0-a47f-81d7eefac344 None None] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 11:03:47.900643 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb21405c-7248-40b0-a47f-81d7eefac344 None None] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 11:03:47.960564 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffcd3a3f-f228-4204-8827-648cb4a63426 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5ac47e3b09f43d19a23a0f457fe9e23', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 11:03:47.961099 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffcd3a3f-f228-4204-8827-648cb4a63426 admin admin] REQUEST_METHOD: `POST` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 11:03:47.961379 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffcd3a3f-f228-4204-8827-648cb4a63426 admin admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 11:03:47.961651 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffcd3a3f-f228-4204-8827-648cb4a63426 admin admin] PATH_INFO: `/v3/users` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 11:03:47.962616 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffcd3a3f-f228-4204-8827-648cb4a63426 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 11:03:47.964696 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffcd3a3f-f228-4204-8827-648cb4a63426 admin admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 11:03:47.991602 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 1236/2443] 10.222.0.25 () {60 vars in 1144 bytes} [Sun Aug 7 11:03:47 2022] POST /identity/v3/users => generated 298 bytes in 100 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 07 11:03:47.992885 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcc0c8c1-9927-41b8-b345-38fa76ba3972 admin admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 11:03:48.003536 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcc0c8c1-9927-41b8-b345-38fa76ba3972 admin admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 11:03:48.008226 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4191cc6-9fa7-4cdc-b750-feefd5b6d3e9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5ac47e3b09f43d19a23a0f457fe9e23', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 11:03:48.008827 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4191cc6-9fa7-4cdc-b750-feefd5b6d3e9 admin admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 11:03:48.009106 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4191cc6-9fa7-4cdc-b750-feefd5b6d3e9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 11:03:48.009377 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4191cc6-9fa7-4cdc-b750-feefd5b6d3e9 admin admin] PATH_INFO: `/v3/roles` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 11:03:48.010050 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4191cc6-9fa7-4cdc-b750-feefd5b6d3e9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 11:03:48.011807 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4191cc6-9fa7-4cdc-b750-feefd5b6d3e9 admin admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 11:03:48.023363 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1208/2444] 10.222.0.25 () {58 vars in 1122 bytes} [Sun Aug 7 11:03:47 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 07 11:03:48.028318 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffcd3a3f-f228-4204-8827-648cb4a63426 admin admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 11:03:48.038592 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffcd3a3f-f228-4204-8827-648cb4a63426 admin admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 11:03:48.043282 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-179c33fc-6137-4942-a92a-c5232c28a7d9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5ac47e3b09f43d19a23a0f457fe9e23', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 11:03:48.044212 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-179c33fc-6137-4942-a92a-c5232c28a7d9 admin admin] REQUEST_METHOD: `PUT` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 11:03:48.044498 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-179c33fc-6137-4942-a92a-c5232c28a7d9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 11:03:48.044778 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-179c33fc-6137-4942-a92a-c5232c28a7d9 admin admin] PATH_INFO: `/v3/projects/0066665ede414b158726f0e4f3568a7e/users/acb7b2ed6a1f4d6abe7657182e1a140c/roles/81b1f1bcee2a473088fbbd327c1c5b23` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 11:03:48.059704 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-179c33fc-6137-4942-a92a-c5232c28a7d9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0066665ede414b158726f0e4f3568a7e, user_id=acb7b2ed6a1f4d6abe7657182e1a140c, role_id=81b1f1bcee2a473088fbbd327c1c5b23)` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 11:03:48.062389 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-179c33fc-6137-4942-a92a-c5232c28a7d9 admin admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 11:03:48.077055 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 1237/2445] 10.222.0.25 () {60 vars in 1483 bytes} [Sun Aug 7 11:03:48 2022] PUT /identity/v3/projects/0066665ede414b158726f0e4f3568a7e/users/acb7b2ed6a1f4d6abe7657182e1a140c/roles/81b1f1bcee2a473088fbbd327c1c5b23 => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 07 11:03:48.081143 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4191cc6-9fa7-4cdc-b750-feefd5b6d3e9 admin admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 11:03:48.092885 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4191cc6-9fa7-4cdc-b750-feefd5b6d3e9 admin admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 11:03:48.153836 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ef91f4e-32fb-4e0c-8d5b-d4128289fe12 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5ac47e3b09f43d19a23a0f457fe9e23', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 11:03:48.154642 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ef91f4e-32fb-4e0c-8d5b-d4128289fe12 admin admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 11:03:48.155096 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ef91f4e-32fb-4e0c-8d5b-d4128289fe12 admin admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 11:03:48.155491 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ef91f4e-32fb-4e0c-8d5b-d4128289fe12 admin admin] PATH_INFO: `/v3/roles` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 11:03:48.156216 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ef91f4e-32fb-4e0c-8d5b-d4128289fe12 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 11:03:48.157923 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ef91f4e-32fb-4e0c-8d5b-d4128289fe12 admin admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 11:03:48.170610 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1209/2446] 10.222.0.25 () {58 vars in 1122 bytes} [Sun Aug 7 11:03:48 2022] GET /identity/v3/roles => generated 1420 bytes in 90 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 07 11:03:48.175041 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-179c33fc-6137-4942-a92a-c5232c28a7d9 admin admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 11:03:48.186252 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-179c33fc-6137-4942-a92a-c5232c28a7d9 admin admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 11:03:48.191036 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7a03789-8eec-4321-9320-aa62b54b80e2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5ac47e3b09f43d19a23a0f457fe9e23', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 11:03:48.191749 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7a03789-8eec-4321-9320-aa62b54b80e2 admin admin] REQUEST_METHOD: `PUT` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 11:03:48.192038 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7a03789-8eec-4321-9320-aa62b54b80e2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 11:03:48.192317 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7a03789-8eec-4321-9320-aa62b54b80e2 admin admin] PATH_INFO: `/v3/projects/0066665ede414b158726f0e4f3568a7e/users/acb7b2ed6a1f4d6abe7657182e1a140c/roles/81b1f1bcee2a473088fbbd327c1c5b23` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 11:03:48.194242 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7a03789-8eec-4321-9320-aa62b54b80e2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0066665ede414b158726f0e4f3568a7e, user_id=acb7b2ed6a1f4d6abe7657182e1a140c, role_id=81b1f1bcee2a473088fbbd327c1c5b23)` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 11:03:48.196918 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7a03789-8eec-4321-9320-aa62b54b80e2 admin admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 11:03:48.208654 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 1238/2447] 10.222.0.25 () {60 vars in 1483 bytes} [Sun Aug 7 11:03:48 2022] PUT /identity/v3/projects/0066665ede414b158726f0e4f3568a7e/users/acb7b2ed6a1f4d6abe7657182e1a140c/roles/81b1f1bcee2a473088fbbd327c1c5b23 => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 07 11:03:48.219610 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e6e4543-8e91-4be9-b87e-eaed58f18a46 None None] REQUEST_METHOD: `POST` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 11:03:48.219864 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e6e4543-8e91-4be9-b87e-eaed58f18a46 None None] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 11:03:48.220149 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e6e4543-8e91-4be9-b87e-eaed58f18a46 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 11:03:48.248775 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.auth.core [None req-7e6e4543-8e91-4be9-b87e-eaed58f18a46 None None] MFA Rules not processed for user `acb7b2ed6a1f4d6abe7657182e1a140c`. Rule list: `[]` (Enabled: `True`). {{(pid=80143) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 11:03:48.250002 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-7e6e4543-8e91-4be9-b87e-eaed58f18a46 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 11:03:48.258221 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 11:03:48.322514 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1210/2448] 10.222.0.25 () {58 vars in 958 bytes} [Sun Aug 7 11:03:48 2022] POST /identity/v3/auth/tokens => generated 3327 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 07 11:03:48.334125 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7a03789-8eec-4321-9320-aa62b54b80e2 admin admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 11:03:48.335856 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.fernet_utils [None req-e7a03789-8eec-4321-9320-aa62b54b80e2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80144) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 11:03:48.396173 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7a03789-8eec-4321-9320-aa62b54b80e2 admin admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 11:03:48.399918 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46ddeacd-d662-49d7-9d87-05d2c18abaaf service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88dbd6dfd1164994a3e16da553561910', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6de1f3cac02c4bcdb05dd2337176033d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 11:03:48.400550 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46ddeacd-d662-49d7-9d87-05d2c18abaaf service cinder] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 11:03:48.400894 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46ddeacd-d662-49d7-9d87-05d2c18abaaf service cinder] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 11:03:48.401169 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46ddeacd-d662-49d7-9d87-05d2c18abaaf service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 11:03:48.416666 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46ddeacd-d662-49d7-9d87-05d2c18abaaf service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 11:03:48.418303 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46ddeacd-d662-49d7-9d87-05d2c18abaaf service cinder] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 11:03:48.428714 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 1239/2449] 10.222.0.25 () {60 vars in 1365 bytes} [Sun Aug 7 11:03:48 2022] GET /identity/v3/auth/tokens => generated 3327 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 07 11:03:49.021530 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e6e4543-8e91-4be9-b87e-eaed58f18a46 None None] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 11:03:49.085044 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e6e4543-8e91-4be9-b87e-eaed58f18a46 None None] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 11:03:49.092973 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dc7114a-e7dd-4a63-8077-8174cfa23d96 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5ac47e3b09f43d19a23a0f457fe9e23', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 11:03:49.093730 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dc7114a-e7dd-4a63-8077-8174cfa23d96 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 11:03:49.094008 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dc7114a-e7dd-4a63-8077-8174cfa23d96 admin admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 11:03:49.094278 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dc7114a-e7dd-4a63-8077-8174cfa23d96 admin admin] PATH_INFO: `/v3/users/0b609ab66a5b47f892135d5cb24dd120` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 11:03:49.095439 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dc7114a-e7dd-4a63-8077-8174cfa23d96 admin admin] RBAC: Authorizing `identity:delete_user(user_id=0b609ab66a5b47f892135d5cb24dd120)` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 11:03:49.097421 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dc7114a-e7dd-4a63-8077-8174cfa23d96 admin admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 11:03:49.145820 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-0dc7114a-e7dd-4a63-8077-8174cfa23d96 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0b609ab66a5b47f892135d5cb24dd120'} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 11:03:49.153114 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-0dc7114a-e7dd-4a63-8077-8174cfa23d96 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0b609ab66a5b47f892135d5cb24dd120'} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 11:03:49.165338 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-0dc7114a-e7dd-4a63-8077-8174cfa23d96 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0b609ab66a5b47f892135d5cb24dd120'} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 11:03:49.166051 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-0dc7114a-e7dd-4a63-8077-8174cfa23d96 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0b609ab66a5b47f892135d5cb24dd120'} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 11:03:49.176192 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-0dc7114a-e7dd-4a63-8077-8174cfa23d96 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0b609ab66a5b47f892135d5cb24dd120'} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 11:03:49.182314 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1211/2450] 10.222.0.25 () {58 vars in 1224 bytes} [Sun Aug 7 11:03:49 2022] DELETE /identity/v3/users/0b609ab66a5b47f892135d5cb24dd120 => generated 0 bytes in 161 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 07 11:03:49.186944 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29afbd3a-5bf3-4b65-940f-5d615ae75158 None None] REQUEST_METHOD: `POST` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 11:03:49.187407 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29afbd3a-5bf3-4b65-940f-5d615ae75158 None None] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 11:03:49.187532 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29afbd3a-5bf3-4b65-940f-5d615ae75158 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 11:03:49.206589 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.auth.core [None req-29afbd3a-5bf3-4b65-940f-5d615ae75158 None None] MFA Rules not processed for user `bac4e6b76fe64217ac8fa44e71cffe6a`. Rule list: `[]` (Enabled: `True`). {{(pid=80144) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 11:03:49.207804 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.fernet_utils [None req-29afbd3a-5bf3-4b65-940f-5d615ae75158 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80144) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 11:03:49.221403 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 11:03:49.270550 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 1240/2451] 10.222.0.25 () {58 vars in 958 bytes} [Sun Aug 7 11:03:49 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 07 11:03:49.281975 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dc7114a-e7dd-4a63-8077-8174cfa23d96 admin admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 11:03:49.283532 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-0dc7114a-e7dd-4a63-8077-8174cfa23d96 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 11:03:49.313153 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 11:03:49.345750 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dc7114a-e7dd-4a63-8077-8174cfa23d96 admin admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 11:03:49.349494 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5288f92-95a0-4113-aaac-f7c0d445b323 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be1283c9bf47424c9aa3ad27cab03027', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6de1f3cac02c4bcdb05dd2337176033d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 11:03:49.350101 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5288f92-95a0-4113-aaac-f7c0d445b323 service neutron] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 11:03:49.350389 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5288f92-95a0-4113-aaac-f7c0d445b323 service neutron] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 11:03:49.350665 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5288f92-95a0-4113-aaac-f7c0d445b323 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 11:03:49.360062 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5288f92-95a0-4113-aaac-f7c0d445b323 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 11:03:49.361546 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5288f92-95a0-4113-aaac-f7c0d445b323 service neutron] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 11:03:49.371207 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1212/2452] 10.222.0.25 () {60 vars in 1365 bytes} [Sun Aug 7 11:03:49 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 07 11:03:49.643836 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29afbd3a-5bf3-4b65-940f-5d615ae75158 None None] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 11:03:49.645289 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.fernet_utils [None req-29afbd3a-5bf3-4b65-940f-5d615ae75158 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80144) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 11:03:49.658026 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29afbd3a-5bf3-4b65-940f-5d615ae75158 None None] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 11:03:49.662668 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60f3e350-7bb8-4502-b68e-508d00e17dfa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5ac47e3b09f43d19a23a0f457fe9e23', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 11:03:49.663357 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60f3e350-7bb8-4502-b68e-508d00e17dfa admin admin] REQUEST_METHOD: `DELETE` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 11:03:49.663637 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60f3e350-7bb8-4502-b68e-508d00e17dfa admin admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 11:03:49.663912 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60f3e350-7bb8-4502-b68e-508d00e17dfa admin admin] PATH_INFO: `/v3/projects/0755d5c9e2a1402bbf6a5ec53b79aca2` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 11:03:49.665040 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60f3e350-7bb8-4502-b68e-508d00e17dfa admin admin] RBAC: Authorizing `identity:delete_project(project_id=0755d5c9e2a1402bbf6a5ec53b79aca2)` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 11:03:49.667010 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60f3e350-7bb8-4502-b68e-508d00e17dfa admin admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 11:03:49.718311 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.notifications [None req-60f3e350-7bb8-4502-b68e-508d00e17dfa admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0755d5c9e2a1402bbf6a5ec53b79aca2', 'request_id': 'req-60f3e350-7bb8-4502-b68e-508d00e17dfa'} {{(pid=80144) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 11:03:49.724739 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.notifications [None req-60f3e350-7bb8-4502-b68e-508d00e17dfa admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0755d5c9e2a1402bbf6a5ec53b79aca2', 'request_id': 'req-60f3e350-7bb8-4502-b68e-508d00e17dfa'} {{(pid=80144) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 11:03:49.732588 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.notifications [None req-60f3e350-7bb8-4502-b68e-508d00e17dfa admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0755d5c9e2a1402bbf6a5ec53b79aca2', 'request_id': 'req-60f3e350-7bb8-4502-b68e-508d00e17dfa'} {{(pid=80144) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 11:03:49.743884 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.notifications [None req-60f3e350-7bb8-4502-b68e-508d00e17dfa admin admin] The token cache is being invalidate because project 0755d5c9e2a1402bbf6a5ec53b79aca2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80144) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 07 11:03:49.744178 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.notifications [None req-60f3e350-7bb8-4502-b68e-508d00e17dfa admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80144) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 11:03:49.744821 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.notifications [None req-60f3e350-7bb8-4502-b68e-508d00e17dfa admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80144) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 11:03:49.746819 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 1241/2453] 10.222.0.25 () {58 vars in 1233 bytes} [Sun Aug 7 11:03:49 2022] DELETE /identity/v3/projects/0755d5c9e2a1402bbf6a5ec53b79aca2 => generated 0 bytes in 103 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 07 11:03:49.750782 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5288f92-95a0-4113-aaac-f7c0d445b323 service neutron] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 11:03:49.760601 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5288f92-95a0-4113-aaac-f7c0d445b323 service neutron] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 11:03:49.816974 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44b625aa-d055-4c19-9d57-f4693889600a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5ac47e3b09f43d19a23a0f457fe9e23', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 11:03:49.817524 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44b625aa-d055-4c19-9d57-f4693889600a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 11:03:49.817810 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44b625aa-d055-4c19-9d57-f4693889600a admin admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 11:03:49.818090 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44b625aa-d055-4c19-9d57-f4693889600a admin admin] PATH_INFO: `/v3/users/acb7b2ed6a1f4d6abe7657182e1a140c` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 11:03:49.819269 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44b625aa-d055-4c19-9d57-f4693889600a admin admin] RBAC: Authorizing `identity:delete_user(user_id=acb7b2ed6a1f4d6abe7657182e1a140c)` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 11:03:49.821185 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44b625aa-d055-4c19-9d57-f4693889600a admin admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 11:03:49.866321 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-44b625aa-d055-4c19-9d57-f4693889600a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'acb7b2ed6a1f4d6abe7657182e1a140c'} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 11:03:49.873788 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-44b625aa-d055-4c19-9d57-f4693889600a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'acb7b2ed6a1f4d6abe7657182e1a140c'} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 11:03:49.886026 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-44b625aa-d055-4c19-9d57-f4693889600a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'acb7b2ed6a1f4d6abe7657182e1a140c'} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 11:03:49.886727 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-44b625aa-d055-4c19-9d57-f4693889600a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'acb7b2ed6a1f4d6abe7657182e1a140c'} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 11:03:49.894757 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-44b625aa-d055-4c19-9d57-f4693889600a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'acb7b2ed6a1f4d6abe7657182e1a140c'} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 11:03:49.900611 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1213/2454] 10.222.0.25 () {58 vars in 1224 bytes} [Sun Aug 7 11:03:49 2022] DELETE /identity/v3/users/acb7b2ed6a1f4d6abe7657182e1a140c => generated 0 bytes in 150 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 07 11:03:50.178179 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60f3e350-7bb8-4502-b68e-508d00e17dfa admin admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 11:03:50.179960 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.fernet_utils [None req-60f3e350-7bb8-4502-b68e-508d00e17dfa admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80144) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 11:03:50.196181 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60f3e350-7bb8-4502-b68e-508d00e17dfa admin admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 11:03:50.271065 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-428ea98a-a675-45ac-a055-c2a26013e1ec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5ac47e3b09f43d19a23a0f457fe9e23', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 11:03:50.271760 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-428ea98a-a675-45ac-a055-c2a26013e1ec admin admin] REQUEST_METHOD: `DELETE` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 11:03:50.272046 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-428ea98a-a675-45ac-a055-c2a26013e1ec admin admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 11:03:50.272367 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-428ea98a-a675-45ac-a055-c2a26013e1ec admin admin] PATH_INFO: `/v3/projects/0066665ede414b158726f0e4f3568a7e` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 11:03:50.273540 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-428ea98a-a675-45ac-a055-c2a26013e1ec admin admin] RBAC: Authorizing `identity:delete_project(project_id=0066665ede414b158726f0e4f3568a7e)` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 11:03:50.275767 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-428ea98a-a675-45ac-a055-c2a26013e1ec admin admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 11:03:50.322499 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 11:03:50.325895 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.notifications [None req-428ea98a-a675-45ac-a055-c2a26013e1ec admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0066665ede414b158726f0e4f3568a7e', 'request_id': 'req-428ea98a-a675-45ac-a055-c2a26013e1ec'} {{(pid=80144) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 11:03:50.332714 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.notifications [None req-428ea98a-a675-45ac-a055-c2a26013e1ec admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0066665ede414b158726f0e4f3568a7e', 'request_id': 'req-428ea98a-a675-45ac-a055-c2a26013e1ec'} {{(pid=80144) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 11:03:50.342741 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.notifications [None req-428ea98a-a675-45ac-a055-c2a26013e1ec admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0066665ede414b158726f0e4f3568a7e', 'request_id': 'req-428ea98a-a675-45ac-a055-c2a26013e1ec'} {{(pid=80144) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 11:03:50.354052 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.notifications [None req-428ea98a-a675-45ac-a055-c2a26013e1ec admin admin] The token cache is being invalidate because project 0066665ede414b158726f0e4f3568a7e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80144) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 07 11:03:50.354349 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.notifications [None req-428ea98a-a675-45ac-a055-c2a26013e1ec admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80144) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 11:03:50.355033 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.notifications [None req-428ea98a-a675-45ac-a055-c2a26013e1ec admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80144) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 11:03:50.357289 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 1242/2455] 10.222.0.25 () {58 vars in 1233 bytes} [Sun Aug 7 11:03:50 2022] DELETE /identity/v3/projects/0066665ede414b158726f0e4f3568a7e => generated 0 bytes in 180 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 07 11:03:50.363174 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c7e02cb-083c-469b-a0b1-05a0ff812c22 None None] REQUEST_METHOD: `POST` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 11:03:50.363680 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c7e02cb-083c-469b-a0b1-05a0ff812c22 None None] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 11:03:50.363927 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c7e02cb-083c-469b-a0b1-05a0ff812c22 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 11:03:50.383899 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.auth.core [None req-9c7e02cb-083c-469b-a0b1-05a0ff812c22 None None] MFA Rules not processed for user `bac4e6b76fe64217ac8fa44e71cffe6a`. Rule list: `[]` (Enabled: `True`). {{(pid=80143) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 11:03:50.385202 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-9c7e02cb-083c-469b-a0b1-05a0ff812c22 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 11:03:50.440685 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 11:03:50.462256 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1214/2456] 10.222.0.25 () {58 vars in 958 bytes} [Sun Aug 7 11:03:50 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 07 11:03:50.471008 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80fb171b-72f8-4c3d-9d43-4769d9aa0b35 None None] REQUEST_METHOD: `POST` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 11:03:50.471383 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80fb171b-72f8-4c3d-9d43-4769d9aa0b35 None None] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 11:03:50.471723 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80fb171b-72f8-4c3d-9d43-4769d9aa0b35 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 11:03:50.495701 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.auth.core [None req-80fb171b-72f8-4c3d-9d43-4769d9aa0b35 None None] MFA Rules not processed for user `bac4e6b76fe64217ac8fa44e71cffe6a`. Rule list: `[]` (Enabled: `True`). {{(pid=80144) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 11:03:50.497039 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.fernet_utils [None req-80fb171b-72f8-4c3d-9d43-4769d9aa0b35 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80144) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 11:03:50.508788 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 1243/2457] 10.222.0.25 () {58 vars in 958 bytes} [Sun Aug 7 11:03:50 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 07 11:03:50.510131 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c7e02cb-083c-469b-a0b1-05a0ff812c22 None None] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 11:03:50.522564 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c7e02cb-083c-469b-a0b1-05a0ff812c22 None None] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 11:03:50.529139 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1feb55ad-9d3b-4e7d-8e50-be9fb53d0c0b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5ac47e3b09f43d19a23a0f457fe9e23', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 11:03:50.529860 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1feb55ad-9d3b-4e7d-8e50-be9fb53d0c0b admin admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 11:03:50.530275 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1feb55ad-9d3b-4e7d-8e50-be9fb53d0c0b admin admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 11:03:50.530654 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1feb55ad-9d3b-4e7d-8e50-be9fb53d0c0b admin admin] PATH_INFO: `/v3/domains` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 11:03:50.531425 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1feb55ad-9d3b-4e7d-8e50-be9fb53d0c0b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80143) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 11:03:50.531792 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1feb55ad-9d3b-4e7d-8e50-be9fb53d0c0b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 11:03:50.534009 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1feb55ad-9d3b-4e7d-8e50-be9fb53d0c0b admin admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 11:03:50.550983 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1215/2458] 10.222.0.25 () {58 vars in 1153 bytes} [Sun Aug 7 11:03:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 11:03:50.552789 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80fb171b-72f8-4c3d-9d43-4769d9aa0b35 None None] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 11:03:50.563488 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80fb171b-72f8-4c3d-9d43-4769d9aa0b35 None None] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 11:03:50.570064 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c161d60-f47b-4afa-b311-ef0c609c88d4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5ac47e3b09f43d19a23a0f457fe9e23', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 11:03:50.571260 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c161d60-f47b-4afa-b311-ef0c609c88d4 admin admin] REQUEST_METHOD: `POST` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 11:03:50.571575 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c161d60-f47b-4afa-b311-ef0c609c88d4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 11:03:50.571858 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c161d60-f47b-4afa-b311-ef0c609c88d4 admin admin] PATH_INFO: `/v3/projects` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 11:03:50.572496 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c161d60-f47b-4afa-b311-ef0c609c88d4 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 11:03:50.574643 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c161d60-f47b-4afa-b311-ef0c609c88d4 admin admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 11:03:50.604901 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 1244/2459] 10.222.0.25 () {60 vars in 1153 bytes} [Sun Aug 7 11:03:50 2022] POST /identity/v3/projects => generated 381 bytes in 52 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 07 11:03:50.609812 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1feb55ad-9d3b-4e7d-8e50-be9fb53d0c0b admin admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 11:03:50.620085 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1feb55ad-9d3b-4e7d-8e50-be9fb53d0c0b admin admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 11:03:50.675887 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-285f3a31-b6e5-4f9e-a92c-e02b2938282d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5ac47e3b09f43d19a23a0f457fe9e23', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 11:03:50.676492 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-285f3a31-b6e5-4f9e-a92c-e02b2938282d admin admin] REQUEST_METHOD: `POST` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 11:03:50.676780 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-285f3a31-b6e5-4f9e-a92c-e02b2938282d admin admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 11:03:50.677060 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-285f3a31-b6e5-4f9e-a92c-e02b2938282d admin admin] PATH_INFO: `/v3/users` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 11:03:50.677676 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-285f3a31-b6e5-4f9e-a92c-e02b2938282d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 11:03:50.679721 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-285f3a31-b6e5-4f9e-a92c-e02b2938282d admin admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 11:03:50.700534 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1216/2460] 10.222.0.25 () {60 vars in 1144 bytes} [Sun Aug 7 11:03:50 2022] POST /identity/v3/users => generated 291 bytes in 91 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 07 11:03:50.704681 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c161d60-f47b-4afa-b311-ef0c609c88d4 admin admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 11:03:50.715687 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c161d60-f47b-4afa-b311-ef0c609c88d4 admin admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 11:03:50.720562 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60b1581a-fbde-4a70-8122-1df85b332c9a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5ac47e3b09f43d19a23a0f457fe9e23', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 11:03:50.721161 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60b1581a-fbde-4a70-8122-1df85b332c9a admin admin] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 11:03:50.721472 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60b1581a-fbde-4a70-8122-1df85b332c9a admin admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 11:03:50.721774 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60b1581a-fbde-4a70-8122-1df85b332c9a admin admin] PATH_INFO: `/v3/roles` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 11:03:50.722428 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60b1581a-fbde-4a70-8122-1df85b332c9a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 11:03:50.724037 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60b1581a-fbde-4a70-8122-1df85b332c9a admin admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 11:03:50.734866 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 1245/2461] 10.222.0.25 () {58 vars in 1122 bytes} [Sun Aug 7 11:03:50 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 07 11:03:50.738775 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-285f3a31-b6e5-4f9e-a92c-e02b2938282d admin admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 11:03:50.748595 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-285f3a31-b6e5-4f9e-a92c-e02b2938282d admin admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 11:03:50.753836 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9b714a3-6115-41d8-8f44-a0c0de42e05a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5ac47e3b09f43d19a23a0f457fe9e23', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 11:03:50.754446 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9b714a3-6115-41d8-8f44-a0c0de42e05a admin admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 11:03:50.754713 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9b714a3-6115-41d8-8f44-a0c0de42e05a admin admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 11:03:50.755017 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9b714a3-6115-41d8-8f44-a0c0de42e05a admin admin] PATH_INFO: `/v3/roles` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 11:03:50.755676 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9b714a3-6115-41d8-8f44-a0c0de42e05a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 11:03:50.757271 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9b714a3-6115-41d8-8f44-a0c0de42e05a admin admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 11:03:50.768256 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1217/2462] 10.222.0.25 () {58 vars in 1122 bytes} [Sun Aug 7 11:03:50 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 07 11:03:50.772692 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60b1581a-fbde-4a70-8122-1df85b332c9a admin admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 11:03:50.783696 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60b1581a-fbde-4a70-8122-1df85b332c9a admin admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 11:03:50.788368 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d676763d-e496-471b-a613-169dbd7bfd4c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5ac47e3b09f43d19a23a0f457fe9e23', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 11:03:50.789011 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d676763d-e496-471b-a613-169dbd7bfd4c admin admin] REQUEST_METHOD: `PUT` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 11:03:50.789293 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d676763d-e496-471b-a613-169dbd7bfd4c admin admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 11:03:50.789570 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d676763d-e496-471b-a613-169dbd7bfd4c admin admin] PATH_INFO: `/v3/projects/7a9a8f92281f4380b5ff21ece0aaaff3/users/c41e1ce5aeda49c4ade51ccafa4eb3e2/roles/b98dfb3b9fee4644af2887a3ac3fd2e6` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 11:03:50.805154 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d676763d-e496-471b-a613-169dbd7bfd4c admin admin] RBAC: Authorizing `identity:create_grant(project_id=7a9a8f92281f4380b5ff21ece0aaaff3, user_id=c41e1ce5aeda49c4ade51ccafa4eb3e2, role_id=b98dfb3b9fee4644af2887a3ac3fd2e6)` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 11:03:50.807810 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d676763d-e496-471b-a613-169dbd7bfd4c admin admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 11:03:50.823364 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 1246/2463] 10.222.0.25 () {60 vars in 1483 bytes} [Sun Aug 7 11:03:50 2022] PUT /identity/v3/projects/7a9a8f92281f4380b5ff21ece0aaaff3/users/c41e1ce5aeda49c4ade51ccafa4eb3e2/roles/b98dfb3b9fee4644af2887a3ac3fd2e6 => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 07 11:03:50.836096 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3c028da-0c58-4bfe-879f-a0b269f79976 None None] REQUEST_METHOD: `POST` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 11:03:50.836389 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3c028da-0c58-4bfe-879f-a0b269f79976 None None] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 11:03:50.836668 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3c028da-0c58-4bfe-879f-a0b269f79976 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 11:03:50.866285 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.auth.core [None req-c3c028da-0c58-4bfe-879f-a0b269f79976 None None] MFA Rules not processed for user `c41e1ce5aeda49c4ade51ccafa4eb3e2`. Rule list: `[]` (Enabled: `True`). {{(pid=80143) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 11:03:50.867602 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-c3c028da-0c58-4bfe-879f-a0b269f79976 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 11:03:50.937580 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1218/2464] 10.222.0.25 () {58 vars in 958 bytes} [Sun Aug 7 11:03:50 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 07 11:03:50.942297 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d676763d-e496-471b-a613-169dbd7bfd4c admin admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 11:03:51.004409 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d676763d-e496-471b-a613-169dbd7bfd4c admin admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 11:03:51.009184 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-885750a3-ae28-498e-a162-c45547a652f6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5ac47e3b09f43d19a23a0f457fe9e23', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 11:03:51.009787 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-885750a3-ae28-498e-a162-c45547a652f6 admin admin] REQUEST_METHOD: `POST` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 11:03:51.010066 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-885750a3-ae28-498e-a162-c45547a652f6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 11:03:51.010366 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-885750a3-ae28-498e-a162-c45547a652f6 admin admin] PATH_INFO: `/v3/projects` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 11:03:51.011043 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-885750a3-ae28-498e-a162-c45547a652f6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 11:03:51.013029 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-885750a3-ae28-498e-a162-c45547a652f6 admin admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 11:03:51.041183 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 1247/2465] 10.222.0.25 () {60 vars in 1153 bytes} [Sun Aug 7 11:03:50 2022] POST /identity/v3/projects => generated 381 bytes in 99 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 07 11:03:51.046366 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3c028da-0c58-4bfe-879f-a0b269f79976 None None] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 11:03:51.057005 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3c028da-0c58-4bfe-879f-a0b269f79976 None None] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 11:03:51.100838 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 11:03:51.119929 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f3fc1de-70ae-4179-b8ba-dd773d86379b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5ac47e3b09f43d19a23a0f457fe9e23', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 11:03:51.120467 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f3fc1de-70ae-4179-b8ba-dd773d86379b admin admin] REQUEST_METHOD: `POST` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 11:03:51.120752 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f3fc1de-70ae-4179-b8ba-dd773d86379b admin admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 11:03:51.121023 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f3fc1de-70ae-4179-b8ba-dd773d86379b admin admin] PATH_INFO: `/v3/users` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 11:03:51.122666 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f3fc1de-70ae-4179-b8ba-dd773d86379b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 11:03:51.124642 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f3fc1de-70ae-4179-b8ba-dd773d86379b admin admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 11:03:51.145830 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1219/2466] 10.222.0.25 () {60 vars in 1144 bytes} [Sun Aug 7 11:03:51 2022] POST /identity/v3/users => generated 297 bytes in 100 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 07 11:03:51.149898 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-885750a3-ae28-498e-a162-c45547a652f6 admin admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 11:03:51.160665 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-885750a3-ae28-498e-a162-c45547a652f6 admin admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 11:03:51.165534 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ce58f9f-cf62-4b9d-bcd6-18fe2593983a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5ac47e3b09f43d19a23a0f457fe9e23', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 11:03:51.166142 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ce58f9f-cf62-4b9d-bcd6-18fe2593983a admin admin] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 11:03:51.166423 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ce58f9f-cf62-4b9d-bcd6-18fe2593983a admin admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 11:03:51.166697 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ce58f9f-cf62-4b9d-bcd6-18fe2593983a admin admin] PATH_INFO: `/v3/roles` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 11:03:51.167387 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ce58f9f-cf62-4b9d-bcd6-18fe2593983a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 11:03:51.169810 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ce58f9f-cf62-4b9d-bcd6-18fe2593983a admin admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 11:03:51.180510 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 1248/2467] 10.222.0.25 () {58 vars in 1122 bytes} [Sun Aug 7 11:03:51 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 07 11:03:51.185405 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f3fc1de-70ae-4179-b8ba-dd773d86379b admin admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 11:03:51.195367 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f3fc1de-70ae-4179-b8ba-dd773d86379b admin admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 11:03:51.200208 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaa2654f-49e4-46a5-beb7-bf3554234eaa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5ac47e3b09f43d19a23a0f457fe9e23', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 11:03:51.200854 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaa2654f-49e4-46a5-beb7-bf3554234eaa admin admin] REQUEST_METHOD: `PUT` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 11:03:51.201137 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaa2654f-49e4-46a5-beb7-bf3554234eaa admin admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 11:03:51.201428 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaa2654f-49e4-46a5-beb7-bf3554234eaa admin admin] PATH_INFO: `/v3/projects/eb7a63234e524e82ac320e34db0292d1/users/5fefe2a999654c70a75bdf67dfd52a31/roles/81b1f1bcee2a473088fbbd327c1c5b23` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 11:03:51.219259 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaa2654f-49e4-46a5-beb7-bf3554234eaa admin admin] RBAC: Authorizing `identity:create_grant(project_id=eb7a63234e524e82ac320e34db0292d1, user_id=5fefe2a999654c70a75bdf67dfd52a31, role_id=81b1f1bcee2a473088fbbd327c1c5b23)` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 11:03:51.221949 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaa2654f-49e4-46a5-beb7-bf3554234eaa admin admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 11:03:51.236103 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1220/2468] 10.222.0.25 () {60 vars in 1483 bytes} [Sun Aug 7 11:03:51 2022] PUT /identity/v3/projects/eb7a63234e524e82ac320e34db0292d1/users/5fefe2a999654c70a75bdf67dfd52a31/roles/81b1f1bcee2a473088fbbd327c1c5b23 => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 07 11:03:51.240203 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ce58f9f-cf62-4b9d-bcd6-18fe2593983a admin admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 11:03:51.250695 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ce58f9f-cf62-4b9d-bcd6-18fe2593983a admin admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 11:03:51.293006 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:DELETE=4,keystone:SELECT=92,keystone:INSERT=4 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 11:03:51.309701 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f21f8edc-d3fc-4e96-a69e-6cdf6b44dbab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5ac47e3b09f43d19a23a0f457fe9e23', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 11:03:51.310304 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f21f8edc-d3fc-4e96-a69e-6cdf6b44dbab admin admin] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 11:03:51.310583 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f21f8edc-d3fc-4e96-a69e-6cdf6b44dbab admin admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 11:03:51.310853 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f21f8edc-d3fc-4e96-a69e-6cdf6b44dbab admin admin] PATH_INFO: `/v3/roles` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 11:03:51.311516 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f21f8edc-d3fc-4e96-a69e-6cdf6b44dbab admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 11:03:51.313080 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f21f8edc-d3fc-4e96-a69e-6cdf6b44dbab admin admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 11:03:51.323133 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 1249/2469] 10.222.0.25 () {58 vars in 1122 bytes} [Sun Aug 7 11:03:51 2022] GET /identity/v3/roles => generated 1420 bytes in 83 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 07 11:03:51.327558 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaa2654f-49e4-46a5-beb7-bf3554234eaa admin admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 11:03:51.337737 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaa2654f-49e4-46a5-beb7-bf3554234eaa admin admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 11:03:51.342942 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fddf534d-d84b-4ec1-9638-f74f955d5419 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5ac47e3b09f43d19a23a0f457fe9e23', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 11:03:51.343603 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fddf534d-d84b-4ec1-9638-f74f955d5419 admin admin] REQUEST_METHOD: `PUT` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 11:03:51.343881 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fddf534d-d84b-4ec1-9638-f74f955d5419 admin admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 11:03:51.344154 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fddf534d-d84b-4ec1-9638-f74f955d5419 admin admin] PATH_INFO: `/v3/projects/eb7a63234e524e82ac320e34db0292d1/users/5fefe2a999654c70a75bdf67dfd52a31/roles/81b1f1bcee2a473088fbbd327c1c5b23` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 11:03:51.346043 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fddf534d-d84b-4ec1-9638-f74f955d5419 admin admin] RBAC: Authorizing `identity:create_grant(project_id=eb7a63234e524e82ac320e34db0292d1, user_id=5fefe2a999654c70a75bdf67dfd52a31, role_id=81b1f1bcee2a473088fbbd327c1c5b23)` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 11:03:51.348689 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fddf534d-d84b-4ec1-9638-f74f955d5419 admin admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 11:03:51.359539 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1221/2470] 10.222.0.25 () {60 vars in 1483 bytes} [Sun Aug 7 11:03:51 2022] PUT /identity/v3/projects/eb7a63234e524e82ac320e34db0292d1/users/5fefe2a999654c70a75bdf67dfd52a31/roles/81b1f1bcee2a473088fbbd327c1c5b23 => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 07 11:03:51.369439 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5855bdcc-e1c6-4c75-96f5-e645660142e2 None None] REQUEST_METHOD: `POST` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 11:03:51.369816 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5855bdcc-e1c6-4c75-96f5-e645660142e2 None None] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 11:03:51.370153 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5855bdcc-e1c6-4c75-96f5-e645660142e2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 11:03:51.398159 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.auth.core [None req-5855bdcc-e1c6-4c75-96f5-e645660142e2 None None] MFA Rules not processed for user `5fefe2a999654c70a75bdf67dfd52a31`. Rule list: `[]` (Enabled: `True`). {{(pid=80144) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 11:03:51.399415 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.fernet_utils [None req-5855bdcc-e1c6-4c75-96f5-e645660142e2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80144) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 11:03:51.468370 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 1250/2471] 10.222.0.25 () {58 vars in 958 bytes} [Sun Aug 7 11:03:51 2022] POST /identity/v3/auth/tokens => generated 3325 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 07 11:03:51.479717 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fddf534d-d84b-4ec1-9638-f74f955d5419 admin admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 11:03:51.482522 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-fddf534d-d84b-4ec1-9638-f74f955d5419 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 11:03:51.540378 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fddf534d-d84b-4ec1-9638-f74f955d5419 admin admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 11:03:51.544955 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23f03f28-93d1-4318-b83b-b850346ba275 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88dbd6dfd1164994a3e16da553561910', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6de1f3cac02c4bcdb05dd2337176033d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 11:03:51.545552 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23f03f28-93d1-4318-b83b-b850346ba275 service cinder] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 11:03:51.545830 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23f03f28-93d1-4318-b83b-b850346ba275 service cinder] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 11:03:51.546100 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23f03f28-93d1-4318-b83b-b850346ba275 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 11:03:51.555583 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23f03f28-93d1-4318-b83b-b850346ba275 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 11:03:51.557048 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23f03f28-93d1-4318-b83b-b850346ba275 service cinder] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 11:03:51.566985 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1222/2472] 10.222.0.25 () {60 vars in 1365 bytes} [Sun Aug 7 11:03:51 2022] GET /identity/v3/auth/tokens => generated 3325 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 07 11:03:51.641517 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5855bdcc-e1c6-4c75-96f5-e645660142e2 None None] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 11:03:51.651206 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5855bdcc-e1c6-4c75-96f5-e645660142e2 None None] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 11:03:51.655484 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb06968f-f43e-4b22-b489-3b4109ad8804 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88dbd6dfd1164994a3e16da553561910', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6de1f3cac02c4bcdb05dd2337176033d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 11:03:51.656094 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb06968f-f43e-4b22-b489-3b4109ad8804 service cinder] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 11:03:51.656378 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb06968f-f43e-4b22-b489-3b4109ad8804 service cinder] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 11:03:51.656655 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb06968f-f43e-4b22-b489-3b4109ad8804 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 11:03:51.716331 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb06968f-f43e-4b22-b489-3b4109ad8804 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 11:03:51.717765 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb06968f-f43e-4b22-b489-3b4109ad8804 service cinder] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 11:03:51.726674 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 1251/2473] 10.222.0.25 () {60 vars in 1365 bytes} [Sun Aug 7 11:03:51 2022] GET /identity/v3/auth/tokens => generated 3258 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 07 11:03:52.015803 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23f03f28-93d1-4318-b83b-b850346ba275 service cinder] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 11:03:52.093713 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23f03f28-93d1-4318-b83b-b850346ba275 service cinder] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 11:03:52.098485 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8adcf18a-38b3-4a20-84e0-23d0725fd110 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5ac47e3b09f43d19a23a0f457fe9e23', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 11:03:52.099146 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8adcf18a-38b3-4a20-84e0-23d0725fd110 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 11:03:52.099426 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8adcf18a-38b3-4a20-84e0-23d0725fd110 admin admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 11:03:52.099742 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8adcf18a-38b3-4a20-84e0-23d0725fd110 admin admin] PATH_INFO: `/v3/users/c41e1ce5aeda49c4ade51ccafa4eb3e2` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 11:03:52.100813 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8adcf18a-38b3-4a20-84e0-23d0725fd110 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c41e1ce5aeda49c4ade51ccafa4eb3e2)` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 11:03:52.102711 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8adcf18a-38b3-4a20-84e0-23d0725fd110 admin admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 11:03:52.146752 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=90,keystone:INSERT=5,keystone:DELETE=5 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 11:03:52.153399 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-8adcf18a-38b3-4a20-84e0-23d0725fd110 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c41e1ce5aeda49c4ade51ccafa4eb3e2'} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 11:03:52.163058 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-8adcf18a-38b3-4a20-84e0-23d0725fd110 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c41e1ce5aeda49c4ade51ccafa4eb3e2'} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 11:03:52.176158 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-8adcf18a-38b3-4a20-84e0-23d0725fd110 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c41e1ce5aeda49c4ade51ccafa4eb3e2'} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 11:03:52.176955 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-8adcf18a-38b3-4a20-84e0-23d0725fd110 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c41e1ce5aeda49c4ade51ccafa4eb3e2'} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 11:03:52.190774 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-8adcf18a-38b3-4a20-84e0-23d0725fd110 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c41e1ce5aeda49c4ade51ccafa4eb3e2'} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 11:03:52.196992 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1223/2474] 10.222.0.25 () {58 vars in 1224 bytes} [Sun Aug 7 11:03:52 2022] DELETE /identity/v3/users/c41e1ce5aeda49c4ade51ccafa4eb3e2 => generated 0 bytes in 182 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 07 11:03:52.202053 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dc267f8-7cbc-496e-90e3-4d40191f0a40 None None] REQUEST_METHOD: `POST` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 11:03:52.202498 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dc267f8-7cbc-496e-90e3-4d40191f0a40 None None] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 11:03:52.202847 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dc267f8-7cbc-496e-90e3-4d40191f0a40 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 11:03:52.223424 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.auth.core [None req-0dc267f8-7cbc-496e-90e3-4d40191f0a40 None None] MFA Rules not processed for user `bac4e6b76fe64217ac8fa44e71cffe6a`. Rule list: `[]` (Enabled: `True`). {{(pid=80144) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 11:03:52.224807 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.fernet_utils [None req-0dc267f8-7cbc-496e-90e3-4d40191f0a40 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80144) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 11:03:52.286106 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 1252/2475] 10.222.0.25 () {58 vars in 958 bytes} [Sun Aug 7 11:03:52 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 07 11:03:52.296223 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8adcf18a-38b3-4a20-84e0-23d0725fd110 admin admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 11:03:52.297647 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-8adcf18a-38b3-4a20-84e0-23d0725fd110 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 11:03:52.357736 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8adcf18a-38b3-4a20-84e0-23d0725fd110 admin admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 11:03:52.361619 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7b219ce-9ecb-44e9-9ed2-0af89fed4265 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be1283c9bf47424c9aa3ad27cab03027', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6de1f3cac02c4bcdb05dd2337176033d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 11:03:52.362221 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7b219ce-9ecb-44e9-9ed2-0af89fed4265 service neutron] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 11:03:52.362501 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7b219ce-9ecb-44e9-9ed2-0af89fed4265 service neutron] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 11:03:52.362773 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7b219ce-9ecb-44e9-9ed2-0af89fed4265 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 11:03:52.372153 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7b219ce-9ecb-44e9-9ed2-0af89fed4265 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 11:03:52.373566 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7b219ce-9ecb-44e9-9ed2-0af89fed4265 service neutron] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 11:03:52.382960 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1224/2476] 10.222.0.25 () {60 vars in 1365 bytes} [Sun Aug 7 11:03:52 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 07 11:03:52.667955 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9879bc68-2cba-43a0-ade2-55c8f9b796ae None None] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 11:03:52.669063 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9879bc68-2cba-43a0-ade2-55c8f9b796ae None None] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 11:03:52.669366 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9879bc68-2cba-43a0-ade2-55c8f9b796ae None None] PATH_INFO: `/` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 11:03:52.670708 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 1253/2477] 10.222.0.25 () {54 vars in 921 bytes} [Sun Aug 7 11:03:52 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 07 11:03:52.675925 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50526e87-f2ab-400e-80ca-388012643c4a None None] REQUEST_METHOD: `POST` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 11:03:52.676474 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50526e87-f2ab-400e-80ca-388012643c4a None None] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 11:03:52.676474 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50526e87-f2ab-400e-80ca-388012643c4a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 11:03:52.688854 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9879bc68-2cba-43a0-ade2-55c8f9b796ae None None] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 11:03:52.691788 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.fernet_utils [None req-9879bc68-2cba-43a0-ade2-55c8f9b796ae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80144) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 11:03:52.707466 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=100 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 11:03:52.713950 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9879bc68-2cba-43a0-ade2-55c8f9b796ae None None] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 11:03:52.721303 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33d77b3c-f0e6-41d9-af6d-630464ffb27d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5ac47e3b09f43d19a23a0f457fe9e23', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 11:03:52.722294 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33d77b3c-f0e6-41d9-af6d-630464ffb27d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 11:03:52.722648 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33d77b3c-f0e6-41d9-af6d-630464ffb27d admin admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 11:03:52.723048 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33d77b3c-f0e6-41d9-af6d-630464ffb27d admin admin] PATH_INFO: `/v3/projects/7a9a8f92281f4380b5ff21ece0aaaff3` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 11:03:52.724413 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33d77b3c-f0e6-41d9-af6d-630464ffb27d admin admin] RBAC: Authorizing `identity:delete_project(project_id=7a9a8f92281f4380b5ff21ece0aaaff3)` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 11:03:52.726599 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33d77b3c-f0e6-41d9-af6d-630464ffb27d admin admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 11:03:52.729881 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.auth.core [None req-50526e87-f2ab-400e-80ca-388012643c4a None None] MFA Rules not processed for user `31d16d1b96104b4eb690398432b9829e`. Rule list: `[]` (Enabled: `True`). {{(pid=80143) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 11:03:52.732375 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-50526e87-f2ab-400e-80ca-388012643c4a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 11:03:52.787360 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.notifications [None req-33d77b3c-f0e6-41d9-af6d-630464ffb27d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7a9a8f92281f4380b5ff21ece0aaaff3', 'request_id': 'req-33d77b3c-f0e6-41d9-af6d-630464ffb27d'} {{(pid=80144) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 11:03:52.795269 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.notifications [None req-33d77b3c-f0e6-41d9-af6d-630464ffb27d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7a9a8f92281f4380b5ff21ece0aaaff3', 'request_id': 'req-33d77b3c-f0e6-41d9-af6d-630464ffb27d'} {{(pid=80144) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 11:03:52.805439 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.notifications [None req-33d77b3c-f0e6-41d9-af6d-630464ffb27d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7a9a8f92281f4380b5ff21ece0aaaff3', 'request_id': 'req-33d77b3c-f0e6-41d9-af6d-630464ffb27d'} {{(pid=80144) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 11:03:52.817250 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.notifications [None req-33d77b3c-f0e6-41d9-af6d-630464ffb27d admin admin] The token cache is being invalidate because project 7a9a8f92281f4380b5ff21ece0aaaff3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80144) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 07 11:03:52.817543 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.notifications [None req-33d77b3c-f0e6-41d9-af6d-630464ffb27d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80144) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 11:03:52.818581 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.notifications [None req-33d77b3c-f0e6-41d9-af6d-630464ffb27d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80144) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 11:03:52.823579 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 1254/2478] 10.222.0.25 () {58 vars in 1233 bytes} [Sun Aug 7 11:03:52 2022] DELETE /identity/v3/projects/7a9a8f92281f4380b5ff21ece0aaaff3 => generated 0 bytes in 133 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 07 11:03:52.824695 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33d77b3c-f0e6-41d9-af6d-630464ffb27d admin admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 11:03:52.826281 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.fernet_utils [None req-33d77b3c-f0e6-41d9-af6d-630464ffb27d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80144) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 11:03:52.828149 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1225/2479] 10.222.0.25 () {58 vars in 1018 bytes} [Sun Aug 7 11:03:52 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 153 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 07 11:03:52.839757 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50526e87-f2ab-400e-80ca-388012643c4a None None] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 11:03:52.842310 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-50526e87-f2ab-400e-80ca-388012643c4a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 11:03:52.877297 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:DELETE=5,keystone:SELECT=94,keystone:INSERT=1 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 11:03:52.900209 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33d77b3c-f0e6-41d9-af6d-630464ffb27d admin admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 11:03:52.906779 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50526e87-f2ab-400e-80ca-388012643c4a None None] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 11:03:52.907160 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a87a53eb-08ed-418e-993a-3fdfc9ac5eb7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5ac47e3b09f43d19a23a0f457fe9e23', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 11:03:52.908073 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a87a53eb-08ed-418e-993a-3fdfc9ac5eb7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 11:03:52.908411 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a87a53eb-08ed-418e-993a-3fdfc9ac5eb7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 11:03:52.908827 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a87a53eb-08ed-418e-993a-3fdfc9ac5eb7 admin admin] PATH_INFO: `/v3/users/5fefe2a999654c70a75bdf67dfd52a31` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 11:03:52.910270 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a87a53eb-08ed-418e-993a-3fdfc9ac5eb7 admin admin] RBAC: Authorizing `identity:delete_user(user_id=5fefe2a999654c70a75bdf67dfd52a31)` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 11:03:52.912719 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a87a53eb-08ed-418e-993a-3fdfc9ac5eb7 admin admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 11:03:52.961284 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc641e10-973e-485a-b87e-f01b95f25521 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a11186d9d0d6488b953ede2f1628148d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6de1f3cac02c4bcdb05dd2337176033d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 11:03:52.961898 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc641e10-973e-485a-b87e-f01b95f25521 service glance] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 11:03:52.962177 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc641e10-973e-485a-b87e-f01b95f25521 service glance] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 11:03:52.962446 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc641e10-973e-485a-b87e-f01b95f25521 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 11:03:52.962828 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.notifications [None req-a87a53eb-08ed-418e-993a-3fdfc9ac5eb7 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5fefe2a999654c70a75bdf67dfd52a31'} {{(pid=80144) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 11:03:52.964239 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-cc641e10-973e-485a-b87e-f01b95f25521 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 11:03:52.970989 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.notifications [None req-a87a53eb-08ed-418e-993a-3fdfc9ac5eb7 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5fefe2a999654c70a75bdf67dfd52a31'} {{(pid=80144) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 11:03:52.978828 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.notifications [None req-a87a53eb-08ed-418e-993a-3fdfc9ac5eb7 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5fefe2a999654c70a75bdf67dfd52a31'} {{(pid=80144) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 11:03:52.979476 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.notifications [None req-a87a53eb-08ed-418e-993a-3fdfc9ac5eb7 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5fefe2a999654c70a75bdf67dfd52a31'} {{(pid=80144) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 11:03:52.980107 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.notifications [None req-a87a53eb-08ed-418e-993a-3fdfc9ac5eb7 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5fefe2a999654c70a75bdf67dfd52a31'} {{(pid=80144) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 11:03:53.000776 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 1255/2480] 10.222.0.25 () {58 vars in 1224 bytes} [Sun Aug 7 11:03:52 2022] DELETE /identity/v3/users/5fefe2a999654c70a75bdf67dfd52a31 => generated 0 bytes in 177 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 07 11:03:53.038068 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc641e10-973e-485a-b87e-f01b95f25521 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 11:03:53.039863 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc641e10-973e-485a-b87e-f01b95f25521 service glance] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 11:03:53.052496 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1226/2481] 10.222.0.25 () {60 vars in 1365 bytes} [Sun Aug 7 11:03:52 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 213 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 07 11:03:53.320180 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a87a53eb-08ed-418e-993a-3fdfc9ac5eb7 admin admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 11:03:53.321685 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.fernet_utils [None req-a87a53eb-08ed-418e-993a-3fdfc9ac5eb7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80144) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 11:03:53.346546 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 11:03:53.415792 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a87a53eb-08ed-418e-993a-3fdfc9ac5eb7 admin admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 11:03:53.423048 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-293731df-0b70-48e9-95dd-6eede414a265 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5ac47e3b09f43d19a23a0f457fe9e23', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 11:03:53.424135 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-293731df-0b70-48e9-95dd-6eede414a265 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 11:03:53.424135 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-293731df-0b70-48e9-95dd-6eede414a265 admin admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 11:03:53.424326 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-293731df-0b70-48e9-95dd-6eede414a265 admin admin] PATH_INFO: `/v3/projects/eb7a63234e524e82ac320e34db0292d1` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 11:03:53.425365 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-293731df-0b70-48e9-95dd-6eede414a265 admin admin] RBAC: Authorizing `identity:delete_project(project_id=eb7a63234e524e82ac320e34db0292d1)` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 11:03:53.427414 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-293731df-0b70-48e9-95dd-6eede414a265 admin admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 11:03:53.480262 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.notifications [None req-293731df-0b70-48e9-95dd-6eede414a265 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'eb7a63234e524e82ac320e34db0292d1', 'request_id': 'req-293731df-0b70-48e9-95dd-6eede414a265'} {{(pid=80144) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 11:03:53.487927 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.notifications [None req-293731df-0b70-48e9-95dd-6eede414a265 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'eb7a63234e524e82ac320e34db0292d1', 'request_id': 'req-293731df-0b70-48e9-95dd-6eede414a265'} {{(pid=80144) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 11:03:53.495759 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.notifications [None req-293731df-0b70-48e9-95dd-6eede414a265 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'eb7a63234e524e82ac320e34db0292d1', 'request_id': 'req-293731df-0b70-48e9-95dd-6eede414a265'} {{(pid=80144) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 11:03:53.507978 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.notifications [None req-293731df-0b70-48e9-95dd-6eede414a265 admin admin] The token cache is being invalidate because project eb7a63234e524e82ac320e34db0292d1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80144) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 07 11:03:53.508274 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.notifications [None req-293731df-0b70-48e9-95dd-6eede414a265 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80144) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 11:03:53.508936 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.notifications [None req-293731df-0b70-48e9-95dd-6eede414a265 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80144) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 11:03:53.510816 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 1256/2482] 10.222.0.25 () {58 vars in 1233 bytes} [Sun Aug 7 11:03:53 2022] DELETE /identity/v3/projects/eb7a63234e524e82ac320e34db0292d1 => generated 0 bytes in 191 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 07 11:03:53.515611 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-866b97c8-5932-40f1-9c8c-d7d2a7c84add None None] REQUEST_METHOD: `POST` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 11:03:53.515901 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-866b97c8-5932-40f1-9c8c-d7d2a7c84add None None] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 11:03:53.516184 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-866b97c8-5932-40f1-9c8c-d7d2a7c84add None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 11:03:53.538258 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.auth.core [None req-866b97c8-5932-40f1-9c8c-d7d2a7c84add None None] MFA Rules not processed for user `bac4e6b76fe64217ac8fa44e71cffe6a`. Rule list: `[]` (Enabled: `True`). {{(pid=80143) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 11:03:53.539997 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-866b97c8-5932-40f1-9c8c-d7d2a7c84add None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 11:03:53.603898 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1227/2483] 10.222.0.25 () {58 vars in 958 bytes} [Sun Aug 7 11:03:53 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 07 11:03:53.613856 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90615485-5ee1-41d4-a06d-dce2970ee2b7 None None] REQUEST_METHOD: `POST` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 11:03:53.614143 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90615485-5ee1-41d4-a06d-dce2970ee2b7 None None] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 11:03:53.614422 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90615485-5ee1-41d4-a06d-dce2970ee2b7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 11:03:53.634702 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.auth.core [None req-90615485-5ee1-41d4-a06d-dce2970ee2b7 None None] MFA Rules not processed for user `bac4e6b76fe64217ac8fa44e71cffe6a`. Rule list: `[]` (Enabled: `True`). {{(pid=80144) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 11:03:53.636010 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.fernet_utils [None req-90615485-5ee1-41d4-a06d-dce2970ee2b7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80144) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 11:03:53.645101 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 1257/2484] 10.222.0.25 () {58 vars in 958 bytes} [Sun Aug 7 11:03:53 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 07 11:03:53.650093 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-866b97c8-5932-40f1-9c8c-d7d2a7c84add None None] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 11:03:53.660087 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-866b97c8-5932-40f1-9c8c-d7d2a7c84add None None] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 11:03:53.666654 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e25101f2-d84a-4af8-8d15-bec8d6e6d7ea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5ac47e3b09f43d19a23a0f457fe9e23', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 11:03:53.667299 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e25101f2-d84a-4af8-8d15-bec8d6e6d7ea admin admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 11:03:53.667581 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e25101f2-d84a-4af8-8d15-bec8d6e6d7ea admin admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 11:03:53.667857 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e25101f2-d84a-4af8-8d15-bec8d6e6d7ea admin admin] PATH_INFO: `/v3/domains` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 11:03:53.668546 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e25101f2-d84a-4af8-8d15-bec8d6e6d7ea admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80143) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 11:03:53.668852 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e25101f2-d84a-4af8-8d15-bec8d6e6d7ea admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 11:03:53.670509 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e25101f2-d84a-4af8-8d15-bec8d6e6d7ea admin admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 11:03:53.685554 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1228/2485] 10.222.0.25 () {58 vars in 1153 bytes} [Sun Aug 7 11:03:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 11:03:53.687635 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90615485-5ee1-41d4-a06d-dce2970ee2b7 None None] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 11:03:53.697017 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90615485-5ee1-41d4-a06d-dce2970ee2b7 None None] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 11:03:53.703059 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d89e159-2ae2-4770-a7de-af872baa5f00 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5ac47e3b09f43d19a23a0f457fe9e23', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 11:03:53.703689 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d89e159-2ae2-4770-a7de-af872baa5f00 admin admin] REQUEST_METHOD: `POST` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 11:03:53.703973 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d89e159-2ae2-4770-a7de-af872baa5f00 admin admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 11:03:53.704247 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d89e159-2ae2-4770-a7de-af872baa5f00 admin admin] PATH_INFO: `/v3/projects` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 11:03:53.704911 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d89e159-2ae2-4770-a7de-af872baa5f00 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 11:03:53.706811 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d89e159-2ae2-4770-a7de-af872baa5f00 admin admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 11:03:53.740351 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 1258/2486] 10.222.0.25 () {60 vars in 1153 bytes} [Sun Aug 7 11:03:53 2022] POST /identity/v3/projects => generated 371 bytes in 50 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 07 11:03:53.741459 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e25101f2-d84a-4af8-8d15-bec8d6e6d7ea admin admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 11:03:53.752319 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e25101f2-d84a-4af8-8d15-bec8d6e6d7ea admin admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 11:03:53.765541 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=100 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 11:03:53.818324 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0abcb105-2bdb-4afd-a578-203ba080cd4b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5ac47e3b09f43d19a23a0f457fe9e23', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 11:03:53.818968 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0abcb105-2bdb-4afd-a578-203ba080cd4b admin admin] REQUEST_METHOD: `POST` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 11:03:53.819284 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0abcb105-2bdb-4afd-a578-203ba080cd4b admin admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 11:03:53.819545 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0abcb105-2bdb-4afd-a578-203ba080cd4b admin admin] PATH_INFO: `/v3/users` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 11:03:53.820202 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0abcb105-2bdb-4afd-a578-203ba080cd4b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 11:03:53.822131 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0abcb105-2bdb-4afd-a578-203ba080cd4b admin admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 11:03:53.844884 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1229/2487] 10.222.0.25 () {60 vars in 1144 bytes} [Sun Aug 7 11:03:53 2022] POST /identity/v3/users => generated 286 bytes in 104 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 07 11:03:53.849053 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d89e159-2ae2-4770-a7de-af872baa5f00 admin admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 11:03:53.864665 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d89e159-2ae2-4770-a7de-af872baa5f00 admin admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 11:03:53.869822 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96301000-d9f1-4a8a-9c55-b74d7338cc29 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5ac47e3b09f43d19a23a0f457fe9e23', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 11:03:53.870468 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96301000-d9f1-4a8a-9c55-b74d7338cc29 admin admin] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 11:03:53.870747 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96301000-d9f1-4a8a-9c55-b74d7338cc29 admin admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 11:03:53.871050 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96301000-d9f1-4a8a-9c55-b74d7338cc29 admin admin] PATH_INFO: `/v3/roles` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 11:03:53.872277 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96301000-d9f1-4a8a-9c55-b74d7338cc29 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 11:03:53.873844 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96301000-d9f1-4a8a-9c55-b74d7338cc29 admin admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 11:03:53.884893 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 1259/2488] 10.222.0.25 () {58 vars in 1122 bytes} [Sun Aug 7 11:03:53 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 07 11:03:53.890376 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0abcb105-2bdb-4afd-a578-203ba080cd4b admin admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 11:03:53.901537 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0abcb105-2bdb-4afd-a578-203ba080cd4b admin admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 11:03:53.906487 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9249444c-9608-49d7-ab8c-f8d75a2d0ded admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5ac47e3b09f43d19a23a0f457fe9e23', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 11:03:53.907132 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9249444c-9608-49d7-ab8c-f8d75a2d0ded admin admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 11:03:53.907411 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9249444c-9608-49d7-ab8c-f8d75a2d0ded admin admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 11:03:53.907683 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9249444c-9608-49d7-ab8c-f8d75a2d0ded admin admin] PATH_INFO: `/v3/roles` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 11:03:53.908324 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9249444c-9608-49d7-ab8c-f8d75a2d0ded admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 11:03:53.909897 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9249444c-9608-49d7-ab8c-f8d75a2d0ded admin admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 11:03:53.923703 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1230/2489] 10.222.0.25 () {58 vars in 1122 bytes} [Sun Aug 7 11:03:53 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 07 11:03:53.927859 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96301000-d9f1-4a8a-9c55-b74d7338cc29 admin admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 11:03:53.938835 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96301000-d9f1-4a8a-9c55-b74d7338cc29 admin admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 11:03:53.946413 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaadf06e-fa26-494b-8adf-ac59913de66c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5ac47e3b09f43d19a23a0f457fe9e23', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 11:03:53.948383 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaadf06e-fa26-494b-8adf-ac59913de66c admin admin] REQUEST_METHOD: `PUT` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 11:03:53.948567 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaadf06e-fa26-494b-8adf-ac59913de66c admin admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 11:03:53.948850 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaadf06e-fa26-494b-8adf-ac59913de66c admin admin] PATH_INFO: `/v3/projects/d218b2dc5f6a404781afd3bbf74acb40/users/423d7a5d5ada4480a2ad121d8f49665a/roles/b98dfb3b9fee4644af2887a3ac3fd2e6` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 11:03:53.968659 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaadf06e-fa26-494b-8adf-ac59913de66c admin admin] RBAC: Authorizing `identity:create_grant(project_id=d218b2dc5f6a404781afd3bbf74acb40, user_id=423d7a5d5ada4480a2ad121d8f49665a, role_id=b98dfb3b9fee4644af2887a3ac3fd2e6)` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 11:03:53.971398 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaadf06e-fa26-494b-8adf-ac59913de66c admin admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 11:03:53.987195 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 1260/2490] 10.222.0.25 () {60 vars in 1483 bytes} [Sun Aug 7 11:03:53 2022] PUT /identity/v3/projects/d218b2dc5f6a404781afd3bbf74acb40/users/423d7a5d5ada4480a2ad121d8f49665a/roles/b98dfb3b9fee4644af2887a3ac3fd2e6 => generated 0 bytes in 60 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 07 11:03:53.998315 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb719fca-472e-4acd-b839-e2a1107c57a9 None None] REQUEST_METHOD: `POST` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 11:03:53.998529 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb719fca-472e-4acd-b839-e2a1107c57a9 None None] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 11:03:53.998887 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb719fca-472e-4acd-b839-e2a1107c57a9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 11:03:54.028718 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.auth.core [None req-fb719fca-472e-4acd-b839-e2a1107c57a9 None None] MFA Rules not processed for user `423d7a5d5ada4480a2ad121d8f49665a`. Rule list: `[]` (Enabled: `True`). {{(pid=80143) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 11:03:54.029950 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-fb719fca-472e-4acd-b839-e2a1107c57a9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 11:03:54.095063 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1231/2491] 10.222.0.25 () {58 vars in 958 bytes} [Sun Aug 7 11:03:53 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 07 11:03:54.107395 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaadf06e-fa26-494b-8adf-ac59913de66c admin admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 11:03:54.109480 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.fernet_utils [None req-aaadf06e-fa26-494b-8adf-ac59913de66c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80144) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 11:03:54.146316 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 11:03:54.177006 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaadf06e-fa26-494b-8adf-ac59913de66c admin admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 11:03:54.181770 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e81dae2e-b485-4d89-84e5-e14be7496e28 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a11186d9d0d6488b953ede2f1628148d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6de1f3cac02c4bcdb05dd2337176033d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 11:03:54.182857 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e81dae2e-b485-4d89-84e5-e14be7496e28 service glance] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 11:03:54.183243 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e81dae2e-b485-4d89-84e5-e14be7496e28 service glance] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 11:03:54.185561 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e81dae2e-b485-4d89-84e5-e14be7496e28 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 11:03:54.196239 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e81dae2e-b485-4d89-84e5-e14be7496e28 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 11:03:54.197761 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e81dae2e-b485-4d89-84e5-e14be7496e28 service glance] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 11:03:54.211507 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 1261/2492] 10.222.0.25 () {60 vars in 1365 bytes} [Sun Aug 7 11:03:54 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 105 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 07 11:03:55.308715 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7877cb56-b1f3-4a71-ac65-1305fa16c74f None None] REQUEST_METHOD: `POST` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 11:03:55.309187 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7877cb56-b1f3-4a71-ac65-1305fa16c74f None None] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 11:03:55.309399 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7877cb56-b1f3-4a71-ac65-1305fa16c74f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 11:03:55.329028 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.auth.core [None req-7877cb56-b1f3-4a71-ac65-1305fa16c74f None None] MFA Rules not processed for user `de1d6c24b1a14d10a20d82a4c65793d8`. Rule list: `[]` (Enabled: `True`). {{(pid=80143) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 11:03:55.330323 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-7877cb56-b1f3-4a71-ac65-1305fa16c74f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 11:03:55.390713 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1232/2493] 10.222.0.25 () {58 vars in 1009 bytes} [Sun Aug 7 11:03:55 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 07 11:03:55.402456 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e81dae2e-b485-4d89-84e5-e14be7496e28 service glance] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 11:03:55.405066 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.fernet_utils [None req-e81dae2e-b485-4d89-84e5-e14be7496e28 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80144) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 11:03:55.474770 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e81dae2e-b485-4d89-84e5-e14be7496e28 service glance] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 11:03:55.479804 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00fdf162-add8-4374-9e6b-de46c8128eed service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69490ea2a9a340bf91aca38bf5655b45', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6de1f3cac02c4bcdb05dd2337176033d', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 11:03:55.480409 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00fdf162-add8-4374-9e6b-de46c8128eed service swift] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 11:03:55.481194 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00fdf162-add8-4374-9e6b-de46c8128eed service swift] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 11:03:55.481478 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00fdf162-add8-4374-9e6b-de46c8128eed service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 11:03:55.489674 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00fdf162-add8-4374-9e6b-de46c8128eed service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 11:03:55.491270 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00fdf162-add8-4374-9e6b-de46c8128eed service swift] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 11:03:55.499546 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 1262/2494] 10.222.0.25 () {60 vars in 1384 bytes} [Sun Aug 7 11:03:55 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 98 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Aug 07 11:04:05.383499 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=84,keystone:INSERT=3 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 11:04:05.496024 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=43 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 11:04:05.848066 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7877cb56-b1f3-4a71-ac65-1305fa16c74f None None] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 11:04:05.849755 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-7877cb56-b1f3-4a71-ac65-1305fa16c74f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 11:04:05.906830 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7877cb56-b1f3-4a71-ac65-1305fa16c74f None None] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 11:04:05.910400 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e40e479-85ae-457d-86cf-e1bcf20dc344 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be1283c9bf47424c9aa3ad27cab03027', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6de1f3cac02c4bcdb05dd2337176033d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 11:04:05.911044 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e40e479-85ae-457d-86cf-e1bcf20dc344 service neutron] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 11:04:05.911326 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e40e479-85ae-457d-86cf-e1bcf20dc344 service neutron] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 11:04:05.911601 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e40e479-85ae-457d-86cf-e1bcf20dc344 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 11:04:05.913292 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-1e40e479-85ae-457d-86cf-e1bcf20dc344 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 11:04:05.982625 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e40e479-85ae-457d-86cf-e1bcf20dc344 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 11:04:05.984272 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e40e479-85ae-457d-86cf-e1bcf20dc344 service neutron] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 11:04:05.995010 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1233/2495] 10.222.0.25 () {60 vars in 1365 bytes} [Sun Aug 7 11:04:05 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 147 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 07 11:04:15.990190 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=54 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 11:04:16.908817 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00fdf162-add8-4374-9e6b-de46c8128eed service swift] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 11:04:16.910452 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.fernet_utils [None req-00fdf162-add8-4374-9e6b-de46c8128eed service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80144) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 11:04:16.970577 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00fdf162-add8-4374-9e6b-de46c8128eed service swift] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 11:04:16.974230 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f502102d-177b-46d3-8cc2-a6f7c4a5e730 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88dbd6dfd1164994a3e16da553561910', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6de1f3cac02c4bcdb05dd2337176033d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 11:04:16.974823 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f502102d-177b-46d3-8cc2-a6f7c4a5e730 service cinder] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 11:04:16.975136 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f502102d-177b-46d3-8cc2-a6f7c4a5e730 service cinder] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 11:04:16.975407 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f502102d-177b-46d3-8cc2-a6f7c4a5e730 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 11:04:16.977136 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.fernet_utils [None req-f502102d-177b-46d3-8cc2-a6f7c4a5e730 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80144) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 11:04:16.991018 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f502102d-177b-46d3-8cc2-a6f7c4a5e730 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 11:04:16.992663 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f502102d-177b-46d3-8cc2-a6f7c4a5e730 service cinder] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 11:04:17.004283 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 1263/2496] 10.222.0.25 () {60 vars in 1365 bytes} [Sun Aug 7 11:04:16 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 07 11:04:20.285207 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c38ee4a-16c3-4b24-9738-0533b28a162a None None] REQUEST_METHOD: `POST` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 11:04:20.285653 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c38ee4a-16c3-4b24-9738-0533b28a162a None None] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 11:04:20.285741 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c38ee4a-16c3-4b24-9738-0533b28a162a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 11:04:20.305980 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.auth.core [None req-2c38ee4a-16c3-4b24-9738-0533b28a162a None None] MFA Rules not processed for user `de1d6c24b1a14d10a20d82a4c65793d8`. Rule list: `[]` (Enabled: `True`). {{(pid=80143) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 11:04:20.307213 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-2c38ee4a-16c3-4b24-9738-0533b28a162a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 11:04:20.321437 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1234/2497] 10.222.0.25 () {58 vars in 967 bytes} [Sun Aug 7 11:04:20 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 07 11:04:20.329782 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f502102d-177b-46d3-8cc2-a6f7c4a5e730 service cinder] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 11:04:20.342352 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f502102d-177b-46d3-8cc2-a6f7c4a5e730 service cinder] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 11:04:20.347572 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad3d0086-bbb8-40d8-bf92-4c254121e40d service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69490ea2a9a340bf91aca38bf5655b45', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6de1f3cac02c4bcdb05dd2337176033d', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 11:04:20.348386 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad3d0086-bbb8-40d8-bf92-4c254121e40d service swift] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 11:04:20.348486 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad3d0086-bbb8-40d8-bf92-4c254121e40d service swift] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 11:04:20.348711 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad3d0086-bbb8-40d8-bf92-4c254121e40d service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 11:04:20.359332 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad3d0086-bbb8-40d8-bf92-4c254121e40d service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 11:04:20.361036 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad3d0086-bbb8-40d8-bf92-4c254121e40d service swift] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 11:04:20.369998 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 1264/2498] 10.222.0.25 () {60 vars in 1384 bytes} [Sun Aug 7 11:04:20 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 07 11:04:21.799696 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c38ee4a-16c3-4b24-9738-0533b28a162a None None] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 11:04:21.801039 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-2c38ee4a-16c3-4b24-9738-0533b28a162a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 11:04:21.816915 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c38ee4a-16c3-4b24-9738-0533b28a162a None None] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 11:04:21.822332 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34e99e0e-ad9e-4ed0-9569-c3ebc5c6d479 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31d16d1b96104b4eb690398432b9829e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6de1f3cac02c4bcdb05dd2337176033d', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'reader', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 11:04:21.822947 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34e99e0e-ad9e-4ed0-9569-c3ebc5c6d479 service nova] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 11:04:21.823245 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34e99e0e-ad9e-4ed0-9569-c3ebc5c6d479 service nova] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 11:04:21.824249 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34e99e0e-ad9e-4ed0-9569-c3ebc5c6d479 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 11:04:21.825903 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-34e99e0e-ad9e-4ed0-9569-c3ebc5c6d479 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 11:04:21.904582 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34e99e0e-ad9e-4ed0-9569-c3ebc5c6d479 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 11:04:21.906154 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34e99e0e-ad9e-4ed0-9569-c3ebc5c6d479 service nova] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 11:04:21.924613 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1235/2499] 10.222.0.25 () {60 vars in 1365 bytes} [Sun Aug 7 11:04:21 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 125 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 07 11:04:30.366099 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=32 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 11:04:30.880071 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad3d0086-bbb8-40d8-bf92-4c254121e40d service swift] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 11:04:30.895558 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad3d0086-bbb8-40d8-bf92-4c254121e40d service swift] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 11:04:30.906135 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01462213-fea5-4696-91d7-14cab6b5bf65 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be1283c9bf47424c9aa3ad27cab03027', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6de1f3cac02c4bcdb05dd2337176033d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 11:04:30.906766 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01462213-fea5-4696-91d7-14cab6b5bf65 service neutron] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 11:04:30.907135 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01462213-fea5-4696-91d7-14cab6b5bf65 service neutron] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 11:04:30.907411 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01462213-fea5-4696-91d7-14cab6b5bf65 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 11:04:30.909291 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.fernet_utils [None req-01462213-fea5-4696-91d7-14cab6b5bf65 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80144) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 11:04:30.947018 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01462213-fea5-4696-91d7-14cab6b5bf65 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 11:04:30.948600 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01462213-fea5-4696-91d7-14cab6b5bf65 service neutron] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 11:04:30.966743 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 1265/2500] 10.222.0.25 () {60 vars in 1365 bytes} [Sun Aug 7 11:04:30 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 07 11:04:31.492522 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34e99e0e-ad9e-4ed0-9569-c3ebc5c6d479 service nova] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 11:04:31.578580 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34e99e0e-ad9e-4ed0-9569-c3ebc5c6d479 service nova] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 11:04:31.584061 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d201c62e-d425-4a8d-b01d-93283ecc522f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5ac47e3b09f43d19a23a0f457fe9e23', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 11:04:31.584743 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d201c62e-d425-4a8d-b01d-93283ecc522f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 11:04:31.585030 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d201c62e-d425-4a8d-b01d-93283ecc522f admin admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 11:04:31.585310 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d201c62e-d425-4a8d-b01d-93283ecc522f admin admin] PATH_INFO: `/v3/users/423d7a5d5ada4480a2ad121d8f49665a` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 11:04:31.586607 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d201c62e-d425-4a8d-b01d-93283ecc522f admin admin] RBAC: Authorizing `identity:delete_user(user_id=423d7a5d5ada4480a2ad121d8f49665a)` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 11:04:31.592580 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d201c62e-d425-4a8d-b01d-93283ecc522f admin admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 11:04:31.653942 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-d201c62e-d425-4a8d-b01d-93283ecc522f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '423d7a5d5ada4480a2ad121d8f49665a'} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 11:04:31.661241 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-d201c62e-d425-4a8d-b01d-93283ecc522f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '423d7a5d5ada4480a2ad121d8f49665a'} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 11:04:31.670297 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=92,keystone:DELETE=8 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 11:04:31.677026 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-d201c62e-d425-4a8d-b01d-93283ecc522f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '423d7a5d5ada4480a2ad121d8f49665a'} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 11:04:31.678608 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-d201c62e-d425-4a8d-b01d-93283ecc522f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '423d7a5d5ada4480a2ad121d8f49665a'} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 11:04:31.687847 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-d201c62e-d425-4a8d-b01d-93283ecc522f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '423d7a5d5ada4480a2ad121d8f49665a'} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 11:04:31.696976 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1236/2501] 10.222.0.25 () {58 vars in 1224 bytes} [Sun Aug 7 11:04:31 2022] DELETE /identity/v3/users/423d7a5d5ada4480a2ad121d8f49665a => generated 0 bytes in 205 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 07 11:04:31.702697 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49cf6e74-a4ec-4c5b-b333-dece55341acd None None] REQUEST_METHOD: `POST` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 11:04:31.703294 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49cf6e74-a4ec-4c5b-b333-dece55341acd None None] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 11:04:31.703294 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49cf6e74-a4ec-4c5b-b333-dece55341acd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 11:04:31.723738 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.auth.core [None req-49cf6e74-a4ec-4c5b-b333-dece55341acd None None] MFA Rules not processed for user `bac4e6b76fe64217ac8fa44e71cffe6a`. Rule list: `[]` (Enabled: `True`). {{(pid=80144) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 11:04:31.724961 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.fernet_utils [None req-49cf6e74-a4ec-4c5b-b333-dece55341acd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80144) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 11:04:31.792468 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 1266/2502] 10.222.0.25 () {58 vars in 958 bytes} [Sun Aug 7 11:04:31 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 07 11:04:31.803459 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d201c62e-d425-4a8d-b01d-93283ecc522f admin admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 11:04:31.805003 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-d201c62e-d425-4a8d-b01d-93283ecc522f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 11:04:31.871897 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d201c62e-d425-4a8d-b01d-93283ecc522f admin admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 11:04:31.875482 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c022197c-83cb-4a88-a0e9-9896e9942100 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be1283c9bf47424c9aa3ad27cab03027', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6de1f3cac02c4bcdb05dd2337176033d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 11:04:31.876090 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c022197c-83cb-4a88-a0e9-9896e9942100 service neutron] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 11:04:31.876367 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c022197c-83cb-4a88-a0e9-9896e9942100 service neutron] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 11:04:31.876636 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c022197c-83cb-4a88-a0e9-9896e9942100 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 11:04:31.885358 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c022197c-83cb-4a88-a0e9-9896e9942100 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 11:04:31.887488 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c022197c-83cb-4a88-a0e9-9896e9942100 service neutron] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 11:04:31.896644 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1237/2503] 10.222.0.25 () {60 vars in 1365 bytes} [Sun Aug 7 11:04:31 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 07 11:04:32.185957 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49cf6e74-a4ec-4c5b-b333-dece55341acd None None] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 11:04:32.187423 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.fernet_utils [None req-49cf6e74-a4ec-4c5b-b333-dece55341acd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80144) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 11:04:32.199685 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49cf6e74-a4ec-4c5b-b333-dece55341acd None None] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 11:04:32.204192 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd40d9f5-baf1-487a-a9ab-1ff4f3dd07eb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5ac47e3b09f43d19a23a0f457fe9e23', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 11:04:32.204957 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd40d9f5-baf1-487a-a9ab-1ff4f3dd07eb admin admin] REQUEST_METHOD: `DELETE` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 11:04:32.205251 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd40d9f5-baf1-487a-a9ab-1ff4f3dd07eb admin admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 11:04:32.205495 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd40d9f5-baf1-487a-a9ab-1ff4f3dd07eb admin admin] PATH_INFO: `/v3/projects/d218b2dc5f6a404781afd3bbf74acb40` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 11:04:32.206705 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd40d9f5-baf1-487a-a9ab-1ff4f3dd07eb admin admin] RBAC: Authorizing `identity:delete_project(project_id=d218b2dc5f6a404781afd3bbf74acb40)` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 11:04:32.208693 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd40d9f5-baf1-487a-a9ab-1ff4f3dd07eb admin admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 11:04:32.257323 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.notifications [None req-dd40d9f5-baf1-487a-a9ab-1ff4f3dd07eb admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd218b2dc5f6a404781afd3bbf74acb40', 'request_id': 'req-dd40d9f5-baf1-487a-a9ab-1ff4f3dd07eb'} {{(pid=80144) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 11:04:32.263467 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.notifications [None req-dd40d9f5-baf1-487a-a9ab-1ff4f3dd07eb admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd218b2dc5f6a404781afd3bbf74acb40', 'request_id': 'req-dd40d9f5-baf1-487a-a9ab-1ff4f3dd07eb'} {{(pid=80144) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 11:04:32.271044 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.notifications [None req-dd40d9f5-baf1-487a-a9ab-1ff4f3dd07eb admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd218b2dc5f6a404781afd3bbf74acb40', 'request_id': 'req-dd40d9f5-baf1-487a-a9ab-1ff4f3dd07eb'} {{(pid=80144) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 11:04:32.282740 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.notifications [None req-dd40d9f5-baf1-487a-a9ab-1ff4f3dd07eb admin admin] The token cache is being invalidate because project d218b2dc5f6a404781afd3bbf74acb40 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80144) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 07 11:04:32.283178 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.notifications [None req-dd40d9f5-baf1-487a-a9ab-1ff4f3dd07eb admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80144) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 11:04:32.283822 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.notifications [None req-dd40d9f5-baf1-487a-a9ab-1ff4f3dd07eb admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80144) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 11:04:32.285315 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 1267/2504] 10.222.0.25 () {58 vars in 1233 bytes} [Sun Aug 7 11:04:32 2022] DELETE /identity/v3/projects/d218b2dc5f6a404781afd3bbf74acb40 => generated 0 bytes in 100 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 07 11:04:41.892571 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=31,keystone:DELETE=2,keystone:INSERT=1 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 11:04:42.279848 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=60,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 11:04:44.594626 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1ec2454-80b1-4721-9422-0e46229f9839 None None] REQUEST_METHOD: `POST` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 11:04:44.595132 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1ec2454-80b1-4721-9422-0e46229f9839 None None] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 11:04:44.595229 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1ec2454-80b1-4721-9422-0e46229f9839 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 11:04:44.614157 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.auth.core [None req-e1ec2454-80b1-4721-9422-0e46229f9839 None None] MFA Rules not processed for user `06231f178e0d494084915b9368fa3b9b`. Rule list: `[]` (Enabled: `True`). {{(pid=80143) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 11:04:44.615387 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-e1ec2454-80b1-4721-9422-0e46229f9839 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 11:04:44.669423 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1238/2505] 10.222.0.25 () {58 vars in 958 bytes} [Sun Aug 7 11:04:44 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 76 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 07 11:04:44.973557 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd40d9f5-baf1-487a-a9ab-1ff4f3dd07eb admin admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 11:04:44.975858 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.fernet_utils [None req-dd40d9f5-baf1-487a-a9ab-1ff4f3dd07eb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80144) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 11:04:45.032598 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd40d9f5-baf1-487a-a9ab-1ff4f3dd07eb admin admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 11:04:45.036182 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e36877b1-2527-456e-b237-68f1cb4fefe1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be1283c9bf47424c9aa3ad27cab03027', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6de1f3cac02c4bcdb05dd2337176033d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 11:04:45.036724 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e36877b1-2527-456e-b237-68f1cb4fefe1 service neutron] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 11:04:45.037008 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e36877b1-2527-456e-b237-68f1cb4fefe1 service neutron] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 11:04:45.037283 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e36877b1-2527-456e-b237-68f1cb4fefe1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 11:04:45.039022 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.fernet_utils [None req-e36877b1-2527-456e-b237-68f1cb4fefe1 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80144) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 11:04:45.048690 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e36877b1-2527-456e-b237-68f1cb4fefe1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 11:04:45.050100 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e36877b1-2527-456e-b237-68f1cb4fefe1 service neutron] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 11:04:45.057872 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 1268/2506] 10.222.0.25 () {60 vars in 1365 bytes} [Sun Aug 7 11:04:44 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 07 11:04:45.378622 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1ec2454-80b1-4721-9422-0e46229f9839 None None] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 11:04:45.380167 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-e1ec2454-80b1-4721-9422-0e46229f9839 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 11:04:45.452839 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1ec2454-80b1-4721-9422-0e46229f9839 None None] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 11:04:45.457784 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb4642de-a9aa-4fb7-905c-06def25ed6d3 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6ee2e4c08af04c219ed395d7ca7085a6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6de1f3cac02c4bcdb05dd2337176033d', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 11:04:45.458387 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb4642de-a9aa-4fb7-905c-06def25ed6d3 service placement] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 11:04:45.458666 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb4642de-a9aa-4fb7-905c-06def25ed6d3 service placement] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 11:04:45.458962 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb4642de-a9aa-4fb7-905c-06def25ed6d3 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 11:04:45.460686 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-fb4642de-a9aa-4fb7-905c-06def25ed6d3 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 11:04:45.472168 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb4642de-a9aa-4fb7-905c-06def25ed6d3 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 11:04:45.473736 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb4642de-a9aa-4fb7-905c-06def25ed6d3 service placement] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 11:04:45.483491 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1239/2507] 10.222.0.25 () {60 vars in 1365 bytes} [Sun Aug 7 11:04:45 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 105 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 07 11:04:54.580361 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e36877b1-2527-456e-b237-68f1cb4fefe1 service neutron] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 11:04:54.582462 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.fernet_utils [None req-e36877b1-2527-456e-b237-68f1cb4fefe1 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80144) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 11:04:54.677574 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e36877b1-2527-456e-b237-68f1cb4fefe1 service neutron] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 11:04:54.683572 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-448f6ff1-25f6-4d5e-b0c4-2e453329e86f service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31d16d1b96104b4eb690398432b9829e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6de1f3cac02c4bcdb05dd2337176033d', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'reader', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 11:04:54.684195 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-448f6ff1-25f6-4d5e-b0c4-2e453329e86f service nova] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 11:04:54.684479 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-448f6ff1-25f6-4d5e-b0c4-2e453329e86f service nova] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 11:04:54.684757 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-448f6ff1-25f6-4d5e-b0c4-2e453329e86f service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 11:04:54.686752 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.fernet_utils [None req-448f6ff1-25f6-4d5e-b0c4-2e453329e86f service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80144) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 11:04:54.699833 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-448f6ff1-25f6-4d5e-b0c4-2e453329e86f service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 11:04:54.701554 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-448f6ff1-25f6-4d5e-b0c4-2e453329e86f service nova] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 11:04:54.713111 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 1269/2508] 10.222.0.25 () {60 vars in 1365 bytes} [Sun Aug 7 11:04:54 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 133 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 07 11:04:55.479253 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=57 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 11:04:58.675536 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb4642de-a9aa-4fb7-905c-06def25ed6d3 service placement] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 11:04:58.707536 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb4642de-a9aa-4fb7-905c-06def25ed6d3 service placement] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 11:04:58.714900 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17266037-f9f0-4a52-834c-692bbb574097 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31d16d1b96104b4eb690398432b9829e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6de1f3cac02c4bcdb05dd2337176033d', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'reader', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 11:04:58.715897 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17266037-f9f0-4a52-834c-692bbb574097 service nova] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 11:04:58.716384 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17266037-f9f0-4a52-834c-692bbb574097 service nova] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 11:04:58.716871 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17266037-f9f0-4a52-834c-692bbb574097 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 11:04:58.719070 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-17266037-f9f0-4a52-834c-692bbb574097 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 11:04:58.734456 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17266037-f9f0-4a52-834c-692bbb574097 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 11:04:58.736581 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17266037-f9f0-4a52-834c-692bbb574097 service nova] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 11:04:58.748864 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1240/2509] 10.222.0.25 () {60 vars in 1365 bytes} [Sun Aug 7 11:04:58 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 79 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 07 11:05:02.628833 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-448f6ff1-25f6-4d5e-b0c4-2e453329e86f service nova] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 11:05:02.637970 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-448f6ff1-25f6-4d5e-b0c4-2e453329e86f service nova] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 11:05:02.641531 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f97fa8ad-bf41-4b65-9235-6d65cea220c3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be1283c9bf47424c9aa3ad27cab03027', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6de1f3cac02c4bcdb05dd2337176033d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 11:05:02.642786 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f97fa8ad-bf41-4b65-9235-6d65cea220c3 service neutron] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 11:05:02.643101 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f97fa8ad-bf41-4b65-9235-6d65cea220c3 service neutron] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 11:05:02.643377 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f97fa8ad-bf41-4b65-9235-6d65cea220c3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 11:05:02.645085 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.fernet_utils [None req-f97fa8ad-bf41-4b65-9235-6d65cea220c3 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80144) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 11:05:02.708155 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f97fa8ad-bf41-4b65-9235-6d65cea220c3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 11:05:02.709542 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f97fa8ad-bf41-4b65-9235-6d65cea220c3 service neutron] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 11:05:02.719322 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 1270/2510] 10.222.0.25 () {60 vars in 1365 bytes} [Sun Aug 7 11:05:02 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 07 11:05:05.857091 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17266037-f9f0-4a52-834c-692bbb574097 service nova] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 11:05:05.858863 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-17266037-f9f0-4a52-834c-692bbb574097 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 11:05:05.874373 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17266037-f9f0-4a52-834c-692bbb574097 service nova] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 11:05:05.882435 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1d05239-8724-44ed-8a43-a2ec91574f56 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5ac47e3b09f43d19a23a0f457fe9e23', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 11:05:05.883120 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1d05239-8724-44ed-8a43-a2ec91574f56 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 11:05:05.883400 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1d05239-8724-44ed-8a43-a2ec91574f56 admin admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 11:05:05.883671 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1d05239-8724-44ed-8a43-a2ec91574f56 admin admin] PATH_INFO: `/v3/users/2b64b05230b940eba992a063ea7cd778` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 11:05:05.898311 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1d05239-8724-44ed-8a43-a2ec91574f56 admin admin] RBAC: Authorizing `identity:delete_user(user_id=2b64b05230b940eba992a063ea7cd778)` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 11:05:05.900402 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1d05239-8724-44ed-8a43-a2ec91574f56 admin admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 11:05:05.945569 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-f1d05239-8724-44ed-8a43-a2ec91574f56 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2b64b05230b940eba992a063ea7cd778'} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 11:05:05.952333 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-f1d05239-8724-44ed-8a43-a2ec91574f56 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2b64b05230b940eba992a063ea7cd778'} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 11:05:05.964835 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-f1d05239-8724-44ed-8a43-a2ec91574f56 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2b64b05230b940eba992a063ea7cd778'} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 11:05:05.965504 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-f1d05239-8724-44ed-8a43-a2ec91574f56 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2b64b05230b940eba992a063ea7cd778'} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 11:05:05.973203 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-f1d05239-8724-44ed-8a43-a2ec91574f56 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2b64b05230b940eba992a063ea7cd778'} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 11:05:05.979116 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1241/2511] 10.222.0.25 () {58 vars in 1224 bytes} [Sun Aug 7 11:05:05 2022] DELETE /identity/v3/users/2b64b05230b940eba992a063ea7cd778 => generated 0 bytes in 123 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 07 11:05:06.159582 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f97fa8ad-bf41-4b65-9235-6d65cea220c3 service neutron] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 11:05:06.169865 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f97fa8ad-bf41-4b65-9235-6d65cea220c3 service neutron] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 11:05:06.190856 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=100 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 11:05:06.231920 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03a260a5-0536-46af-b5cf-317ec24ef725 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5ac47e3b09f43d19a23a0f457fe9e23', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 11:05:06.232560 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03a260a5-0536-46af-b5cf-317ec24ef725 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 11:05:06.232846 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03a260a5-0536-46af-b5cf-317ec24ef725 admin admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 11:05:06.233122 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03a260a5-0536-46af-b5cf-317ec24ef725 admin admin] PATH_INFO: `/v3/projects/3c3be73ff916420f8c9eb418ef5f6716` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 11:05:06.243663 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03a260a5-0536-46af-b5cf-317ec24ef725 admin admin] RBAC: Authorizing `identity:delete_project(project_id=3c3be73ff916420f8c9eb418ef5f6716)` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 11:05:06.245571 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03a260a5-0536-46af-b5cf-317ec24ef725 admin admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 11:05:06.299295 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.notifications [None req-03a260a5-0536-46af-b5cf-317ec24ef725 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3c3be73ff916420f8c9eb418ef5f6716', 'request_id': 'req-03a260a5-0536-46af-b5cf-317ec24ef725'} {{(pid=80144) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 11:05:06.305083 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.notifications [None req-03a260a5-0536-46af-b5cf-317ec24ef725 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3c3be73ff916420f8c9eb418ef5f6716', 'request_id': 'req-03a260a5-0536-46af-b5cf-317ec24ef725'} {{(pid=80144) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 11:05:06.311594 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.notifications [None req-03a260a5-0536-46af-b5cf-317ec24ef725 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3c3be73ff916420f8c9eb418ef5f6716', 'request_id': 'req-03a260a5-0536-46af-b5cf-317ec24ef725'} {{(pid=80144) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 11:05:06.324859 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.notifications [None req-03a260a5-0536-46af-b5cf-317ec24ef725 admin admin] The token cache is being invalidate because project 3c3be73ff916420f8c9eb418ef5f6716 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80144) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 07 11:05:06.325529 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.notifications [None req-03a260a5-0536-46af-b5cf-317ec24ef725 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80144) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 11:05:06.326389 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.notifications [None req-03a260a5-0536-46af-b5cf-317ec24ef725 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80144) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 11:05:06.328043 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 1271/2512] 10.222.0.25 () {58 vars in 1233 bytes} [Sun Aug 7 11:05:06 2022] DELETE /identity/v3/projects/3c3be73ff916420f8c9eb418ef5f6716 => generated 0 bytes in 169 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 07 11:05:06.334846 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b857cdac-80cb-49eb-97d9-c50f5ecdf28f None None] REQUEST_METHOD: `POST` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 11:05:06.335209 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b857cdac-80cb-49eb-97d9-c50f5ecdf28f None None] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 11:05:06.335502 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b857cdac-80cb-49eb-97d9-c50f5ecdf28f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 11:05:06.354486 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.auth.core [None req-b857cdac-80cb-49eb-97d9-c50f5ecdf28f None None] MFA Rules not processed for user `bac4e6b76fe64217ac8fa44e71cffe6a`. Rule list: `[]` (Enabled: `True`). {{(pid=80143) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 11:05:06.355702 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-b857cdac-80cb-49eb-97d9-c50f5ecdf28f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 11:05:06.412826 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1242/2513] 10.222.0.25 () {58 vars in 958 bytes} [Sun Aug 7 11:05:06 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 07 11:05:06.420564 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ce46d50-7881-4aee-bc08-488a5efc0070 None None] REQUEST_METHOD: `POST` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 11:05:06.420854 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ce46d50-7881-4aee-bc08-488a5efc0070 None None] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 11:05:06.421134 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ce46d50-7881-4aee-bc08-488a5efc0070 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 11:05:06.441276 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.auth.core [None req-3ce46d50-7881-4aee-bc08-488a5efc0070 None None] MFA Rules not processed for user `bac4e6b76fe64217ac8fa44e71cffe6a`. Rule list: `[]` (Enabled: `True`). {{(pid=80144) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 11:05:06.442469 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.fernet_utils [None req-3ce46d50-7881-4aee-bc08-488a5efc0070 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80144) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 11:05:06.450258 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 1272/2514] 10.222.0.25 () {58 vars in 958 bytes} [Sun Aug 7 11:05:06 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 07 11:05:06.454435 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b857cdac-80cb-49eb-97d9-c50f5ecdf28f None None] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 11:05:06.464085 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b857cdac-80cb-49eb-97d9-c50f5ecdf28f None None] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 11:05:06.468649 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ebb7fca-f409-46a7-b1ee-e1d2435c12a4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5ac47e3b09f43d19a23a0f457fe9e23', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 11:05:06.469239 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ebb7fca-f409-46a7-b1ee-e1d2435c12a4 admin admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 11:05:06.469517 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ebb7fca-f409-46a7-b1ee-e1d2435c12a4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 11:05:06.469788 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ebb7fca-f409-46a7-b1ee-e1d2435c12a4 admin admin] PATH_INFO: `/v3/domains` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 11:05:06.471404 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ebb7fca-f409-46a7-b1ee-e1d2435c12a4 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80143) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 11:05:06.471707 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ebb7fca-f409-46a7-b1ee-e1d2435c12a4 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 11:05:06.473285 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ebb7fca-f409-46a7-b1ee-e1d2435c12a4 admin admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 11:05:06.484740 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1243/2515] 10.222.0.25 () {58 vars in 1153 bytes} [Sun Aug 7 11:05:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 11:05:06.489544 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ce46d50-7881-4aee-bc08-488a5efc0070 None None] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 11:05:06.498771 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ce46d50-7881-4aee-bc08-488a5efc0070 None None] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 11:05:06.504463 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c4918ed-73e8-4c9d-9938-fec119c7adaa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5ac47e3b09f43d19a23a0f457fe9e23', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 11:05:06.505064 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c4918ed-73e8-4c9d-9938-fec119c7adaa admin admin] REQUEST_METHOD: `POST` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 11:05:06.505343 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c4918ed-73e8-4c9d-9938-fec119c7adaa admin admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 11:05:06.505617 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c4918ed-73e8-4c9d-9938-fec119c7adaa admin admin] PATH_INFO: `/v3/projects` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 11:05:06.506237 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c4918ed-73e8-4c9d-9938-fec119c7adaa admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 11:05:06.508177 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c4918ed-73e8-4c9d-9938-fec119c7adaa admin admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 11:05:06.543601 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 1273/2516] 10.222.0.25 () {60 vars in 1153 bytes} [Sun Aug 7 11:05:06 2022] POST /identity/v3/projects => generated 381 bytes in 54 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 07 11:05:06.547861 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ebb7fca-f409-46a7-b1ee-e1d2435c12a4 admin admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 11:05:06.558372 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ebb7fca-f409-46a7-b1ee-e1d2435c12a4 admin admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 11:05:06.589852 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 11:05:06.615268 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad8b221a-c4f7-4758-b2e4-8529cb6d019d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5ac47e3b09f43d19a23a0f457fe9e23', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 11:05:06.615869 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad8b221a-c4f7-4758-b2e4-8529cb6d019d admin admin] REQUEST_METHOD: `POST` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 11:05:06.616147 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad8b221a-c4f7-4758-b2e4-8529cb6d019d admin admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 11:05:06.616413 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad8b221a-c4f7-4758-b2e4-8529cb6d019d admin admin] PATH_INFO: `/v3/users` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 11:05:06.617037 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad8b221a-c4f7-4758-b2e4-8529cb6d019d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 11:05:06.618944 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad8b221a-c4f7-4758-b2e4-8529cb6d019d admin admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 11:05:06.640308 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1244/2517] 10.222.0.25 () {60 vars in 1144 bytes} [Sun Aug 7 11:05:06 2022] POST /identity/v3/users => generated 291 bytes in 93 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 07 11:05:06.644711 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c4918ed-73e8-4c9d-9938-fec119c7adaa admin admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 11:05:06.654881 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c4918ed-73e8-4c9d-9938-fec119c7adaa admin admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 11:05:06.659597 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36054165-5280-4021-ac59-2c1748a0652e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5ac47e3b09f43d19a23a0f457fe9e23', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 11:05:06.660191 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36054165-5280-4021-ac59-2c1748a0652e admin admin] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 11:05:06.660473 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36054165-5280-4021-ac59-2c1748a0652e admin admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 11:05:06.660777 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36054165-5280-4021-ac59-2c1748a0652e admin admin] PATH_INFO: `/v3/roles` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 11:05:06.661426 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36054165-5280-4021-ac59-2c1748a0652e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 11:05:06.663027 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36054165-5280-4021-ac59-2c1748a0652e admin admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 11:05:06.672702 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 1274/2518] 10.222.0.25 () {58 vars in 1122 bytes} [Sun Aug 7 11:05:06 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 07 11:05:06.677002 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad8b221a-c4f7-4758-b2e4-8529cb6d019d admin admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 11:05:06.686643 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad8b221a-c4f7-4758-b2e4-8529cb6d019d admin admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 11:05:06.691919 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d25d6674-74be-40fe-8a8d-5d1b34e40cd6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5ac47e3b09f43d19a23a0f457fe9e23', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 11:05:06.692501 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d25d6674-74be-40fe-8a8d-5d1b34e40cd6 admin admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 11:05:06.692778 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d25d6674-74be-40fe-8a8d-5d1b34e40cd6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 11:05:06.693049 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d25d6674-74be-40fe-8a8d-5d1b34e40cd6 admin admin] PATH_INFO: `/v3/roles` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 11:05:06.693677 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d25d6674-74be-40fe-8a8d-5d1b34e40cd6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 11:05:06.695285 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d25d6674-74be-40fe-8a8d-5d1b34e40cd6 admin admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 11:05:06.705988 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1245/2519] 10.222.0.25 () {58 vars in 1122 bytes} [Sun Aug 7 11:05:06 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 07 11:05:06.709923 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36054165-5280-4021-ac59-2c1748a0652e admin admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 11:05:06.720200 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36054165-5280-4021-ac59-2c1748a0652e admin admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 11:05:06.724793 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45ed80fc-ba5d-400a-a39f-c728a36aa5b6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5ac47e3b09f43d19a23a0f457fe9e23', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 11:05:06.725461 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45ed80fc-ba5d-400a-a39f-c728a36aa5b6 admin admin] REQUEST_METHOD: `PUT` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 11:05:06.725740 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45ed80fc-ba5d-400a-a39f-c728a36aa5b6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 11:05:06.726012 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45ed80fc-ba5d-400a-a39f-c728a36aa5b6 admin admin] PATH_INFO: `/v3/projects/d2081ed3efcc48f9b553cc6e94b7a600/users/81c25f007cd24f878fada9dca47fc8e6/roles/b98dfb3b9fee4644af2887a3ac3fd2e6` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 11:05:06.741286 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45ed80fc-ba5d-400a-a39f-c728a36aa5b6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d2081ed3efcc48f9b553cc6e94b7a600, user_id=81c25f007cd24f878fada9dca47fc8e6, role_id=b98dfb3b9fee4644af2887a3ac3fd2e6)` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 11:05:06.744053 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45ed80fc-ba5d-400a-a39f-c728a36aa5b6 admin admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 11:05:06.757173 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 1275/2520] 10.222.0.25 () {60 vars in 1483 bytes} [Sun Aug 7 11:05:06 2022] PUT /identity/v3/projects/d2081ed3efcc48f9b553cc6e94b7a600/users/81c25f007cd24f878fada9dca47fc8e6/roles/b98dfb3b9fee4644af2887a3ac3fd2e6 => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 07 11:05:06.768751 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17c6d960-467c-41d7-902f-386297485882 None None] REQUEST_METHOD: `POST` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 11:05:06.769382 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17c6d960-467c-41d7-902f-386297485882 None None] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 11:05:06.769664 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17c6d960-467c-41d7-902f-386297485882 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 11:05:06.796749 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.auth.core [None req-17c6d960-467c-41d7-902f-386297485882 None None] MFA Rules not processed for user `81c25f007cd24f878fada9dca47fc8e6`. Rule list: `[]` (Enabled: `True`). {{(pid=80143) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 11:05:06.797930 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-17c6d960-467c-41d7-902f-386297485882 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 11:05:06.860248 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1246/2521] 10.222.0.25 () {58 vars in 958 bytes} [Sun Aug 7 11:05:06 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 07 11:05:06.865494 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45ed80fc-ba5d-400a-a39f-c728a36aa5b6 admin admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 11:05:06.897463 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 11:05:06.928457 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45ed80fc-ba5d-400a-a39f-c728a36aa5b6 admin admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 11:05:06.933439 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-715dfaf7-c096-456c-893e-5b88b6d944cb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5ac47e3b09f43d19a23a0f457fe9e23', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 11:05:06.934078 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-715dfaf7-c096-456c-893e-5b88b6d944cb admin admin] REQUEST_METHOD: `POST` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 11:05:06.934357 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-715dfaf7-c096-456c-893e-5b88b6d944cb admin admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 11:05:06.934632 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-715dfaf7-c096-456c-893e-5b88b6d944cb admin admin] PATH_INFO: `/v3/projects` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 11:05:06.935286 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-715dfaf7-c096-456c-893e-5b88b6d944cb admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 11:05:06.937236 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-715dfaf7-c096-456c-893e-5b88b6d944cb admin admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 11:05:06.965884 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 1276/2522] 10.222.0.25 () {60 vars in 1153 bytes} [Sun Aug 7 11:05:06 2022] POST /identity/v3/projects => generated 383 bytes in 98 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 07 11:05:06.967016 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17c6d960-467c-41d7-902f-386297485882 None None] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 11:05:06.979026 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17c6d960-467c-41d7-902f-386297485882 None None] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 11:05:07.035883 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4b5e1c6-9e4e-4c04-b353-c3d202f5bf2e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5ac47e3b09f43d19a23a0f457fe9e23', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 11:05:07.036486 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4b5e1c6-9e4e-4c04-b353-c3d202f5bf2e admin admin] REQUEST_METHOD: `POST` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 11:05:07.036771 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4b5e1c6-9e4e-4c04-b353-c3d202f5bf2e admin admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 11:05:07.037052 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4b5e1c6-9e4e-4c04-b353-c3d202f5bf2e admin admin] PATH_INFO: `/v3/users` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 11:05:07.037670 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4b5e1c6-9e4e-4c04-b353-c3d202f5bf2e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 11:05:07.039610 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4b5e1c6-9e4e-4c04-b353-c3d202f5bf2e admin admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 11:05:07.061444 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1247/2523] 10.222.0.25 () {60 vars in 1144 bytes} [Sun Aug 7 11:05:06 2022] POST /identity/v3/users => generated 298 bytes in 95 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 07 11:05:07.065724 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-715dfaf7-c096-456c-893e-5b88b6d944cb admin admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 11:05:07.076156 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-715dfaf7-c096-456c-893e-5b88b6d944cb admin admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 11:05:07.081044 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-034d39fd-4296-4e1c-97cc-43bab88be24d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5ac47e3b09f43d19a23a0f457fe9e23', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 11:05:07.081616 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-034d39fd-4296-4e1c-97cc-43bab88be24d admin admin] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 11:05:07.081896 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-034d39fd-4296-4e1c-97cc-43bab88be24d admin admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 11:05:07.082167 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-034d39fd-4296-4e1c-97cc-43bab88be24d admin admin] PATH_INFO: `/v3/roles` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 11:05:07.082793 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-034d39fd-4296-4e1c-97cc-43bab88be24d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 11:05:07.084407 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-034d39fd-4296-4e1c-97cc-43bab88be24d admin admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 11:05:07.094577 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 1277/2524] 10.222.0.25 () {58 vars in 1122 bytes} [Sun Aug 7 11:05:07 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 07 11:05:07.099647 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4b5e1c6-9e4e-4c04-b353-c3d202f5bf2e admin admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 11:05:07.110124 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4b5e1c6-9e4e-4c04-b353-c3d202f5bf2e admin admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 11:05:07.115914 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a52f1ba8-e973-4376-bfe9-0a3584984704 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5ac47e3b09f43d19a23a0f457fe9e23', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 11:05:07.116503 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a52f1ba8-e973-4376-bfe9-0a3584984704 admin admin] REQUEST_METHOD: `PUT` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 11:05:07.116790 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a52f1ba8-e973-4376-bfe9-0a3584984704 admin admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 11:05:07.117065 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a52f1ba8-e973-4376-bfe9-0a3584984704 admin admin] PATH_INFO: `/v3/projects/85d8dfe2f72b4f67953a253948697866/users/b85d8db6198e42e3bdd234c0abd77b26/roles/81b1f1bcee2a473088fbbd327c1c5b23` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 11:05:07.131825 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a52f1ba8-e973-4376-bfe9-0a3584984704 admin admin] RBAC: Authorizing `identity:create_grant(project_id=85d8dfe2f72b4f67953a253948697866, user_id=b85d8db6198e42e3bdd234c0abd77b26, role_id=81b1f1bcee2a473088fbbd327c1c5b23)` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 11:05:07.134449 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a52f1ba8-e973-4376-bfe9-0a3584984704 admin admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 11:05:07.147620 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1248/2525] 10.222.0.25 () {60 vars in 1483 bytes} [Sun Aug 7 11:05:07 2022] PUT /identity/v3/projects/85d8dfe2f72b4f67953a253948697866/users/b85d8db6198e42e3bdd234c0abd77b26/roles/81b1f1bcee2a473088fbbd327c1c5b23 => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 07 11:05:07.151622 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-034d39fd-4296-4e1c-97cc-43bab88be24d admin admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 11:05:07.162383 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-034d39fd-4296-4e1c-97cc-43bab88be24d admin admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 11:05:07.221189 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aacec465-2e89-4cdf-a2a3-75137ad7632f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5ac47e3b09f43d19a23a0f457fe9e23', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 11:05:07.221796 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aacec465-2e89-4cdf-a2a3-75137ad7632f admin admin] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 11:05:07.222077 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aacec465-2e89-4cdf-a2a3-75137ad7632f admin admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 11:05:07.222352 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aacec465-2e89-4cdf-a2a3-75137ad7632f admin admin] PATH_INFO: `/v3/roles` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 11:05:07.223021 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aacec465-2e89-4cdf-a2a3-75137ad7632f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 11:05:07.224603 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aacec465-2e89-4cdf-a2a3-75137ad7632f admin admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 11:05:07.234485 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 1278/2526] 10.222.0.25 () {58 vars in 1122 bytes} [Sun Aug 7 11:05:07 2022] GET /identity/v3/roles => generated 1420 bytes in 83 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 07 11:05:07.238780 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a52f1ba8-e973-4376-bfe9-0a3584984704 admin admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 11:05:07.248976 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a52f1ba8-e973-4376-bfe9-0a3584984704 admin admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 11:05:07.254275 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a1a3c80-43f9-4df5-8c8b-8ff5205dc81a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5ac47e3b09f43d19a23a0f457fe9e23', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 11:05:07.254951 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a1a3c80-43f9-4df5-8c8b-8ff5205dc81a admin admin] REQUEST_METHOD: `PUT` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 11:05:07.255251 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a1a3c80-43f9-4df5-8c8b-8ff5205dc81a admin admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 11:05:07.255526 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a1a3c80-43f9-4df5-8c8b-8ff5205dc81a admin admin] PATH_INFO: `/v3/projects/85d8dfe2f72b4f67953a253948697866/users/b85d8db6198e42e3bdd234c0abd77b26/roles/81b1f1bcee2a473088fbbd327c1c5b23` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 11:05:07.257447 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a1a3c80-43f9-4df5-8c8b-8ff5205dc81a admin admin] RBAC: Authorizing `identity:create_grant(project_id=85d8dfe2f72b4f67953a253948697866, user_id=b85d8db6198e42e3bdd234c0abd77b26, role_id=81b1f1bcee2a473088fbbd327c1c5b23)` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 11:05:07.260095 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a1a3c80-43f9-4df5-8c8b-8ff5205dc81a admin admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 11:05:07.263124 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=92,keystone:INSERT=8 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 11:05:07.272799 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1249/2527] 10.222.0.25 () {60 vars in 1483 bytes} [Sun Aug 7 11:05:07 2022] PUT /identity/v3/projects/85d8dfe2f72b4f67953a253948697866/users/b85d8db6198e42e3bdd234c0abd77b26/roles/81b1f1bcee2a473088fbbd327c1c5b23 => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 07 11:05:07.283704 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9eeec94-ebf7-417c-888a-92c54ed15b4b None None] REQUEST_METHOD: `POST` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 11:05:07.283995 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9eeec94-ebf7-417c-888a-92c54ed15b4b None None] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 11:05:07.284273 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9eeec94-ebf7-417c-888a-92c54ed15b4b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 11:05:07.311812 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.auth.core [None req-c9eeec94-ebf7-417c-888a-92c54ed15b4b None None] MFA Rules not processed for user `b85d8db6198e42e3bdd234c0abd77b26`. Rule list: `[]` (Enabled: `True`). {{(pid=80144) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 11:05:07.313055 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.fernet_utils [None req-c9eeec94-ebf7-417c-888a-92c54ed15b4b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80144) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 11:05:07.382194 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 1279/2528] 10.222.0.25 () {58 vars in 958 bytes} [Sun Aug 7 11:05:07 2022] POST /identity/v3/auth/tokens => generated 3327 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 07 11:05:07.393341 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a1a3c80-43f9-4df5-8c8b-8ff5205dc81a admin admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 11:05:07.395072 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-3a1a3c80-43f9-4df5-8c8b-8ff5205dc81a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 11:05:07.457082 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a1a3c80-43f9-4df5-8c8b-8ff5205dc81a admin admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 11:05:07.460767 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-062a708f-b781-4870-85da-761612ca8b47 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88dbd6dfd1164994a3e16da553561910', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6de1f3cac02c4bcdb05dd2337176033d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 11:05:07.461383 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-062a708f-b781-4870-85da-761612ca8b47 service cinder] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 11:05:07.461671 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-062a708f-b781-4870-85da-761612ca8b47 service cinder] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 11:05:07.461947 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-062a708f-b781-4870-85da-761612ca8b47 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 11:05:07.474130 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-062a708f-b781-4870-85da-761612ca8b47 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 11:05:07.475672 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-062a708f-b781-4870-85da-761612ca8b47 service cinder] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 11:05:07.485503 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1250/2529] 10.222.0.25 () {60 vars in 1365 bytes} [Sun Aug 7 11:05:07 2022] GET /identity/v3/auth/tokens => generated 3327 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 07 11:05:07.588627 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9eeec94-ebf7-417c-888a-92c54ed15b4b None None] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 11:05:07.598537 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9eeec94-ebf7-417c-888a-92c54ed15b4b None None] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 11:05:07.602065 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db60c9be-8055-4249-bf45-cd4a964f623a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88dbd6dfd1164994a3e16da553561910', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6de1f3cac02c4bcdb05dd2337176033d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 11:05:07.602664 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db60c9be-8055-4249-bf45-cd4a964f623a service cinder] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 11:05:07.602967 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db60c9be-8055-4249-bf45-cd4a964f623a service cinder] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 11:05:07.603244 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db60c9be-8055-4249-bf45-cd4a964f623a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 11:05:07.627893 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 11:05:07.661342 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db60c9be-8055-4249-bf45-cd4a964f623a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 11:05:07.662689 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db60c9be-8055-4249-bf45-cd4a964f623a service cinder] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 11:05:07.671412 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 1280/2530] 10.222.0.25 () {60 vars in 1365 bytes} [Sun Aug 7 11:05:07 2022] GET /identity/v3/auth/tokens => generated 3258 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 07 11:05:10.244102 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-062a708f-b781-4870-85da-761612ca8b47 service cinder] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 11:05:10.305785 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-062a708f-b781-4870-85da-761612ca8b47 service cinder] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 11:05:10.310310 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bb76885-9a44-4854-9e75-f0da86f8a8c0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5ac47e3b09f43d19a23a0f457fe9e23', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 11:05:10.310954 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bb76885-9a44-4854-9e75-f0da86f8a8c0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 11:05:10.311235 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bb76885-9a44-4854-9e75-f0da86f8a8c0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 11:05:10.311509 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bb76885-9a44-4854-9e75-f0da86f8a8c0 admin admin] PATH_INFO: `/v3/users/81c25f007cd24f878fada9dca47fc8e6` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 11:05:10.312625 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bb76885-9a44-4854-9e75-f0da86f8a8c0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=81c25f007cd24f878fada9dca47fc8e6)` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 11:05:10.314836 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bb76885-9a44-4854-9e75-f0da86f8a8c0 admin admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 11:05:10.360946 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-1bb76885-9a44-4854-9e75-f0da86f8a8c0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '81c25f007cd24f878fada9dca47fc8e6'} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 11:05:10.368129 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-1bb76885-9a44-4854-9e75-f0da86f8a8c0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '81c25f007cd24f878fada9dca47fc8e6'} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 11:05:10.380191 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-1bb76885-9a44-4854-9e75-f0da86f8a8c0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '81c25f007cd24f878fada9dca47fc8e6'} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 11:05:10.380856 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-1bb76885-9a44-4854-9e75-f0da86f8a8c0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '81c25f007cd24f878fada9dca47fc8e6'} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 11:05:10.387690 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-1bb76885-9a44-4854-9e75-f0da86f8a8c0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '81c25f007cd24f878fada9dca47fc8e6'} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 11:05:10.394500 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1251/2531] 10.222.0.25 () {58 vars in 1224 bytes} [Sun Aug 7 11:05:10 2022] DELETE /identity/v3/users/81c25f007cd24f878fada9dca47fc8e6 => generated 0 bytes in 151 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 07 11:05:10.399085 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ec1913c-feb5-4d04-a2e1-99aac29b48c6 None None] REQUEST_METHOD: `POST` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 11:05:10.399578 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ec1913c-feb5-4d04-a2e1-99aac29b48c6 None None] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 11:05:10.399702 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ec1913c-feb5-4d04-a2e1-99aac29b48c6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 11:05:10.419594 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.auth.core [None req-9ec1913c-feb5-4d04-a2e1-99aac29b48c6 None None] MFA Rules not processed for user `bac4e6b76fe64217ac8fa44e71cffe6a`. Rule list: `[]` (Enabled: `True`). {{(pid=80144) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 11:05:10.420857 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.fernet_utils [None req-9ec1913c-feb5-4d04-a2e1-99aac29b48c6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80144) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 11:05:10.479166 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 1281/2532] 10.222.0.25 () {58 vars in 958 bytes} [Sun Aug 7 11:05:10 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 07 11:05:10.489345 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bb76885-9a44-4854-9e75-f0da86f8a8c0 admin admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 11:05:10.490751 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-1bb76885-9a44-4854-9e75-f0da86f8a8c0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 11:05:10.511410 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 11:05:10.550723 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bb76885-9a44-4854-9e75-f0da86f8a8c0 admin admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 11:05:10.554468 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80059a44-8b5f-4ca1-8f5f-f01fe339acb6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be1283c9bf47424c9aa3ad27cab03027', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6de1f3cac02c4bcdb05dd2337176033d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 11:05:10.555104 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80059a44-8b5f-4ca1-8f5f-f01fe339acb6 service neutron] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 11:05:10.555385 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80059a44-8b5f-4ca1-8f5f-f01fe339acb6 service neutron] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 11:05:10.555653 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80059a44-8b5f-4ca1-8f5f-f01fe339acb6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 11:05:10.565186 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80059a44-8b5f-4ca1-8f5f-f01fe339acb6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 11:05:10.566589 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80059a44-8b5f-4ca1-8f5f-f01fe339acb6 service neutron] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 11:05:10.575888 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1252/2533] 10.222.0.25 () {60 vars in 1365 bytes} [Sun Aug 7 11:05:10 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 07 11:05:10.843343 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ec1913c-feb5-4d04-a2e1-99aac29b48c6 None None] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 11:05:10.845396 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.fernet_utils [None req-9ec1913c-feb5-4d04-a2e1-99aac29b48c6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80144) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 11:05:10.861890 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ec1913c-feb5-4d04-a2e1-99aac29b48c6 None None] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 11:05:10.867540 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65726f5c-0961-4523-8c48-b8c20a8277f1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5ac47e3b09f43d19a23a0f457fe9e23', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 11:05:10.868241 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65726f5c-0961-4523-8c48-b8c20a8277f1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 11:05:10.868443 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65726f5c-0961-4523-8c48-b8c20a8277f1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 11:05:10.868717 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65726f5c-0961-4523-8c48-b8c20a8277f1 admin admin] PATH_INFO: `/v3/projects/d2081ed3efcc48f9b553cc6e94b7a600` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 11:05:10.869937 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65726f5c-0961-4523-8c48-b8c20a8277f1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d2081ed3efcc48f9b553cc6e94b7a600)` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 11:05:10.871898 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65726f5c-0961-4523-8c48-b8c20a8277f1 admin admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 11:05:10.919313 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.notifications [None req-65726f5c-0961-4523-8c48-b8c20a8277f1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd2081ed3efcc48f9b553cc6e94b7a600', 'request_id': 'req-65726f5c-0961-4523-8c48-b8c20a8277f1'} {{(pid=80144) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 11:05:10.924890 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.notifications [None req-65726f5c-0961-4523-8c48-b8c20a8277f1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd2081ed3efcc48f9b553cc6e94b7a600', 'request_id': 'req-65726f5c-0961-4523-8c48-b8c20a8277f1'} {{(pid=80144) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 11:05:10.931444 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.notifications [None req-65726f5c-0961-4523-8c48-b8c20a8277f1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd2081ed3efcc48f9b553cc6e94b7a600', 'request_id': 'req-65726f5c-0961-4523-8c48-b8c20a8277f1'} {{(pid=80144) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 11:05:10.942355 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.notifications [None req-65726f5c-0961-4523-8c48-b8c20a8277f1 admin admin] The token cache is being invalidate because project d2081ed3efcc48f9b553cc6e94b7a600 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80144) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 07 11:05:10.942647 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.notifications [None req-65726f5c-0961-4523-8c48-b8c20a8277f1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80144) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 11:05:10.943339 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.notifications [None req-65726f5c-0961-4523-8c48-b8c20a8277f1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80144) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 11:05:10.944898 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 1282/2534] 10.222.0.25 () {58 vars in 1233 bytes} [Sun Aug 7 11:05:10 2022] DELETE /identity/v3/projects/d2081ed3efcc48f9b553cc6e94b7a600 => generated 0 bytes in 102 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 07 11:05:10.948914 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80059a44-8b5f-4ca1-8f5f-f01fe339acb6 service neutron] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 11:05:10.959234 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80059a44-8b5f-4ca1-8f5f-f01fe339acb6 service neutron] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 11:05:11.013940 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db6940df-c1f3-45ac-893a-6e2902048a82 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5ac47e3b09f43d19a23a0f457fe9e23', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 11:05:11.014562 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db6940df-c1f3-45ac-893a-6e2902048a82 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 11:05:11.014843 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db6940df-c1f3-45ac-893a-6e2902048a82 admin admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 11:05:11.015145 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db6940df-c1f3-45ac-893a-6e2902048a82 admin admin] PATH_INFO: `/v3/users/b85d8db6198e42e3bdd234c0abd77b26` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 11:05:11.016313 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db6940df-c1f3-45ac-893a-6e2902048a82 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b85d8db6198e42e3bdd234c0abd77b26)` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 11:05:11.018750 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db6940df-c1f3-45ac-893a-6e2902048a82 admin admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 11:05:11.063325 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-db6940df-c1f3-45ac-893a-6e2902048a82 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b85d8db6198e42e3bdd234c0abd77b26'} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 11:05:11.070045 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-db6940df-c1f3-45ac-893a-6e2902048a82 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b85d8db6198e42e3bdd234c0abd77b26'} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 11:05:11.082250 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-db6940df-c1f3-45ac-893a-6e2902048a82 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b85d8db6198e42e3bdd234c0abd77b26'} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 11:05:11.082908 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-db6940df-c1f3-45ac-893a-6e2902048a82 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b85d8db6198e42e3bdd234c0abd77b26'} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 11:05:11.089769 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-db6940df-c1f3-45ac-893a-6e2902048a82 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b85d8db6198e42e3bdd234c0abd77b26'} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 11:05:11.096168 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1253/2535] 10.222.0.25 () {58 vars in 1224 bytes} [Sun Aug 7 11:05:10 2022] DELETE /identity/v3/users/b85d8db6198e42e3bdd234c0abd77b26 => generated 0 bytes in 148 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 07 11:05:11.368185 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65726f5c-0961-4523-8c48-b8c20a8277f1 admin admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 11:05:11.369681 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.fernet_utils [None req-65726f5c-0961-4523-8c48-b8c20a8277f1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80144) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 11:05:11.385455 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65726f5c-0961-4523-8c48-b8c20a8277f1 admin admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 11:05:11.432979 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 11:05:11.447113 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b1f17ed-3dd1-4a48-8be8-533cede36c1e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5ac47e3b09f43d19a23a0f457fe9e23', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 11:05:11.447753 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b1f17ed-3dd1-4a48-8be8-533cede36c1e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 11:05:11.448037 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b1f17ed-3dd1-4a48-8be8-533cede36c1e admin admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 11:05:11.448314 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b1f17ed-3dd1-4a48-8be8-533cede36c1e admin admin] PATH_INFO: `/v3/projects/85d8dfe2f72b4f67953a253948697866` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 11:05:11.449461 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b1f17ed-3dd1-4a48-8be8-533cede36c1e admin admin] RBAC: Authorizing `identity:delete_project(project_id=85d8dfe2f72b4f67953a253948697866)` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 11:05:11.451425 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b1f17ed-3dd1-4a48-8be8-533cede36c1e admin admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 11:05:11.497188 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.notifications [None req-7b1f17ed-3dd1-4a48-8be8-533cede36c1e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '85d8dfe2f72b4f67953a253948697866', 'request_id': 'req-7b1f17ed-3dd1-4a48-8be8-533cede36c1e'} {{(pid=80144) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 11:05:11.503442 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.notifications [None req-7b1f17ed-3dd1-4a48-8be8-533cede36c1e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '85d8dfe2f72b4f67953a253948697866', 'request_id': 'req-7b1f17ed-3dd1-4a48-8be8-533cede36c1e'} {{(pid=80144) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 11:05:11.510808 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.notifications [None req-7b1f17ed-3dd1-4a48-8be8-533cede36c1e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '85d8dfe2f72b4f67953a253948697866', 'request_id': 'req-7b1f17ed-3dd1-4a48-8be8-533cede36c1e'} {{(pid=80144) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 11:05:11.522841 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.notifications [None req-7b1f17ed-3dd1-4a48-8be8-533cede36c1e admin admin] The token cache is being invalidate because project 85d8dfe2f72b4f67953a253948697866 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80144) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 07 11:05:11.523161 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.notifications [None req-7b1f17ed-3dd1-4a48-8be8-533cede36c1e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80144) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 11:05:11.523788 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.notifications [None req-7b1f17ed-3dd1-4a48-8be8-533cede36c1e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80144) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 11:05:11.525255 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 1283/2536] 10.222.0.25 () {58 vars in 1233 bytes} [Sun Aug 7 11:05:11 2022] DELETE /identity/v3/projects/85d8dfe2f72b4f67953a253948697866 => generated 0 bytes in 158 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 07 11:05:11.530202 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e28d4b9a-a72f-479f-8f83-e98f15f6acab None None] REQUEST_METHOD: `POST` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 11:05:11.530892 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e28d4b9a-a72f-479f-8f83-e98f15f6acab None None] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 11:05:11.531048 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e28d4b9a-a72f-479f-8f83-e98f15f6acab None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 11:05:11.551152 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.auth.core [None req-e28d4b9a-a72f-479f-8f83-e98f15f6acab None None] MFA Rules not processed for user `bac4e6b76fe64217ac8fa44e71cffe6a`. Rule list: `[]` (Enabled: `True`). {{(pid=80143) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 11:05:11.552364 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-e28d4b9a-a72f-479f-8f83-e98f15f6acab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 11:05:11.586463 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 11:05:11.614172 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1254/2537] 10.222.0.25 () {58 vars in 958 bytes} [Sun Aug 7 11:05:11 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 07 11:05:11.622129 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6649c173-496c-4456-8ef1-db223aeb71a0 None None] REQUEST_METHOD: `POST` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 11:05:11.622417 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6649c173-496c-4456-8ef1-db223aeb71a0 None None] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 11:05:11.622698 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6649c173-496c-4456-8ef1-db223aeb71a0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 11:05:11.643444 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.auth.core [None req-6649c173-496c-4456-8ef1-db223aeb71a0 None None] MFA Rules not processed for user `bac4e6b76fe64217ac8fa44e71cffe6a`. Rule list: `[]` (Enabled: `True`). {{(pid=80144) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 11:05:11.644639 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.fernet_utils [None req-6649c173-496c-4456-8ef1-db223aeb71a0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80144) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 11:05:11.652630 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 1284/2538] 10.222.0.25 () {58 vars in 958 bytes} [Sun Aug 7 11:05:11 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 07 11:05:11.656839 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e28d4b9a-a72f-479f-8f83-e98f15f6acab None None] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 11:05:11.666704 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e28d4b9a-a72f-479f-8f83-e98f15f6acab None None] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 11:05:11.671446 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cddb37e5-dddb-4dfa-9dce-f2d10e119634 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5ac47e3b09f43d19a23a0f457fe9e23', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 11:05:11.672042 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cddb37e5-dddb-4dfa-9dce-f2d10e119634 admin admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 11:05:11.672320 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cddb37e5-dddb-4dfa-9dce-f2d10e119634 admin admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 11:05:11.672608 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cddb37e5-dddb-4dfa-9dce-f2d10e119634 admin admin] PATH_INFO: `/v3/domains` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 11:05:11.674098 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cddb37e5-dddb-4dfa-9dce-f2d10e119634 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80143) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 11:05:11.674402 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cddb37e5-dddb-4dfa-9dce-f2d10e119634 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 11:05:11.676024 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cddb37e5-dddb-4dfa-9dce-f2d10e119634 admin admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 11:05:11.687252 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1255/2539] 10.222.0.25 () {58 vars in 1153 bytes} [Sun Aug 7 11:05:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 11:05:11.692403 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6649c173-496c-4456-8ef1-db223aeb71a0 None None] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 11:05:11.701732 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6649c173-496c-4456-8ef1-db223aeb71a0 None None] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 11:05:11.706426 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3f07eb9-bbfa-4fe0-9ab9-eba24ddb4da8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5ac47e3b09f43d19a23a0f457fe9e23', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 11:05:11.707075 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3f07eb9-bbfa-4fe0-9ab9-eba24ddb4da8 admin admin] REQUEST_METHOD: `POST` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 11:05:11.707353 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3f07eb9-bbfa-4fe0-9ab9-eba24ddb4da8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 11:05:11.707622 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3f07eb9-bbfa-4fe0-9ab9-eba24ddb4da8 admin admin] PATH_INFO: `/v3/projects` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 11:05:11.708274 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3f07eb9-bbfa-4fe0-9ab9-eba24ddb4da8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 11:05:11.710202 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3f07eb9-bbfa-4fe0-9ab9-eba24ddb4da8 admin admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 11:05:11.737171 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 1285/2540] 10.222.0.25 () {60 vars in 1153 bytes} [Sun Aug 7 11:05:11 2022] POST /identity/v3/projects => generated 383 bytes in 45 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 07 11:05:11.741361 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cddb37e5-dddb-4dfa-9dce-f2d10e119634 admin admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 11:05:11.751943 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cddb37e5-dddb-4dfa-9dce-f2d10e119634 admin admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 11:05:11.807688 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1953039f-e5c5-4ae7-9fb3-d5f961bb8d39 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5ac47e3b09f43d19a23a0f457fe9e23', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 11:05:11.808214 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1953039f-e5c5-4ae7-9fb3-d5f961bb8d39 admin admin] REQUEST_METHOD: `POST` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 11:05:11.808495 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1953039f-e5c5-4ae7-9fb3-d5f961bb8d39 admin admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 11:05:11.808772 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1953039f-e5c5-4ae7-9fb3-d5f961bb8d39 admin admin] PATH_INFO: `/v3/users` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 11:05:11.809406 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1953039f-e5c5-4ae7-9fb3-d5f961bb8d39 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 11:05:11.811356 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1953039f-e5c5-4ae7-9fb3-d5f961bb8d39 admin admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 11:05:11.832546 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1256/2541] 10.222.0.25 () {60 vars in 1144 bytes} [Sun Aug 7 11:05:11 2022] POST /identity/v3/users => generated 292 bytes in 92 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 07 11:05:11.836634 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3f07eb9-bbfa-4fe0-9ab9-eba24ddb4da8 admin admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 11:05:11.847006 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3f07eb9-bbfa-4fe0-9ab9-eba24ddb4da8 admin admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 11:05:11.851608 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9992f911-c016-4f43-8d2f-2abecdacc91b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5ac47e3b09f43d19a23a0f457fe9e23', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 11:05:11.852201 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9992f911-c016-4f43-8d2f-2abecdacc91b admin admin] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 11:05:11.852479 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9992f911-c016-4f43-8d2f-2abecdacc91b admin admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 11:05:11.852749 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9992f911-c016-4f43-8d2f-2abecdacc91b admin admin] PATH_INFO: `/v3/roles` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 11:05:11.853382 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9992f911-c016-4f43-8d2f-2abecdacc91b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 11:05:11.855028 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9992f911-c016-4f43-8d2f-2abecdacc91b admin admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 11:05:11.866731 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 1286/2542] 10.222.0.25 () {58 vars in 1122 bytes} [Sun Aug 7 11:05:11 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 07 11:05:11.870032 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1953039f-e5c5-4ae7-9fb3-d5f961bb8d39 admin admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 11:05:11.881267 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1953039f-e5c5-4ae7-9fb3-d5f961bb8d39 admin admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 11:05:11.885829 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0221e4ef-3bb5-49c6-9aed-6548b06c3dc6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5ac47e3b09f43d19a23a0f457fe9e23', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 11:05:11.886447 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0221e4ef-3bb5-49c6-9aed-6548b06c3dc6 admin admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 11:05:11.886724 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0221e4ef-3bb5-49c6-9aed-6548b06c3dc6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 11:05:11.887042 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0221e4ef-3bb5-49c6-9aed-6548b06c3dc6 admin admin] PATH_INFO: `/v3/roles` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 11:05:11.887682 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0221e4ef-3bb5-49c6-9aed-6548b06c3dc6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 11:05:11.889256 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0221e4ef-3bb5-49c6-9aed-6548b06c3dc6 admin admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 11:05:11.901103 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1257/2543] 10.222.0.25 () {58 vars in 1122 bytes} [Sun Aug 7 11:05:11 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 07 11:05:11.905200 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9992f911-c016-4f43-8d2f-2abecdacc91b admin admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 11:05:11.915299 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9992f911-c016-4f43-8d2f-2abecdacc91b admin admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 11:05:11.920004 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20f09c49-a936-4cd3-9e33-4f879c2bd5ad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5ac47e3b09f43d19a23a0f457fe9e23', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 11:05:11.920655 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20f09c49-a936-4cd3-9e33-4f879c2bd5ad admin admin] REQUEST_METHOD: `PUT` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 11:05:11.920935 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20f09c49-a936-4cd3-9e33-4f879c2bd5ad admin admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 11:05:11.921209 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20f09c49-a936-4cd3-9e33-4f879c2bd5ad admin admin] PATH_INFO: `/v3/projects/d9ec425af7ba4745a4840bcbea279336/users/0e9e18b7845f40c2948a3bf25086ff98/roles/b98dfb3b9fee4644af2887a3ac3fd2e6` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 11:05:11.936347 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20f09c49-a936-4cd3-9e33-4f879c2bd5ad admin admin] RBAC: Authorizing `identity:create_grant(project_id=d9ec425af7ba4745a4840bcbea279336, user_id=0e9e18b7845f40c2948a3bf25086ff98, role_id=b98dfb3b9fee4644af2887a3ac3fd2e6)` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 11:05:11.940254 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20f09c49-a936-4cd3-9e33-4f879c2bd5ad admin admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 11:05:11.952620 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 1287/2544] 10.222.0.25 () {60 vars in 1483 bytes} [Sun Aug 7 11:05:11 2022] PUT /identity/v3/projects/d9ec425af7ba4745a4840bcbea279336/users/0e9e18b7845f40c2948a3bf25086ff98/roles/b98dfb3b9fee4644af2887a3ac3fd2e6 => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 07 11:05:11.964688 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11a14852-cde5-41ae-9f5e-4a2a9028fd1e None None] REQUEST_METHOD: `POST` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 11:05:11.964977 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11a14852-cde5-41ae-9f5e-4a2a9028fd1e None None] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 11:05:11.965262 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11a14852-cde5-41ae-9f5e-4a2a9028fd1e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 11:05:11.993642 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.auth.core [None req-11a14852-cde5-41ae-9f5e-4a2a9028fd1e None None] MFA Rules not processed for user `0e9e18b7845f40c2948a3bf25086ff98`. Rule list: `[]` (Enabled: `True`). {{(pid=80143) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 11:05:11.994868 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-11a14852-cde5-41ae-9f5e-4a2a9028fd1e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 11:05:12.057879 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1258/2545] 10.222.0.25 () {58 vars in 958 bytes} [Sun Aug 7 11:05:11 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 07 11:05:12.069000 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20f09c49-a936-4cd3-9e33-4f879c2bd5ad admin admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 11:05:12.070659 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.fernet_utils [None req-20f09c49-a936-4cd3-9e33-4f879c2bd5ad admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80144) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 11:05:12.126976 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20f09c49-a936-4cd3-9e33-4f879c2bd5ad admin admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 11:05:12.130555 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03cb7515-75ca-4c0d-afd2-337382261419 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88dbd6dfd1164994a3e16da553561910', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6de1f3cac02c4bcdb05dd2337176033d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 11:05:12.131170 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03cb7515-75ca-4c0d-afd2-337382261419 service cinder] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 11:05:12.131464 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03cb7515-75ca-4c0d-afd2-337382261419 service cinder] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 11:05:12.131718 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03cb7515-75ca-4c0d-afd2-337382261419 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 11:05:12.139951 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03cb7515-75ca-4c0d-afd2-337382261419 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 11:05:12.141381 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03cb7515-75ca-4c0d-afd2-337382261419 service cinder] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 11:05:12.150635 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 1288/2546] 10.222.0.25 () {60 vars in 1365 bytes} [Sun Aug 7 11:05:12 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 82 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 07 11:05:12.170677 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11a14852-cde5-41ae-9f5e-4a2a9028fd1e None None] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 11:05:12.208976 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 11:05:12.234160 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11a14852-cde5-41ae-9f5e-4a2a9028fd1e None None] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 11:05:12.239451 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3ce6cf4-0734-492a-b162-c10379f8a9c1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5ac47e3b09f43d19a23a0f457fe9e23', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 11:05:12.240058 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3ce6cf4-0734-492a-b162-c10379f8a9c1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 11:05:12.240334 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3ce6cf4-0734-492a-b162-c10379f8a9c1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 11:05:12.240603 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3ce6cf4-0734-492a-b162-c10379f8a9c1 admin admin] PATH_INFO: `/v3/users/0e9e18b7845f40c2948a3bf25086ff98` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 11:05:12.241723 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3ce6cf4-0734-492a-b162-c10379f8a9c1 admin admin] RBAC: Authorizing `identity:delete_user(user_id=0e9e18b7845f40c2948a3bf25086ff98)` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 11:05:12.243640 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3ce6cf4-0734-492a-b162-c10379f8a9c1 admin admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 11:05:12.289193 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-b3ce6cf4-0734-492a-b162-c10379f8a9c1 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0e9e18b7845f40c2948a3bf25086ff98'} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 11:05:12.296010 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-b3ce6cf4-0734-492a-b162-c10379f8a9c1 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0e9e18b7845f40c2948a3bf25086ff98'} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 11:05:12.308584 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-b3ce6cf4-0734-492a-b162-c10379f8a9c1 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0e9e18b7845f40c2948a3bf25086ff98'} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 11:05:12.309204 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-b3ce6cf4-0734-492a-b162-c10379f8a9c1 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0e9e18b7845f40c2948a3bf25086ff98'} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 11:05:12.315993 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-b3ce6cf4-0734-492a-b162-c10379f8a9c1 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0e9e18b7845f40c2948a3bf25086ff98'} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 11:05:12.322005 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1259/2547] 10.222.0.25 () {58 vars in 1224 bytes} [Sun Aug 7 11:05:12 2022] DELETE /identity/v3/users/0e9e18b7845f40c2948a3bf25086ff98 => generated 0 bytes in 152 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 07 11:05:12.326686 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36782211-a2d8-426d-934f-0c668020da1f None None] REQUEST_METHOD: `POST` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 11:05:12.327023 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36782211-a2d8-426d-934f-0c668020da1f None None] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 11:05:12.327302 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36782211-a2d8-426d-934f-0c668020da1f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 11:05:12.348302 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.auth.core [None req-36782211-a2d8-426d-934f-0c668020da1f None None] MFA Rules not processed for user `bac4e6b76fe64217ac8fa44e71cffe6a`. Rule list: `[]` (Enabled: `True`). {{(pid=80144) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 11:05:12.349489 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.fernet_utils [None req-36782211-a2d8-426d-934f-0c668020da1f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80144) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 11:05:12.361848 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 11:05:12.412737 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 1289/2548] 10.222.0.25 () {58 vars in 958 bytes} [Sun Aug 7 11:05:12 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 07 11:05:12.423798 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3ce6cf4-0734-492a-b162-c10379f8a9c1 admin admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 11:05:12.425473 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-b3ce6cf4-0734-492a-b162-c10379f8a9c1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 11:05:12.484210 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3ce6cf4-0734-492a-b162-c10379f8a9c1 admin admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 11:05:12.488131 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81a0b0fa-3c29-4c10-9732-d3e14f993818 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be1283c9bf47424c9aa3ad27cab03027', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6de1f3cac02c4bcdb05dd2337176033d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 11:05:12.488731 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81a0b0fa-3c29-4c10-9732-d3e14f993818 service neutron] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 11:05:12.489009 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81a0b0fa-3c29-4c10-9732-d3e14f993818 service neutron] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 11:05:12.489282 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81a0b0fa-3c29-4c10-9732-d3e14f993818 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 11:05:12.498043 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81a0b0fa-3c29-4c10-9732-d3e14f993818 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 11:05:12.499486 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81a0b0fa-3c29-4c10-9732-d3e14f993818 service neutron] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 11:05:12.508748 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1260/2549] 10.222.0.25 () {60 vars in 1365 bytes} [Sun Aug 7 11:05:12 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 07 11:05:12.781458 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36782211-a2d8-426d-934f-0c668020da1f None None] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 11:05:12.783173 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.fernet_utils [None req-36782211-a2d8-426d-934f-0c668020da1f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80144) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 11:05:12.797313 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36782211-a2d8-426d-934f-0c668020da1f None None] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 11:05:12.802541 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-279a2831-0016-4bc2-8a6e-adad4741fff6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5ac47e3b09f43d19a23a0f457fe9e23', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 11:05:12.803454 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-279a2831-0016-4bc2-8a6e-adad4741fff6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 11:05:12.803878 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-279a2831-0016-4bc2-8a6e-adad4741fff6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 11:05:12.804367 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-279a2831-0016-4bc2-8a6e-adad4741fff6 admin admin] PATH_INFO: `/v3/projects/d9ec425af7ba4745a4840bcbea279336` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 11:05:12.805852 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-279a2831-0016-4bc2-8a6e-adad4741fff6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d9ec425af7ba4745a4840bcbea279336)` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 11:05:12.808006 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-279a2831-0016-4bc2-8a6e-adad4741fff6 admin admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 11:05:12.854726 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.notifications [None req-279a2831-0016-4bc2-8a6e-adad4741fff6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd9ec425af7ba4745a4840bcbea279336', 'request_id': 'req-279a2831-0016-4bc2-8a6e-adad4741fff6'} {{(pid=80144) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 11:05:12.860550 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.notifications [None req-279a2831-0016-4bc2-8a6e-adad4741fff6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd9ec425af7ba4745a4840bcbea279336', 'request_id': 'req-279a2831-0016-4bc2-8a6e-adad4741fff6'} {{(pid=80144) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 11:05:12.867759 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.notifications [None req-279a2831-0016-4bc2-8a6e-adad4741fff6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd9ec425af7ba4745a4840bcbea279336', 'request_id': 'req-279a2831-0016-4bc2-8a6e-adad4741fff6'} {{(pid=80144) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 11:05:12.879261 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.notifications [None req-279a2831-0016-4bc2-8a6e-adad4741fff6 admin admin] The token cache is being invalidate because project d9ec425af7ba4745a4840bcbea279336 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80144) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 07 11:05:12.879548 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.notifications [None req-279a2831-0016-4bc2-8a6e-adad4741fff6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80144) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 11:05:12.880171 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.notifications [None req-279a2831-0016-4bc2-8a6e-adad4741fff6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80144) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 11:05:12.881627 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 1290/2550] 10.222.0.25 () {58 vars in 1233 bytes} [Sun Aug 7 11:05:12 2022] DELETE /identity/v3/projects/d9ec425af7ba4745a4840bcbea279336 => generated 0 bytes in 101 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 07 11:05:12.886440 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5c59fd8-1aff-4524-aaa1-782da8fef221 None None] REQUEST_METHOD: `POST` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 11:05:12.886955 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5c59fd8-1aff-4524-aaa1-782da8fef221 None None] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 11:05:12.887081 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5c59fd8-1aff-4524-aaa1-782da8fef221 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 11:05:12.906979 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.auth.core [None req-c5c59fd8-1aff-4524-aaa1-782da8fef221 None None] MFA Rules not processed for user `bac4e6b76fe64217ac8fa44e71cffe6a`. Rule list: `[]` (Enabled: `True`). {{(pid=80143) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 11:05:12.908627 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-c5c59fd8-1aff-4524-aaa1-782da8fef221 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 11:05:12.958745 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 11:05:12.969588 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1261/2551] 10.222.0.25 () {58 vars in 958 bytes} [Sun Aug 7 11:05:12 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 07 11:05:12.977358 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29ee7a35-5e4b-4d51-8064-9b7bd69600ca None None] REQUEST_METHOD: `POST` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 11:05:12.977643 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29ee7a35-5e4b-4d51-8064-9b7bd69600ca None None] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 11:05:12.977917 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29ee7a35-5e4b-4d51-8064-9b7bd69600ca None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 11:05:12.997768 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.auth.core [None req-29ee7a35-5e4b-4d51-8064-9b7bd69600ca None None] MFA Rules not processed for user `bac4e6b76fe64217ac8fa44e71cffe6a`. Rule list: `[]` (Enabled: `True`). {{(pid=80144) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 11:05:12.998993 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.fernet_utils [None req-29ee7a35-5e4b-4d51-8064-9b7bd69600ca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80144) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 11:05:13.007394 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 1291/2552] 10.222.0.25 () {58 vars in 958 bytes} [Sun Aug 7 11:05:12 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 07 11:05:13.011514 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5c59fd8-1aff-4524-aaa1-782da8fef221 None None] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 11:05:13.021657 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5c59fd8-1aff-4524-aaa1-782da8fef221 None None] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 11:05:13.026065 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fd57e88-e11b-4d3b-97f2-aab6b721944e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5ac47e3b09f43d19a23a0f457fe9e23', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 11:05:13.026652 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fd57e88-e11b-4d3b-97f2-aab6b721944e admin admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 11:05:13.026951 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fd57e88-e11b-4d3b-97f2-aab6b721944e admin admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 11:05:13.027319 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fd57e88-e11b-4d3b-97f2-aab6b721944e admin admin] PATH_INFO: `/v3/domains` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 11:05:13.027959 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fd57e88-e11b-4d3b-97f2-aab6b721944e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80143) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 11:05:13.028258 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fd57e88-e11b-4d3b-97f2-aab6b721944e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 11:05:13.029834 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fd57e88-e11b-4d3b-97f2-aab6b721944e admin admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 11:05:13.041213 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1262/2553] 10.222.0.25 () {58 vars in 1153 bytes} [Sun Aug 7 11:05:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 11:05:13.045402 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29ee7a35-5e4b-4d51-8064-9b7bd69600ca None None] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 11:05:13.054474 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29ee7a35-5e4b-4d51-8064-9b7bd69600ca None None] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 11:05:13.059169 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c4231eb-917f-42db-8727-efb9ddc49c51 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5ac47e3b09f43d19a23a0f457fe9e23', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 11:05:13.059863 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c4231eb-917f-42db-8727-efb9ddc49c51 admin admin] REQUEST_METHOD: `POST` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 11:05:13.060143 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c4231eb-917f-42db-8727-efb9ddc49c51 admin admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 11:05:13.060411 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c4231eb-917f-42db-8727-efb9ddc49c51 admin admin] PATH_INFO: `/v3/projects` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 11:05:13.061029 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c4231eb-917f-42db-8727-efb9ddc49c51 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 11:05:13.062889 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c4231eb-917f-42db-8727-efb9ddc49c51 admin admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 11:05:13.090505 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 1292/2554] 10.222.0.25 () {60 vars in 1153 bytes} [Sun Aug 7 11:05:13 2022] POST /identity/v3/projects => generated 385 bytes in 45 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 07 11:05:13.094490 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fd57e88-e11b-4d3b-97f2-aab6b721944e admin admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 11:05:13.104357 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fd57e88-e11b-4d3b-97f2-aab6b721944e admin admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 11:05:13.159137 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8ba8b73-15f6-41d9-bec6-e1a425d506a6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5ac47e3b09f43d19a23a0f457fe9e23', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 11:05:13.159661 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8ba8b73-15f6-41d9-bec6-e1a425d506a6 admin admin] REQUEST_METHOD: `POST` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 11:05:13.159938 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8ba8b73-15f6-41d9-bec6-e1a425d506a6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 11:05:13.160209 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8ba8b73-15f6-41d9-bec6-e1a425d506a6 admin admin] PATH_INFO: `/v3/users` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 11:05:13.160852 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8ba8b73-15f6-41d9-bec6-e1a425d506a6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 11:05:13.162745 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8ba8b73-15f6-41d9-bec6-e1a425d506a6 admin admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 11:05:13.183633 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1263/2555] 10.222.0.25 () {60 vars in 1144 bytes} [Sun Aug 7 11:05:13 2022] POST /identity/v3/users => generated 293 bytes in 90 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 07 11:05:13.187683 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c4231eb-917f-42db-8727-efb9ddc49c51 admin admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 11:05:13.198560 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c4231eb-917f-42db-8727-efb9ddc49c51 admin admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 11:05:13.203198 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee6d5262-e1c8-4f8d-bf56-6dc485e77201 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5ac47e3b09f43d19a23a0f457fe9e23', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 11:05:13.203797 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee6d5262-e1c8-4f8d-bf56-6dc485e77201 admin admin] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 11:05:13.204077 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee6d5262-e1c8-4f8d-bf56-6dc485e77201 admin admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 11:05:13.204399 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee6d5262-e1c8-4f8d-bf56-6dc485e77201 admin admin] PATH_INFO: `/v3/roles` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 11:05:13.205320 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee6d5262-e1c8-4f8d-bf56-6dc485e77201 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 11:05:13.206953 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee6d5262-e1c8-4f8d-bf56-6dc485e77201 admin admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 11:05:13.217989 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 1293/2556] 10.222.0.25 () {58 vars in 1122 bytes} [Sun Aug 7 11:05:13 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 07 11:05:13.222342 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8ba8b73-15f6-41d9-bec6-e1a425d506a6 admin admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 11:05:13.232119 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8ba8b73-15f6-41d9-bec6-e1a425d506a6 admin admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 11:05:13.237688 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3fa6c00-f1be-44a2-973c-e2057dc89072 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5ac47e3b09f43d19a23a0f457fe9e23', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 11:05:13.238280 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3fa6c00-f1be-44a2-973c-e2057dc89072 admin admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 11:05:13.238554 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3fa6c00-f1be-44a2-973c-e2057dc89072 admin admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 11:05:13.238820 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3fa6c00-f1be-44a2-973c-e2057dc89072 admin admin] PATH_INFO: `/v3/roles` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 11:05:13.239488 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3fa6c00-f1be-44a2-973c-e2057dc89072 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 11:05:13.241056 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3fa6c00-f1be-44a2-973c-e2057dc89072 admin admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 11:05:13.252473 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1264/2557] 10.222.0.25 () {58 vars in 1122 bytes} [Sun Aug 7 11:05:13 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 07 11:05:13.256528 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee6d5262-e1c8-4f8d-bf56-6dc485e77201 admin admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 11:05:13.266260 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee6d5262-e1c8-4f8d-bf56-6dc485e77201 admin admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 11:05:13.272473 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d4f13a7-ab2e-45de-b4e7-c5fad83b7573 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5ac47e3b09f43d19a23a0f457fe9e23', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 11:05:13.273115 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d4f13a7-ab2e-45de-b4e7-c5fad83b7573 admin admin] REQUEST_METHOD: `PUT` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 11:05:13.273391 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d4f13a7-ab2e-45de-b4e7-c5fad83b7573 admin admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 11:05:13.273660 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d4f13a7-ab2e-45de-b4e7-c5fad83b7573 admin admin] PATH_INFO: `/v3/projects/6568c6fbb4ce4b4cb92e948c587dd0ba/users/108545382db042c78f84f13417791e2e/roles/b98dfb3b9fee4644af2887a3ac3fd2e6` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 11:05:13.287698 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d4f13a7-ab2e-45de-b4e7-c5fad83b7573 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6568c6fbb4ce4b4cb92e948c587dd0ba, user_id=108545382db042c78f84f13417791e2e, role_id=b98dfb3b9fee4644af2887a3ac3fd2e6)` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 11:05:13.290296 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d4f13a7-ab2e-45de-b4e7-c5fad83b7573 admin admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 11:05:13.303254 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 1294/2558] 10.222.0.25 () {60 vars in 1483 bytes} [Sun Aug 7 11:05:13 2022] PUT /identity/v3/projects/6568c6fbb4ce4b4cb92e948c587dd0ba/users/108545382db042c78f84f13417791e2e/roles/b98dfb3b9fee4644af2887a3ac3fd2e6 => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 07 11:05:13.315068 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-164e3e3a-28d8-495d-9fef-975c767ae3f4 None None] REQUEST_METHOD: `POST` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 11:05:13.315355 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-164e3e3a-28d8-495d-9fef-975c767ae3f4 None None] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 11:05:13.315623 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-164e3e3a-28d8-495d-9fef-975c767ae3f4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 11:05:13.342295 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.auth.core [None req-164e3e3a-28d8-495d-9fef-975c767ae3f4 None None] MFA Rules not processed for user `108545382db042c78f84f13417791e2e`. Rule list: `[]` (Enabled: `True`). {{(pid=80143) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 11:05:13.343508 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-164e3e3a-28d8-495d-9fef-975c767ae3f4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 11:05:13.408157 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1265/2559] 10.222.0.25 () {58 vars in 958 bytes} [Sun Aug 7 11:05:13 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 07 11:05:13.419200 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d4f13a7-ab2e-45de-b4e7-c5fad83b7573 admin admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 11:05:13.424073 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.fernet_utils [None req-5d4f13a7-ab2e-45de-b4e7-c5fad83b7573 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80144) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 11:05:13.465976 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 11:05:13.483089 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d4f13a7-ab2e-45de-b4e7-c5fad83b7573 admin admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 11:05:13.487693 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8682ef0-6e0f-40d9-83af-4e8372270d8c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88dbd6dfd1164994a3e16da553561910', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6de1f3cac02c4bcdb05dd2337176033d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 11:05:13.488295 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8682ef0-6e0f-40d9-83af-4e8372270d8c service cinder] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 11:05:13.488572 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8682ef0-6e0f-40d9-83af-4e8372270d8c service cinder] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 11:05:13.488842 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8682ef0-6e0f-40d9-83af-4e8372270d8c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 11:05:13.496784 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8682ef0-6e0f-40d9-83af-4e8372270d8c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 11:05:13.498189 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8682ef0-6e0f-40d9-83af-4e8372270d8c service cinder] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 11:05:13.507685 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 1295/2560] 10.222.0.25 () {60 vars in 1365 bytes} [Sun Aug 7 11:05:13 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 07 11:05:22.174871 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-164e3e3a-28d8-495d-9fef-975c767ae3f4 None None] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 11:05:22.177040 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-164e3e3a-28d8-495d-9fef-975c767ae3f4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 11:05:22.243854 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 11:05:22.272013 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-164e3e3a-28d8-495d-9fef-975c767ae3f4 None None] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 11:05:22.277758 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcf80112-2473-4c3d-9af3-d9fcebcd1754 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31d16d1b96104b4eb690398432b9829e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6de1f3cac02c4bcdb05dd2337176033d', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'reader', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 11:05:22.278452 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcf80112-2473-4c3d-9af3-d9fcebcd1754 service nova] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 11:05:22.278736 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcf80112-2473-4c3d-9af3-d9fcebcd1754 service nova] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 11:05:22.279046 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcf80112-2473-4c3d-9af3-d9fcebcd1754 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 11:05:22.280880 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-bcf80112-2473-4c3d-9af3-d9fcebcd1754 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 11:05:22.295715 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcf80112-2473-4c3d-9af3-d9fcebcd1754 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 11:05:22.297349 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcf80112-2473-4c3d-9af3-d9fcebcd1754 service nova] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 11:05:22.322748 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1266/2561] 10.222.0.25 () {60 vars in 1365 bytes} [Sun Aug 7 11:05:22 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 149 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 07 11:05:22.943794 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8682ef0-6e0f-40d9-83af-4e8372270d8c service cinder] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 11:05:23.014708 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8682ef0-6e0f-40d9-83af-4e8372270d8c service cinder] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 11:05:23.019517 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf0dc3c8-5a16-4635-966f-f0ceb0f72160 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5ac47e3b09f43d19a23a0f457fe9e23', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 11:05:23.020213 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf0dc3c8-5a16-4635-966f-f0ceb0f72160 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 11:05:23.020495 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf0dc3c8-5a16-4635-966f-f0ceb0f72160 admin admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 11:05:23.020766 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf0dc3c8-5a16-4635-966f-f0ceb0f72160 admin admin] PATH_INFO: `/v3/users/108545382db042c78f84f13417791e2e` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 11:05:23.021881 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf0dc3c8-5a16-4635-966f-f0ceb0f72160 admin admin] RBAC: Authorizing `identity:delete_user(user_id=108545382db042c78f84f13417791e2e)` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 11:05:23.024868 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf0dc3c8-5a16-4635-966f-f0ceb0f72160 admin admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 11:05:23.098147 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.notifications [None req-cf0dc3c8-5a16-4635-966f-f0ceb0f72160 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '108545382db042c78f84f13417791e2e'} {{(pid=80144) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 11:05:23.106553 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.notifications [None req-cf0dc3c8-5a16-4635-966f-f0ceb0f72160 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '108545382db042c78f84f13417791e2e'} {{(pid=80144) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 11:05:23.118139 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.notifications [None req-cf0dc3c8-5a16-4635-966f-f0ceb0f72160 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '108545382db042c78f84f13417791e2e'} {{(pid=80144) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 11:05:23.118807 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.notifications [None req-cf0dc3c8-5a16-4635-966f-f0ceb0f72160 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '108545382db042c78f84f13417791e2e'} {{(pid=80144) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 11:05:23.120069 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.notifications [None req-cf0dc3c8-5a16-4635-966f-f0ceb0f72160 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '108545382db042c78f84f13417791e2e'} {{(pid=80144) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 11:05:23.143036 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 1296/2562] 10.222.0.25 () {58 vars in 1224 bytes} [Sun Aug 7 11:05:22 2022] DELETE /identity/v3/users/108545382db042c78f84f13417791e2e => generated 0 bytes in 197 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 07 11:05:23.144379 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e04b4fd-ae5c-4f73-ba76-99fb81770446 None None] REQUEST_METHOD: `POST` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 11:05:23.144665 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e04b4fd-ae5c-4f73-ba76-99fb81770446 None None] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 11:05:23.144939 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e04b4fd-ae5c-4f73-ba76-99fb81770446 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 11:05:23.168824 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.auth.core [None req-4e04b4fd-ae5c-4f73-ba76-99fb81770446 None None] MFA Rules not processed for user `bac4e6b76fe64217ac8fa44e71cffe6a`. Rule list: `[]` (Enabled: `True`). {{(pid=80143) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 11:05:23.170361 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-4e04b4fd-ae5c-4f73-ba76-99fb81770446 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 11:05:23.238630 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1267/2563] 10.222.0.25 () {58 vars in 958 bytes} [Sun Aug 7 11:05:23 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 07 11:05:23.248789 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf0dc3c8-5a16-4635-966f-f0ceb0f72160 admin admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 11:05:23.250328 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.fernet_utils [None req-cf0dc3c8-5a16-4635-966f-f0ceb0f72160 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80144) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 11:05:23.321486 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf0dc3c8-5a16-4635-966f-f0ceb0f72160 admin admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 11:05:23.325322 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a7a091a-a844-4e99-807b-0f6dd3eee4da service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be1283c9bf47424c9aa3ad27cab03027', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6de1f3cac02c4bcdb05dd2337176033d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 11:05:23.326016 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a7a091a-a844-4e99-807b-0f6dd3eee4da service neutron] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 11:05:23.326301 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a7a091a-a844-4e99-807b-0f6dd3eee4da service neutron] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 11:05:23.326579 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a7a091a-a844-4e99-807b-0f6dd3eee4da service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 11:05:23.337823 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a7a091a-a844-4e99-807b-0f6dd3eee4da service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 11:05:23.339391 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a7a091a-a844-4e99-807b-0f6dd3eee4da service neutron] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 11:05:23.350771 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 1297/2564] 10.222.0.25 () {60 vars in 1365 bytes} [Sun Aug 7 11:05:23 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 102 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 07 11:05:23.635829 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e04b4fd-ae5c-4f73-ba76-99fb81770446 None None] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 11:05:23.637419 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-4e04b4fd-ae5c-4f73-ba76-99fb81770446 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 11:05:23.650540 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e04b4fd-ae5c-4f73-ba76-99fb81770446 None None] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 11:05:23.655399 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a69e3904-ba6b-4b62-abe4-ac5b09770524 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5ac47e3b09f43d19a23a0f457fe9e23', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 11:05:23.656048 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a69e3904-ba6b-4b62-abe4-ac5b09770524 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 11:05:23.656332 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a69e3904-ba6b-4b62-abe4-ac5b09770524 admin admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 11:05:23.656607 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a69e3904-ba6b-4b62-abe4-ac5b09770524 admin admin] PATH_INFO: `/v3/projects/6568c6fbb4ce4b4cb92e948c587dd0ba` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 11:05:23.657742 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a69e3904-ba6b-4b62-abe4-ac5b09770524 admin admin] RBAC: Authorizing `identity:delete_project(project_id=6568c6fbb4ce4b4cb92e948c587dd0ba)` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 11:05:23.659742 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a69e3904-ba6b-4b62-abe4-ac5b09770524 admin admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 11:05:23.710645 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-a69e3904-ba6b-4b62-abe4-ac5b09770524 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6568c6fbb4ce4b4cb92e948c587dd0ba', 'request_id': 'req-a69e3904-ba6b-4b62-abe4-ac5b09770524'} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 11:05:23.716637 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-a69e3904-ba6b-4b62-abe4-ac5b09770524 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6568c6fbb4ce4b4cb92e948c587dd0ba', 'request_id': 'req-a69e3904-ba6b-4b62-abe4-ac5b09770524'} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 11:05:23.725691 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-a69e3904-ba6b-4b62-abe4-ac5b09770524 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6568c6fbb4ce4b4cb92e948c587dd0ba', 'request_id': 'req-a69e3904-ba6b-4b62-abe4-ac5b09770524'} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 11:05:23.745734 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-a69e3904-ba6b-4b62-abe4-ac5b09770524 admin admin] The token cache is being invalidate because project 6568c6fbb4ce4b4cb92e948c587dd0ba was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80143) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 07 11:05:23.746030 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-a69e3904-ba6b-4b62-abe4-ac5b09770524 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 11:05:23.746701 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-a69e3904-ba6b-4b62-abe4-ac5b09770524 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 11:05:23.748941 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1268/2565] 10.222.0.25 () {58 vars in 1233 bytes} [Sun Aug 7 11:05:23 2022] DELETE /identity/v3/projects/6568c6fbb4ce4b4cb92e948c587dd0ba => generated 0 bytes in 113 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 07 11:05:23.756046 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54bde671-a2f0-44b5-88aa-30f6435682a4 None None] REQUEST_METHOD: `POST` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 11:05:23.758883 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54bde671-a2f0-44b5-88aa-30f6435682a4 None None] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 11:05:23.759202 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54bde671-a2f0-44b5-88aa-30f6435682a4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 11:05:23.767136 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 11:05:23.782357 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.auth.core [None req-54bde671-a2f0-44b5-88aa-30f6435682a4 None None] MFA Rules not processed for user `bac4e6b76fe64217ac8fa44e71cffe6a`. Rule list: `[]` (Enabled: `True`). {{(pid=80144) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 11:05:23.783630 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.fernet_utils [None req-54bde671-a2f0-44b5-88aa-30f6435682a4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80144) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 11:05:23.849166 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 1298/2566] 10.222.0.25 () {58 vars in 958 bytes} [Sun Aug 7 11:05:23 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 07 11:05:23.856192 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-187c97e0-61ba-4317-8d82-a46569282460 None None] REQUEST_METHOD: `POST` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 11:05:23.856550 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-187c97e0-61ba-4317-8d82-a46569282460 None None] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 11:05:23.856834 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-187c97e0-61ba-4317-8d82-a46569282460 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 11:05:23.877046 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.auth.core [None req-187c97e0-61ba-4317-8d82-a46569282460 None None] MFA Rules not processed for user `bac4e6b76fe64217ac8fa44e71cffe6a`. Rule list: `[]` (Enabled: `True`). {{(pid=80143) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 11:05:23.878375 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-187c97e0-61ba-4317-8d82-a46569282460 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 11:05:23.887680 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1269/2567] 10.222.0.25 () {58 vars in 958 bytes} [Sun Aug 7 11:05:23 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 07 11:05:23.892208 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54bde671-a2f0-44b5-88aa-30f6435682a4 None None] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 11:05:23.903074 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54bde671-a2f0-44b5-88aa-30f6435682a4 None None] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 11:05:23.907692 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f91db93-e378-4899-a753-af2b82efd2dd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5ac47e3b09f43d19a23a0f457fe9e23', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 11:05:23.908298 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f91db93-e378-4899-a753-af2b82efd2dd admin admin] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 11:05:23.909230 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f91db93-e378-4899-a753-af2b82efd2dd admin admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 11:05:23.909698 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f91db93-e378-4899-a753-af2b82efd2dd admin admin] PATH_INFO: `/v3/domains` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 11:05:23.910389 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f91db93-e378-4899-a753-af2b82efd2dd admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80144) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 11:05:23.910713 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f91db93-e378-4899-a753-af2b82efd2dd admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 11:05:23.912376 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f91db93-e378-4899-a753-af2b82efd2dd admin admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 11:05:23.924924 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 1299/2568] 10.222.0.25 () {58 vars in 1153 bytes} [Sun Aug 7 11:05:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 11:05:23.929767 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-187c97e0-61ba-4317-8d82-a46569282460 None None] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 11:05:23.940703 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-187c97e0-61ba-4317-8d82-a46569282460 None None] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 11:05:23.947122 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4397d8bc-cccf-4925-bd88-0301ab255d9c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5ac47e3b09f43d19a23a0f457fe9e23', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 11:05:23.947122 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4397d8bc-cccf-4925-bd88-0301ab255d9c admin admin] REQUEST_METHOD: `POST` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 11:05:23.947122 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4397d8bc-cccf-4925-bd88-0301ab255d9c admin admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 11:05:23.947122 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4397d8bc-cccf-4925-bd88-0301ab255d9c admin admin] PATH_INFO: `/v3/projects` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 11:05:23.947460 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4397d8bc-cccf-4925-bd88-0301ab255d9c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 11:05:23.949150 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4397d8bc-cccf-4925-bd88-0301ab255d9c admin admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 11:05:23.977283 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1270/2569] 10.222.0.25 () {60 vars in 1153 bytes} [Sun Aug 7 11:05:23 2022] POST /identity/v3/projects => generated 377 bytes in 48 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 07 11:05:23.982509 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f91db93-e378-4899-a753-af2b82efd2dd admin admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 11:05:23.994135 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f91db93-e378-4899-a753-af2b82efd2dd admin admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 11:05:24.052122 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a96577d-87d8-4441-a625-87a7957074e3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5ac47e3b09f43d19a23a0f457fe9e23', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 11:05:24.052730 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a96577d-87d8-4441-a625-87a7957074e3 admin admin] REQUEST_METHOD: `POST` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 11:05:24.053041 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a96577d-87d8-4441-a625-87a7957074e3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 11:05:24.053311 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a96577d-87d8-4441-a625-87a7957074e3 admin admin] PATH_INFO: `/v3/users` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 11:05:24.053930 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a96577d-87d8-4441-a625-87a7957074e3 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 11:05:24.055877 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a96577d-87d8-4441-a625-87a7957074e3 admin admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 11:05:24.077975 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 1300/2570] 10.222.0.25 () {60 vars in 1144 bytes} [Sun Aug 7 11:05:23 2022] POST /identity/v3/users => generated 289 bytes in 96 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 07 11:05:24.082624 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4397d8bc-cccf-4925-bd88-0301ab255d9c admin admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 11:05:24.094117 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4397d8bc-cccf-4925-bd88-0301ab255d9c admin admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 11:05:24.099558 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e8d640c-6807-450d-9c21-13b31db5fb0c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5ac47e3b09f43d19a23a0f457fe9e23', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 11:05:24.100169 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e8d640c-6807-450d-9c21-13b31db5fb0c admin admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 11:05:24.100469 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e8d640c-6807-450d-9c21-13b31db5fb0c admin admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 11:05:24.100718 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e8d640c-6807-450d-9c21-13b31db5fb0c admin admin] PATH_INFO: `/v3/roles` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 11:05:24.101361 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e8d640c-6807-450d-9c21-13b31db5fb0c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 11:05:24.103033 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e8d640c-6807-450d-9c21-13b31db5fb0c admin admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 11:05:24.113989 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1271/2571] 10.222.0.25 () {58 vars in 1122 bytes} [Sun Aug 7 11:05:24 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 07 11:05:24.118166 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a96577d-87d8-4441-a625-87a7957074e3 admin admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 11:05:24.128390 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a96577d-87d8-4441-a625-87a7957074e3 admin admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 11:05:24.134226 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c63ea67-2b80-4140-8b03-7dc1ea156063 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5ac47e3b09f43d19a23a0f457fe9e23', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 11:05:24.134870 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c63ea67-2b80-4140-8b03-7dc1ea156063 admin admin] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 11:05:24.135194 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c63ea67-2b80-4140-8b03-7dc1ea156063 admin admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 11:05:24.135471 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c63ea67-2b80-4140-8b03-7dc1ea156063 admin admin] PATH_INFO: `/v3/roles` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 11:05:24.136141 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c63ea67-2b80-4140-8b03-7dc1ea156063 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 11:05:24.137777 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c63ea67-2b80-4140-8b03-7dc1ea156063 admin admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 11:05:24.149207 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 1301/2572] 10.222.0.25 () {58 vars in 1122 bytes} [Sun Aug 7 11:05:24 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 07 11:05:24.153520 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e8d640c-6807-450d-9c21-13b31db5fb0c admin admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 11:05:24.165540 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e8d640c-6807-450d-9c21-13b31db5fb0c admin admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 11:05:24.170253 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29f7f5ef-c4c4-4d1d-ae9b-91df42aa189a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5ac47e3b09f43d19a23a0f457fe9e23', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 11:05:24.170903 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29f7f5ef-c4c4-4d1d-ae9b-91df42aa189a admin admin] REQUEST_METHOD: `PUT` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 11:05:24.171213 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29f7f5ef-c4c4-4d1d-ae9b-91df42aa189a admin admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 11:05:24.171494 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29f7f5ef-c4c4-4d1d-ae9b-91df42aa189a admin admin] PATH_INFO: `/v3/projects/3c41ff204b0049a29bf079b3d8725b83/users/54ce608a2b2d467fb0b3ac19abd41acf/roles/b98dfb3b9fee4644af2887a3ac3fd2e6` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 11:05:24.188010 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29f7f5ef-c4c4-4d1d-ae9b-91df42aa189a admin admin] RBAC: Authorizing `identity:create_grant(project_id=3c41ff204b0049a29bf079b3d8725b83, user_id=54ce608a2b2d467fb0b3ac19abd41acf, role_id=b98dfb3b9fee4644af2887a3ac3fd2e6)` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 11:05:24.190834 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29f7f5ef-c4c4-4d1d-ae9b-91df42aa189a admin admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 11:05:24.198586 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 11:05:24.205032 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1272/2573] 10.222.0.25 () {60 vars in 1483 bytes} [Sun Aug 7 11:05:24 2022] PUT /identity/v3/projects/3c41ff204b0049a29bf079b3d8725b83/users/54ce608a2b2d467fb0b3ac19abd41acf/roles/b98dfb3b9fee4644af2887a3ac3fd2e6 => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 07 11:05:24.215359 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fc7746a-42d9-48c8-a82f-63b13796182a None None] REQUEST_METHOD: `POST` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 11:05:24.215648 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fc7746a-42d9-48c8-a82f-63b13796182a None None] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 11:05:24.215925 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fc7746a-42d9-48c8-a82f-63b13796182a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 11:05:24.244685 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.auth.core [None req-7fc7746a-42d9-48c8-a82f-63b13796182a None None] MFA Rules not processed for user `54ce608a2b2d467fb0b3ac19abd41acf`. Rule list: `[]` (Enabled: `True`). {{(pid=80144) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 11:05:24.245865 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.fernet_utils [None req-7fc7746a-42d9-48c8-a82f-63b13796182a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80144) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 11:05:24.303793 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 11:05:24.312236 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 1302/2574] 10.222.0.25 () {58 vars in 958 bytes} [Sun Aug 7 11:05:24 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 07 11:05:24.321232 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29f7f5ef-c4c4-4d1d-ae9b-91df42aa189a admin admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 11:05:24.322947 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-29f7f5ef-c4c4-4d1d-ae9b-91df42aa189a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 11:05:24.380669 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29f7f5ef-c4c4-4d1d-ae9b-91df42aa189a admin admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 11:05:24.384474 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1db0e3ee-8008-4e92-aef2-e9c994cdc0fd service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88dbd6dfd1164994a3e16da553561910', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6de1f3cac02c4bcdb05dd2337176033d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 11:05:24.385084 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1db0e3ee-8008-4e92-aef2-e9c994cdc0fd service cinder] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 11:05:24.385367 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1db0e3ee-8008-4e92-aef2-e9c994cdc0fd service cinder] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 11:05:24.385643 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1db0e3ee-8008-4e92-aef2-e9c994cdc0fd service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 11:05:24.394848 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1db0e3ee-8008-4e92-aef2-e9c994cdc0fd service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 11:05:24.396298 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1db0e3ee-8008-4e92-aef2-e9c994cdc0fd service cinder] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 11:05:24.405156 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1273/2575] 10.222.0.25 () {60 vars in 1365 bytes} [Sun Aug 7 11:05:24 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 07 11:05:26.257715 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fc7746a-42d9-48c8-a82f-63b13796182a None None] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 11:05:26.259156 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.fernet_utils [None req-7fc7746a-42d9-48c8-a82f-63b13796182a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80144) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 11:05:26.314334 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fc7746a-42d9-48c8-a82f-63b13796182a None None] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 11:05:26.318450 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-628d9bac-e106-457e-b549-7f80e14503e5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be1283c9bf47424c9aa3ad27cab03027', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6de1f3cac02c4bcdb05dd2337176033d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 11:05:26.319082 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-628d9bac-e106-457e-b549-7f80e14503e5 service neutron] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 11:05:26.319363 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-628d9bac-e106-457e-b549-7f80e14503e5 service neutron] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 11:05:26.319641 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-628d9bac-e106-457e-b549-7f80e14503e5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 11:05:26.321329 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.fernet_utils [None req-628d9bac-e106-457e-b549-7f80e14503e5 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80144) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 11:05:26.383633 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-628d9bac-e106-457e-b549-7f80e14503e5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 11:05:26.385269 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-628d9bac-e106-457e-b549-7f80e14503e5 service neutron] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 11:05:26.395522 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 1303/2576] 10.222.0.25 () {60 vars in 1365 bytes} [Sun Aug 7 11:05:26 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 138 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 07 11:05:29.262628 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1db0e3ee-8008-4e92-aef2-e9c994cdc0fd service cinder] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 11:05:29.264175 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-1db0e3ee-8008-4e92-aef2-e9c994cdc0fd service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 11:05:29.282688 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1db0e3ee-8008-4e92-aef2-e9c994cdc0fd service cinder] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 11:05:29.288207 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0b47d70-44de-4722-a633-9dd6e3fb6a22 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5ac47e3b09f43d19a23a0f457fe9e23', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 11:05:29.288895 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0b47d70-44de-4722-a633-9dd6e3fb6a22 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 11:05:29.289180 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0b47d70-44de-4722-a633-9dd6e3fb6a22 admin admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 11:05:29.289457 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0b47d70-44de-4722-a633-9dd6e3fb6a22 admin admin] PATH_INFO: `/v3/users/2b757e3f47e44b9ba3645d12b20f7502` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 11:05:29.290607 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0b47d70-44de-4722-a633-9dd6e3fb6a22 admin admin] RBAC: Authorizing `identity:delete_user(user_id=2b757e3f47e44b9ba3645d12b20f7502)` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 11:05:29.292563 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0b47d70-44de-4722-a633-9dd6e3fb6a22 admin admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 11:05:29.339092 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-e0b47d70-44de-4722-a633-9dd6e3fb6a22 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2b757e3f47e44b9ba3645d12b20f7502'} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 11:05:29.346126 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-e0b47d70-44de-4722-a633-9dd6e3fb6a22 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2b757e3f47e44b9ba3645d12b20f7502'} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 11:05:29.359216 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-e0b47d70-44de-4722-a633-9dd6e3fb6a22 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2b757e3f47e44b9ba3645d12b20f7502'} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 11:05:29.359922 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-e0b47d70-44de-4722-a633-9dd6e3fb6a22 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2b757e3f47e44b9ba3645d12b20f7502'} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 11:05:29.367421 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-e0b47d70-44de-4722-a633-9dd6e3fb6a22 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2b757e3f47e44b9ba3645d12b20f7502'} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 11:05:29.373421 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1274/2577] 10.222.0.25 () {58 vars in 1224 bytes} [Sun Aug 7 11:05:29 2022] DELETE /identity/v3/users/2b757e3f47e44b9ba3645d12b20f7502 => generated 0 bytes in 111 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 07 11:05:29.563805 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-628d9bac-e106-457e-b549-7f80e14503e5 service neutron] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 11:05:29.575434 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-628d9bac-e106-457e-b549-7f80e14503e5 service neutron] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 11:05:29.643383 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52595d8c-6f6b-4ddf-ada3-5df78ccf249e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5ac47e3b09f43d19a23a0f457fe9e23', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 11:05:29.644841 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52595d8c-6f6b-4ddf-ada3-5df78ccf249e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 11:05:29.645124 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52595d8c-6f6b-4ddf-ada3-5df78ccf249e admin admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 11:05:29.645404 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52595d8c-6f6b-4ddf-ada3-5df78ccf249e admin admin] PATH_INFO: `/v3/projects/1f23b23bf2f84fdf8401acf4d73e1f4b` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 11:05:29.646560 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52595d8c-6f6b-4ddf-ada3-5df78ccf249e admin admin] RBAC: Authorizing `identity:delete_project(project_id=1f23b23bf2f84fdf8401acf4d73e1f4b)` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 11:05:29.648537 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52595d8c-6f6b-4ddf-ada3-5df78ccf249e admin admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 11:05:29.694663 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.notifications [None req-52595d8c-6f6b-4ddf-ada3-5df78ccf249e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1f23b23bf2f84fdf8401acf4d73e1f4b', 'request_id': 'req-52595d8c-6f6b-4ddf-ada3-5df78ccf249e'} {{(pid=80144) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 11:05:29.695817 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=96,keystone:DELETE=4 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 11:05:29.701487 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.notifications [None req-52595d8c-6f6b-4ddf-ada3-5df78ccf249e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1f23b23bf2f84fdf8401acf4d73e1f4b', 'request_id': 'req-52595d8c-6f6b-4ddf-ada3-5df78ccf249e'} {{(pid=80144) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 11:05:29.708360 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.notifications [None req-52595d8c-6f6b-4ddf-ada3-5df78ccf249e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1f23b23bf2f84fdf8401acf4d73e1f4b', 'request_id': 'req-52595d8c-6f6b-4ddf-ada3-5df78ccf249e'} {{(pid=80144) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 11:05:29.719547 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.notifications [None req-52595d8c-6f6b-4ddf-ada3-5df78ccf249e admin admin] The token cache is being invalidate because project 1f23b23bf2f84fdf8401acf4d73e1f4b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80144) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 07 11:05:29.721032 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.notifications [None req-52595d8c-6f6b-4ddf-ada3-5df78ccf249e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80144) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 11:05:29.721766 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.notifications [None req-52595d8c-6f6b-4ddf-ada3-5df78ccf249e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80144) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 11:05:29.723592 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 1304/2578] 10.222.0.25 () {58 vars in 1233 bytes} [Sun Aug 7 11:05:29 2022] DELETE /identity/v3/projects/1f23b23bf2f84fdf8401acf4d73e1f4b => generated 0 bytes in 161 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 07 11:05:29.728753 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f083f2a-c9c7-4d85-9b55-2543a94f7827 None None] REQUEST_METHOD: `POST` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 11:05:29.729044 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f083f2a-c9c7-4d85-9b55-2543a94f7827 None None] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 11:05:29.729368 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f083f2a-c9c7-4d85-9b55-2543a94f7827 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 11:05:29.750124 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.auth.core [None req-6f083f2a-c9c7-4d85-9b55-2543a94f7827 None None] MFA Rules not processed for user `bac4e6b76fe64217ac8fa44e71cffe6a`. Rule list: `[]` (Enabled: `True`). {{(pid=80143) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 11:05:29.751362 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-6f083f2a-c9c7-4d85-9b55-2543a94f7827 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 11:05:29.810254 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1275/2579] 10.222.0.25 () {58 vars in 958 bytes} [Sun Aug 7 11:05:29 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 07 11:05:29.819004 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13277715-7a6a-44f4-b8b1-eb2cfbff1f55 None None] REQUEST_METHOD: `POST` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 11:05:29.819292 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13277715-7a6a-44f4-b8b1-eb2cfbff1f55 None None] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 11:05:29.819568 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13277715-7a6a-44f4-b8b1-eb2cfbff1f55 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 11:05:29.838726 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.auth.core [None req-13277715-7a6a-44f4-b8b1-eb2cfbff1f55 None None] MFA Rules not processed for user `bac4e6b76fe64217ac8fa44e71cffe6a`. Rule list: `[]` (Enabled: `True`). {{(pid=80144) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 11:05:29.839968 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.fernet_utils [None req-13277715-7a6a-44f4-b8b1-eb2cfbff1f55 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80144) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 11:05:29.847975 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 1305/2580] 10.222.0.25 () {58 vars in 958 bytes} [Sun Aug 7 11:05:29 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 07 11:05:29.852267 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f083f2a-c9c7-4d85-9b55-2543a94f7827 None None] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 11:05:29.863017 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f083f2a-c9c7-4d85-9b55-2543a94f7827 None None] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 11:05:29.867585 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15be2ec7-2f16-40e0-9e90-92b135f876f8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5ac47e3b09f43d19a23a0f457fe9e23', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 11:05:29.868186 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15be2ec7-2f16-40e0-9e90-92b135f876f8 admin admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 11:05:29.868469 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15be2ec7-2f16-40e0-9e90-92b135f876f8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 11:05:29.868745 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15be2ec7-2f16-40e0-9e90-92b135f876f8 admin admin] PATH_INFO: `/v3/domains` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 11:05:29.869394 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15be2ec7-2f16-40e0-9e90-92b135f876f8 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80143) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 11:05:29.869710 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15be2ec7-2f16-40e0-9e90-92b135f876f8 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 11:05:29.871320 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15be2ec7-2f16-40e0-9e90-92b135f876f8 admin admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 11:05:29.880605 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 11:05:29.887218 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1276/2581] 10.222.0.25 () {58 vars in 1153 bytes} [Sun Aug 7 11:05:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 11:05:29.888593 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13277715-7a6a-44f4-b8b1-eb2cfbff1f55 None None] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 11:05:29.898307 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13277715-7a6a-44f4-b8b1-eb2cfbff1f55 None None] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 11:05:29.902970 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad3adeb1-9c21-4798-b4c5-7f5aa6782192 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5ac47e3b09f43d19a23a0f457fe9e23', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 11:05:29.903567 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad3adeb1-9c21-4798-b4c5-7f5aa6782192 admin admin] REQUEST_METHOD: `POST` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 11:05:29.903846 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad3adeb1-9c21-4798-b4c5-7f5aa6782192 admin admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 11:05:29.904119 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad3adeb1-9c21-4798-b4c5-7f5aa6782192 admin admin] PATH_INFO: `/v3/projects` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 11:05:29.904795 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad3adeb1-9c21-4798-b4c5-7f5aa6782192 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 11:05:29.906688 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad3adeb1-9c21-4798-b4c5-7f5aa6782192 admin admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 11:05:29.934553 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 1306/2582] 10.222.0.25 () {60 vars in 1153 bytes} [Sun Aug 7 11:05:29 2022] POST /identity/v3/projects => generated 375 bytes in 47 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 07 11:05:29.938757 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15be2ec7-2f16-40e0-9e90-92b135f876f8 admin admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 11:05:29.949395 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15be2ec7-2f16-40e0-9e90-92b135f876f8 admin admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 11:05:30.006174 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38194047-d99d-48bc-897b-6e503c9451b6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5ac47e3b09f43d19a23a0f457fe9e23', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 11:05:30.006782 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38194047-d99d-48bc-897b-6e503c9451b6 admin admin] REQUEST_METHOD: `POST` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 11:05:30.007099 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38194047-d99d-48bc-897b-6e503c9451b6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 11:05:30.007402 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38194047-d99d-48bc-897b-6e503c9451b6 admin admin] PATH_INFO: `/v3/users` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 11:05:30.008030 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38194047-d99d-48bc-897b-6e503c9451b6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 11:05:30.009926 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38194047-d99d-48bc-897b-6e503c9451b6 admin admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 11:05:30.033914 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1277/2583] 10.222.0.25 () {60 vars in 1144 bytes} [Sun Aug 7 11:05:29 2022] POST /identity/v3/users => generated 288 bytes in 93 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 07 11:05:30.039908 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad3adeb1-9c21-4798-b4c5-7f5aa6782192 admin admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 11:05:30.050082 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad3adeb1-9c21-4798-b4c5-7f5aa6782192 admin admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 11:05:30.055882 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9ac90cf-9761-449d-b28e-f81032ba3a2c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5ac47e3b09f43d19a23a0f457fe9e23', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 11:05:30.056488 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9ac90cf-9761-449d-b28e-f81032ba3a2c admin admin] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 11:05:30.056765 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9ac90cf-9761-449d-b28e-f81032ba3a2c admin admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 11:05:30.057034 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9ac90cf-9761-449d-b28e-f81032ba3a2c admin admin] PATH_INFO: `/v3/roles` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 11:05:30.057670 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9ac90cf-9761-449d-b28e-f81032ba3a2c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 11:05:30.059272 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9ac90cf-9761-449d-b28e-f81032ba3a2c admin admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 11:05:30.070604 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 1307/2584] 10.222.0.25 () {58 vars in 1122 bytes} [Sun Aug 7 11:05:30 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 07 11:05:30.073979 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38194047-d99d-48bc-897b-6e503c9451b6 admin admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 11:05:30.084743 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38194047-d99d-48bc-897b-6e503c9451b6 admin admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 11:05:30.089553 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bcdf5e0-891c-4eaa-ba76-c05350a08282 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5ac47e3b09f43d19a23a0f457fe9e23', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 11:05:30.090189 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bcdf5e0-891c-4eaa-ba76-c05350a08282 admin admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 11:05:30.090474 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bcdf5e0-891c-4eaa-ba76-c05350a08282 admin admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 11:05:30.090750 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bcdf5e0-891c-4eaa-ba76-c05350a08282 admin admin] PATH_INFO: `/v3/roles` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 11:05:30.091426 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bcdf5e0-891c-4eaa-ba76-c05350a08282 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 11:05:30.093007 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bcdf5e0-891c-4eaa-ba76-c05350a08282 admin admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 11:05:30.108503 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1278/2585] 10.222.0.25 () {58 vars in 1122 bytes} [Sun Aug 7 11:05:30 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 07 11:05:30.110198 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9ac90cf-9761-449d-b28e-f81032ba3a2c admin admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 11:05:30.121346 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9ac90cf-9761-449d-b28e-f81032ba3a2c admin admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 11:05:30.126051 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98ef55d2-1f23-4a7a-9aac-ecb7d0f36b57 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5ac47e3b09f43d19a23a0f457fe9e23', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 11:05:30.126705 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98ef55d2-1f23-4a7a-9aac-ecb7d0f36b57 admin admin] REQUEST_METHOD: `PUT` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 11:05:30.127012 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98ef55d2-1f23-4a7a-9aac-ecb7d0f36b57 admin admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 11:05:30.127288 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98ef55d2-1f23-4a7a-9aac-ecb7d0f36b57 admin admin] PATH_INFO: `/v3/projects/7b689949b03345e3a7ce7bf7a2ac77cd/users/c128883f6b4a4d8ab6841a693c782feb/roles/b98dfb3b9fee4644af2887a3ac3fd2e6` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 11:05:30.141936 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98ef55d2-1f23-4a7a-9aac-ecb7d0f36b57 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7b689949b03345e3a7ce7bf7a2ac77cd, user_id=c128883f6b4a4d8ab6841a693c782feb, role_id=b98dfb3b9fee4644af2887a3ac3fd2e6)` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 11:05:30.144747 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98ef55d2-1f23-4a7a-9aac-ecb7d0f36b57 admin admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 11:05:30.160373 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 1308/2586] 10.222.0.25 () {60 vars in 1483 bytes} [Sun Aug 7 11:05:30 2022] PUT /identity/v3/projects/7b689949b03345e3a7ce7bf7a2ac77cd/users/c128883f6b4a4d8ab6841a693c782feb/roles/b98dfb3b9fee4644af2887a3ac3fd2e6 => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 07 11:05:30.172292 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75098946-8429-4051-97ba-5d1f72663110 None None] REQUEST_METHOD: `POST` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 11:05:30.172589 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75098946-8429-4051-97ba-5d1f72663110 None None] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 11:05:30.172871 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75098946-8429-4051-97ba-5d1f72663110 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 11:05:30.207431 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.auth.core [None req-75098946-8429-4051-97ba-5d1f72663110 None None] MFA Rules not processed for user `c128883f6b4a4d8ab6841a693c782feb`. Rule list: `[]` (Enabled: `True`). {{(pid=80143) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 11:05:30.208654 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-75098946-8429-4051-97ba-5d1f72663110 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 11:05:30.280551 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1279/2587] 10.222.0.25 () {58 vars in 958 bytes} [Sun Aug 7 11:05:30 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 07 11:05:30.292488 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98ef55d2-1f23-4a7a-9aac-ecb7d0f36b57 admin admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 11:05:30.294186 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.fernet_utils [None req-98ef55d2-1f23-4a7a-9aac-ecb7d0f36b57 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80144) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 11:05:30.367643 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98ef55d2-1f23-4a7a-9aac-ecb7d0f36b57 admin admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 11:05:30.373167 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9b39ad2-16d8-422c-80a3-f96ecfd051cf service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31d16d1b96104b4eb690398432b9829e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6de1f3cac02c4bcdb05dd2337176033d', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'reader', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 11:05:30.373762 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9b39ad2-16d8-422c-80a3-f96ecfd051cf service nova] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 11:05:30.374040 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9b39ad2-16d8-422c-80a3-f96ecfd051cf service nova] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 11:05:30.374311 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9b39ad2-16d8-422c-80a3-f96ecfd051cf service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 11:05:30.382152 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9b39ad2-16d8-422c-80a3-f96ecfd051cf service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 11:05:30.383801 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9b39ad2-16d8-422c-80a3-f96ecfd051cf service nova] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 11:05:30.392555 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 1309/2588] 10.222.0.25 () {60 vars in 1365 bytes} [Sun Aug 7 11:05:30 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 101 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 07 11:05:30.598013 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75098946-8429-4051-97ba-5d1f72663110 None None] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 11:05:30.657736 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75098946-8429-4051-97ba-5d1f72663110 None None] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 11:05:30.663010 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cebdddb-84fb-446e-851c-22c933addc89 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5ac47e3b09f43d19a23a0f457fe9e23', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 11:05:30.663631 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cebdddb-84fb-446e-851c-22c933addc89 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 11:05:30.663907 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cebdddb-84fb-446e-851c-22c933addc89 admin admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 11:05:30.664178 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cebdddb-84fb-446e-851c-22c933addc89 admin admin] PATH_INFO: `/v3/users/c128883f6b4a4d8ab6841a693c782feb` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 11:05:30.665281 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cebdddb-84fb-446e-851c-22c933addc89 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c128883f6b4a4d8ab6841a693c782feb)` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 11:05:30.667207 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cebdddb-84fb-446e-851c-22c933addc89 admin admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 11:05:30.677459 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 11:05:30.712657 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-6cebdddb-84fb-446e-851c-22c933addc89 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c128883f6b4a4d8ab6841a693c782feb'} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 11:05:30.719730 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-6cebdddb-84fb-446e-851c-22c933addc89 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c128883f6b4a4d8ab6841a693c782feb'} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 11:05:30.732060 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-6cebdddb-84fb-446e-851c-22c933addc89 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c128883f6b4a4d8ab6841a693c782feb'} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 11:05:30.732731 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-6cebdddb-84fb-446e-851c-22c933addc89 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c128883f6b4a4d8ab6841a693c782feb'} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 11:05:30.740202 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-6cebdddb-84fb-446e-851c-22c933addc89 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c128883f6b4a4d8ab6841a693c782feb'} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 11:05:30.746143 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1280/2589] 10.222.0.25 () {58 vars in 1224 bytes} [Sun Aug 7 11:05:30 2022] DELETE /identity/v3/users/c128883f6b4a4d8ab6841a693c782feb => generated 0 bytes in 149 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 07 11:05:30.750722 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0328ac9a-eae4-4ed7-92e3-194cc9a1e97b None None] REQUEST_METHOD: `POST` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 11:05:30.751221 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0328ac9a-eae4-4ed7-92e3-194cc9a1e97b None None] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 11:05:30.751345 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0328ac9a-eae4-4ed7-92e3-194cc9a1e97b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 11:05:30.776043 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.auth.core [None req-0328ac9a-eae4-4ed7-92e3-194cc9a1e97b None None] MFA Rules not processed for user `bac4e6b76fe64217ac8fa44e71cffe6a`. Rule list: `[]` (Enabled: `True`). {{(pid=80144) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 11:05:30.777246 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.fernet_utils [None req-0328ac9a-eae4-4ed7-92e3-194cc9a1e97b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80144) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 11:05:30.826697 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:DELETE=3,keystone:SELECT=94,keystone:INSERT=3 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 11:05:30.839506 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 1310/2590] 10.222.0.25 () {58 vars in 958 bytes} [Sun Aug 7 11:05:30 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 07 11:05:30.846712 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cebdddb-84fb-446e-851c-22c933addc89 admin admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 11:05:30.848422 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-6cebdddb-84fb-446e-851c-22c933addc89 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 11:05:30.907183 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cebdddb-84fb-446e-851c-22c933addc89 admin admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 11:05:30.911239 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19398cfd-cc7c-4bb5-a137-5c9f01622468 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be1283c9bf47424c9aa3ad27cab03027', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6de1f3cac02c4bcdb05dd2337176033d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 11:05:30.911849 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19398cfd-cc7c-4bb5-a137-5c9f01622468 service neutron] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 11:05:30.912135 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19398cfd-cc7c-4bb5-a137-5c9f01622468 service neutron] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 11:05:30.912402 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19398cfd-cc7c-4bb5-a137-5c9f01622468 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 11:05:30.921837 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19398cfd-cc7c-4bb5-a137-5c9f01622468 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 11:05:30.923363 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19398cfd-cc7c-4bb5-a137-5c9f01622468 service neutron] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 11:05:30.933142 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1281/2591] 10.222.0.25 () {60 vars in 1365 bytes} [Sun Aug 7 11:05:30 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 07 11:05:31.226707 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0328ac9a-eae4-4ed7-92e3-194cc9a1e97b None None] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 11:05:31.228513 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.fernet_utils [None req-0328ac9a-eae4-4ed7-92e3-194cc9a1e97b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80144) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 11:05:31.248282 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0328ac9a-eae4-4ed7-92e3-194cc9a1e97b None None] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 11:05:31.254671 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed15fc92-8aee-4f4d-bd87-02458810c493 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5ac47e3b09f43d19a23a0f457fe9e23', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 11:05:31.255502 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed15fc92-8aee-4f4d-bd87-02458810c493 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 11:05:31.256040 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed15fc92-8aee-4f4d-bd87-02458810c493 admin admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 11:05:31.256375 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed15fc92-8aee-4f4d-bd87-02458810c493 admin admin] PATH_INFO: `/v3/projects/7b689949b03345e3a7ce7bf7a2ac77cd` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 11:05:31.258465 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed15fc92-8aee-4f4d-bd87-02458810c493 admin admin] RBAC: Authorizing `identity:delete_project(project_id=7b689949b03345e3a7ce7bf7a2ac77cd)` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 11:05:31.260423 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed15fc92-8aee-4f4d-bd87-02458810c493 admin admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 11:05:31.322382 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.notifications [None req-ed15fc92-8aee-4f4d-bd87-02458810c493 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7b689949b03345e3a7ce7bf7a2ac77cd', 'request_id': 'req-ed15fc92-8aee-4f4d-bd87-02458810c493'} {{(pid=80144) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 11:05:31.340750 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.notifications [None req-ed15fc92-8aee-4f4d-bd87-02458810c493 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7b689949b03345e3a7ce7bf7a2ac77cd', 'request_id': 'req-ed15fc92-8aee-4f4d-bd87-02458810c493'} {{(pid=80144) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 11:05:31.350401 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.notifications [None req-ed15fc92-8aee-4f4d-bd87-02458810c493 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7b689949b03345e3a7ce7bf7a2ac77cd', 'request_id': 'req-ed15fc92-8aee-4f4d-bd87-02458810c493'} {{(pid=80144) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 11:05:31.363881 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.notifications [None req-ed15fc92-8aee-4f4d-bd87-02458810c493 admin admin] The token cache is being invalidate because project 7b689949b03345e3a7ce7bf7a2ac77cd was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80144) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 07 11:05:31.364855 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.notifications [None req-ed15fc92-8aee-4f4d-bd87-02458810c493 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80144) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 11:05:31.365152 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.notifications [None req-ed15fc92-8aee-4f4d-bd87-02458810c493 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80144) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 11:05:31.367055 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 1311/2592] 10.222.0.25 () {58 vars in 1233 bytes} [Sun Aug 7 11:05:31 2022] DELETE /identity/v3/projects/7b689949b03345e3a7ce7bf7a2ac77cd => generated 0 bytes in 141 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 07 11:05:31.371734 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ece99c5-3067-44eb-ab57-4441d3e7e1e1 None None] REQUEST_METHOD: `POST` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 11:05:31.372214 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ece99c5-3067-44eb-ab57-4441d3e7e1e1 None None] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 11:05:31.372329 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ece99c5-3067-44eb-ab57-4441d3e7e1e1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 11:05:31.395531 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.auth.core [None req-7ece99c5-3067-44eb-ab57-4441d3e7e1e1 None None] MFA Rules not processed for user `bac4e6b76fe64217ac8fa44e71cffe6a`. Rule list: `[]` (Enabled: `True`). {{(pid=80143) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 11:05:31.396350 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-7ece99c5-3067-44eb-ab57-4441d3e7e1e1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 11:05:31.473747 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1282/2593] 10.222.0.25 () {58 vars in 958 bytes} [Sun Aug 7 11:05:31 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 07 11:05:31.482760 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9befda99-500f-4dc7-9401-2a94dbc69b71 None None] REQUEST_METHOD: `POST` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 11:05:31.483076 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9befda99-500f-4dc7-9401-2a94dbc69b71 None None] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 11:05:31.483354 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9befda99-500f-4dc7-9401-2a94dbc69b71 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 11:05:31.511774 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.auth.core [None req-9befda99-500f-4dc7-9401-2a94dbc69b71 None None] MFA Rules not processed for user `bac4e6b76fe64217ac8fa44e71cffe6a`. Rule list: `[]` (Enabled: `True`). {{(pid=80144) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 11:05:31.513136 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.fernet_utils [None req-9befda99-500f-4dc7-9401-2a94dbc69b71 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80144) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 11:05:31.524351 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 1312/2594] 10.222.0.25 () {58 vars in 958 bytes} [Sun Aug 7 11:05:31 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 43 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 07 11:05:31.528869 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ece99c5-3067-44eb-ab57-4441d3e7e1e1 None None] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 11:05:31.542095 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ece99c5-3067-44eb-ab57-4441d3e7e1e1 None None] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 11:05:31.548905 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d02b98a-092b-4ee7-85dd-773da71e934d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5ac47e3b09f43d19a23a0f457fe9e23', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 11:05:31.549528 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d02b98a-092b-4ee7-85dd-773da71e934d admin admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 11:05:31.549807 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d02b98a-092b-4ee7-85dd-773da71e934d admin admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 11:05:31.550169 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d02b98a-092b-4ee7-85dd-773da71e934d admin admin] PATH_INFO: `/v3/domains` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 11:05:31.551419 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d02b98a-092b-4ee7-85dd-773da71e934d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80143) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 11:05:31.551724 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d02b98a-092b-4ee7-85dd-773da71e934d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 11:05:31.554619 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d02b98a-092b-4ee7-85dd-773da71e934d admin admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 11:05:31.569180 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1283/2595] 10.222.0.25 () {58 vars in 1153 bytes} [Sun Aug 7 11:05:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 11:05:31.573613 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9befda99-500f-4dc7-9401-2a94dbc69b71 None None] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 11:05:31.584811 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9befda99-500f-4dc7-9401-2a94dbc69b71 None None] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 11:05:31.591862 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50333456-d54a-482b-b04e-16f40cf58e62 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5ac47e3b09f43d19a23a0f457fe9e23', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 11:05:31.592453 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50333456-d54a-482b-b04e-16f40cf58e62 admin admin] REQUEST_METHOD: `POST` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 11:05:31.592780 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50333456-d54a-482b-b04e-16f40cf58e62 admin admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 11:05:31.593059 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50333456-d54a-482b-b04e-16f40cf58e62 admin admin] PATH_INFO: `/v3/projects` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 11:05:31.593679 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50333456-d54a-482b-b04e-16f40cf58e62 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 11:05:31.595783 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50333456-d54a-482b-b04e-16f40cf58e62 admin admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 11:05:31.630893 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 1313/2596] 10.222.0.25 () {60 vars in 1153 bytes} [Sun Aug 7 11:05:31 2022] POST /identity/v3/projects => generated 403 bytes in 58 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 07 11:05:31.639358 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d02b98a-092b-4ee7-85dd-773da71e934d admin admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 11:05:31.649844 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d02b98a-092b-4ee7-85dd-773da71e934d admin admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 11:05:31.663535 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:DELETE=10,keystone:SELECT=89,keystone:INSERT=1 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 11:05:31.712780 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-875fed4d-2cfe-4f08-b876-20d7825b689d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5ac47e3b09f43d19a23a0f457fe9e23', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 11:05:31.713411 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-875fed4d-2cfe-4f08-b876-20d7825b689d admin admin] REQUEST_METHOD: `POST` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 11:05:31.713917 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-875fed4d-2cfe-4f08-b876-20d7825b689d admin admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 11:05:31.714399 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-875fed4d-2cfe-4f08-b876-20d7825b689d admin admin] PATH_INFO: `/v3/users` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 11:05:31.715488 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-875fed4d-2cfe-4f08-b876-20d7825b689d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 11:05:31.718904 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-875fed4d-2cfe-4f08-b876-20d7825b689d admin admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 11:05:31.746978 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1284/2597] 10.222.0.25 () {60 vars in 1144 bytes} [Sun Aug 7 11:05:31 2022] POST /identity/v3/users => generated 302 bytes in 105 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 07 11:05:31.748793 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50333456-d54a-482b-b04e-16f40cf58e62 admin admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 11:05:31.763627 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50333456-d54a-482b-b04e-16f40cf58e62 admin admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 11:05:31.768458 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3f3b694-7193-4a40-b12c-93642c45feb6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5ac47e3b09f43d19a23a0f457fe9e23', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 11:05:31.769729 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3f3b694-7193-4a40-b12c-93642c45feb6 admin admin] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 11:05:31.770004 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3f3b694-7193-4a40-b12c-93642c45feb6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 11:05:31.770277 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3f3b694-7193-4a40-b12c-93642c45feb6 admin admin] PATH_INFO: `/v3/roles` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 11:05:31.770917 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3f3b694-7193-4a40-b12c-93642c45feb6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 11:05:31.772531 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3f3b694-7193-4a40-b12c-93642c45feb6 admin admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 11:05:31.790781 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 1314/2598] 10.222.0.25 () {58 vars in 1122 bytes} [Sun Aug 7 11:05:31 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 07 11:05:31.792815 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-875fed4d-2cfe-4f08-b876-20d7825b689d admin admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 11:05:31.804931 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-875fed4d-2cfe-4f08-b876-20d7825b689d admin admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 11:05:31.810428 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6b54777-c2ed-49e2-99bd-dceadd71c6bd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5ac47e3b09f43d19a23a0f457fe9e23', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 11:05:31.811083 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6b54777-c2ed-49e2-99bd-dceadd71c6bd admin admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 11:05:31.811367 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6b54777-c2ed-49e2-99bd-dceadd71c6bd admin admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 11:05:31.811636 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6b54777-c2ed-49e2-99bd-dceadd71c6bd admin admin] PATH_INFO: `/v3/roles` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 11:05:31.812283 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6b54777-c2ed-49e2-99bd-dceadd71c6bd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 11:05:31.813877 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6b54777-c2ed-49e2-99bd-dceadd71c6bd admin admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 11:05:31.825654 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1285/2599] 10.222.0.25 () {58 vars in 1122 bytes} [Sun Aug 7 11:05:31 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 07 11:05:31.830416 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3f3b694-7193-4a40-b12c-93642c45feb6 admin admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 11:05:31.841423 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3f3b694-7193-4a40-b12c-93642c45feb6 admin admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 11:05:31.846101 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02d86fc1-dfb4-4118-95a7-5a486ef10a4f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5ac47e3b09f43d19a23a0f457fe9e23', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 11:05:31.846769 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02d86fc1-dfb4-4118-95a7-5a486ef10a4f admin admin] REQUEST_METHOD: `PUT` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 11:05:31.847151 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02d86fc1-dfb4-4118-95a7-5a486ef10a4f admin admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 11:05:31.847401 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02d86fc1-dfb4-4118-95a7-5a486ef10a4f admin admin] PATH_INFO: `/v3/projects/0f10b51b1fb84a68b583978e1973a72c/users/b179c81f8dc24221a600ca718fe3a09e/roles/b98dfb3b9fee4644af2887a3ac3fd2e6` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 11:05:31.862070 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02d86fc1-dfb4-4118-95a7-5a486ef10a4f admin admin] RBAC: Authorizing `identity:create_grant(project_id=0f10b51b1fb84a68b583978e1973a72c, user_id=b179c81f8dc24221a600ca718fe3a09e, role_id=b98dfb3b9fee4644af2887a3ac3fd2e6)` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 11:05:31.865357 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02d86fc1-dfb4-4118-95a7-5a486ef10a4f admin admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 11:05:31.879411 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 1315/2600] 10.222.0.25 () {60 vars in 1483 bytes} [Sun Aug 7 11:05:31 2022] PUT /identity/v3/projects/0f10b51b1fb84a68b583978e1973a72c/users/b179c81f8dc24221a600ca718fe3a09e/roles/b98dfb3b9fee4644af2887a3ac3fd2e6 => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 07 11:05:31.889911 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1a88f3a-b2c3-478e-a509-999373b434e7 None None] REQUEST_METHOD: `POST` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 11:05:31.890201 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1a88f3a-b2c3-478e-a509-999373b434e7 None None] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 11:05:31.890479 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1a88f3a-b2c3-478e-a509-999373b434e7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 11:05:31.917843 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.auth.core [None req-f1a88f3a-b2c3-478e-a509-999373b434e7 None None] MFA Rules not processed for user `b179c81f8dc24221a600ca718fe3a09e`. Rule list: `[]` (Enabled: `True`). {{(pid=80143) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 11:05:31.919166 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-f1a88f3a-b2c3-478e-a509-999373b434e7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 11:05:31.982458 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1286/2601] 10.222.0.25 () {58 vars in 958 bytes} [Sun Aug 7 11:05:31 2022] POST /identity/v3/auth/tokens => generated 3280 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 07 11:05:31.987399 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02d86fc1-dfb4-4118-95a7-5a486ef10a4f admin admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 11:05:32.049333 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02d86fc1-dfb4-4118-95a7-5a486ef10a4f admin admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 11:05:32.054633 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2161ec8a-9e33-40dd-80e4-83569bea3091 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5ac47e3b09f43d19a23a0f457fe9e23', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 11:05:32.055265 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2161ec8a-9e33-40dd-80e4-83569bea3091 admin admin] REQUEST_METHOD: `POST` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 11:05:32.055544 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2161ec8a-9e33-40dd-80e4-83569bea3091 admin admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 11:05:32.055835 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2161ec8a-9e33-40dd-80e4-83569bea3091 admin admin] PATH_INFO: `/v3/projects` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 11:05:32.056466 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2161ec8a-9e33-40dd-80e4-83569bea3091 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 11:05:32.058356 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2161ec8a-9e33-40dd-80e4-83569bea3091 admin admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 11:05:32.086435 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 1316/2602] 10.222.0.25 () {60 vars in 1153 bytes} [Sun Aug 7 11:05:31 2022] POST /identity/v3/projects => generated 405 bytes in 100 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 07 11:05:32.090650 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1a88f3a-b2c3-478e-a509-999373b434e7 None None] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 11:05:32.101866 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1a88f3a-b2c3-478e-a509-999373b434e7 None None] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 11:05:32.158075 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96dc744f-5047-4822-8f8b-4f79ab5b3cdc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5ac47e3b09f43d19a23a0f457fe9e23', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 11:05:32.158685 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96dc744f-5047-4822-8f8b-4f79ab5b3cdc admin admin] REQUEST_METHOD: `POST` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 11:05:32.159071 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96dc744f-5047-4822-8f8b-4f79ab5b3cdc admin admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 11:05:32.159353 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96dc744f-5047-4822-8f8b-4f79ab5b3cdc admin admin] PATH_INFO: `/v3/users` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 11:05:32.160572 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96dc744f-5047-4822-8f8b-4f79ab5b3cdc admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 11:05:32.162508 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96dc744f-5047-4822-8f8b-4f79ab5b3cdc admin admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 11:05:32.183148 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1287/2603] 10.222.0.25 () {60 vars in 1144 bytes} [Sun Aug 7 11:05:32 2022] POST /identity/v3/users => generated 309 bytes in 93 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 07 11:05:32.187215 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2161ec8a-9e33-40dd-80e4-83569bea3091 admin admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 11:05:32.194415 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 11:05:32.198547 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2161ec8a-9e33-40dd-80e4-83569bea3091 admin admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 11:05:32.204322 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61f27a2c-5030-4999-b88c-22f0d765b5d1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5ac47e3b09f43d19a23a0f457fe9e23', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 11:05:32.205098 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61f27a2c-5030-4999-b88c-22f0d765b5d1 admin admin] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 11:05:32.205538 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61f27a2c-5030-4999-b88c-22f0d765b5d1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 11:05:32.205984 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61f27a2c-5030-4999-b88c-22f0d765b5d1 admin admin] PATH_INFO: `/v3/roles` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 11:05:32.207283 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61f27a2c-5030-4999-b88c-22f0d765b5d1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 11:05:32.208967 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61f27a2c-5030-4999-b88c-22f0d765b5d1 admin admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 11:05:32.220615 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 1317/2604] 10.222.0.25 () {58 vars in 1122 bytes} [Sun Aug 7 11:05:32 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 07 11:05:32.225110 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96dc744f-5047-4822-8f8b-4f79ab5b3cdc admin admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 11:05:32.235722 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96dc744f-5047-4822-8f8b-4f79ab5b3cdc admin admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 11:05:32.240731 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04b8e989-6142-42a6-9b6a-c48cd7ed4b18 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5ac47e3b09f43d19a23a0f457fe9e23', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 11:05:32.241387 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04b8e989-6142-42a6-9b6a-c48cd7ed4b18 admin admin] REQUEST_METHOD: `PUT` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 11:05:32.241670 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04b8e989-6142-42a6-9b6a-c48cd7ed4b18 admin admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 11:05:32.241945 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04b8e989-6142-42a6-9b6a-c48cd7ed4b18 admin admin] PATH_INFO: `/v3/projects/8073eb814a2149d685c79aaf6a25a118/users/abc98252214f4d0baf2da6d6c655b3c6/roles/81b1f1bcee2a473088fbbd327c1c5b23` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 11:05:32.255496 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 11:05:32.258511 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04b8e989-6142-42a6-9b6a-c48cd7ed4b18 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8073eb814a2149d685c79aaf6a25a118, user_id=abc98252214f4d0baf2da6d6c655b3c6, role_id=81b1f1bcee2a473088fbbd327c1c5b23)` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 11:05:32.261290 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04b8e989-6142-42a6-9b6a-c48cd7ed4b18 admin admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 11:05:32.276368 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1288/2605] 10.222.0.25 () {60 vars in 1483 bytes} [Sun Aug 7 11:05:32 2022] PUT /identity/v3/projects/8073eb814a2149d685c79aaf6a25a118/users/abc98252214f4d0baf2da6d6c655b3c6/roles/81b1f1bcee2a473088fbbd327c1c5b23 => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 07 11:05:32.280291 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61f27a2c-5030-4999-b88c-22f0d765b5d1 admin admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 11:05:32.290342 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61f27a2c-5030-4999-b88c-22f0d765b5d1 admin admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 11:05:32.346492 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a9f9390-ec26-4445-adb7-75529da3e17d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5ac47e3b09f43d19a23a0f457fe9e23', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 11:05:32.347090 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a9f9390-ec26-4445-adb7-75529da3e17d admin admin] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 11:05:32.347373 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a9f9390-ec26-4445-adb7-75529da3e17d admin admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 11:05:32.347652 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a9f9390-ec26-4445-adb7-75529da3e17d admin admin] PATH_INFO: `/v3/roles` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 11:05:32.348289 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a9f9390-ec26-4445-adb7-75529da3e17d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 11:05:32.349872 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a9f9390-ec26-4445-adb7-75529da3e17d admin admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 11:05:32.359594 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 1318/2606] 10.222.0.25 () {58 vars in 1122 bytes} [Sun Aug 7 11:05:32 2022] GET /identity/v3/roles => generated 1420 bytes in 80 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 07 11:05:32.363784 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04b8e989-6142-42a6-9b6a-c48cd7ed4b18 admin admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 11:05:32.374028 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04b8e989-6142-42a6-9b6a-c48cd7ed4b18 admin admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 11:05:32.379071 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff2dc556-fa67-47ab-9b34-aa5d0b1255b1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5ac47e3b09f43d19a23a0f457fe9e23', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 11:05:32.379718 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff2dc556-fa67-47ab-9b34-aa5d0b1255b1 admin admin] REQUEST_METHOD: `PUT` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 11:05:32.379997 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff2dc556-fa67-47ab-9b34-aa5d0b1255b1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 11:05:32.380272 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff2dc556-fa67-47ab-9b34-aa5d0b1255b1 admin admin] PATH_INFO: `/v3/projects/8073eb814a2149d685c79aaf6a25a118/users/abc98252214f4d0baf2da6d6c655b3c6/roles/81b1f1bcee2a473088fbbd327c1c5b23` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 11:05:32.382173 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff2dc556-fa67-47ab-9b34-aa5d0b1255b1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8073eb814a2149d685c79aaf6a25a118, user_id=abc98252214f4d0baf2da6d6c655b3c6, role_id=81b1f1bcee2a473088fbbd327c1c5b23)` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 11:05:32.384805 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff2dc556-fa67-47ab-9b34-aa5d0b1255b1 admin admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 11:05:32.395344 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1289/2607] 10.222.0.25 () {60 vars in 1483 bytes} [Sun Aug 7 11:05:32 2022] PUT /identity/v3/projects/8073eb814a2149d685c79aaf6a25a118/users/abc98252214f4d0baf2da6d6c655b3c6/roles/81b1f1bcee2a473088fbbd327c1c5b23 => generated 0 bytes in 32 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 07 11:05:32.406747 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b235cd1-cd7d-48fc-b062-2dcde084e1c9 None None] REQUEST_METHOD: `POST` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 11:05:32.407074 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b235cd1-cd7d-48fc-b062-2dcde084e1c9 None None] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 11:05:32.407353 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b235cd1-cd7d-48fc-b062-2dcde084e1c9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 11:05:32.435636 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.auth.core [None req-8b235cd1-cd7d-48fc-b062-2dcde084e1c9 None None] MFA Rules not processed for user `abc98252214f4d0baf2da6d6c655b3c6`. Rule list: `[]` (Enabled: `True`). {{(pid=80144) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 11:05:32.436847 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.fernet_utils [None req-8b235cd1-cd7d-48fc-b062-2dcde084e1c9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80144) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 11:05:32.505651 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 1319/2608] 10.222.0.25 () {58 vars in 958 bytes} [Sun Aug 7 11:05:32 2022] POST /identity/v3/auth/tokens => generated 3349 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 07 11:05:32.516294 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff2dc556-fa67-47ab-9b34-aa5d0b1255b1 admin admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 11:05:32.517985 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-ff2dc556-fa67-47ab-9b34-aa5d0b1255b1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 11:05:32.574087 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff2dc556-fa67-47ab-9b34-aa5d0b1255b1 admin admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 11:05:32.577689 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9abd5e25-eac6-4cbb-8d6e-781a8c657250 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88dbd6dfd1164994a3e16da553561910', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6de1f3cac02c4bcdb05dd2337176033d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 11:05:32.578285 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9abd5e25-eac6-4cbb-8d6e-781a8c657250 service cinder] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 11:05:32.578562 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9abd5e25-eac6-4cbb-8d6e-781a8c657250 service cinder] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 11:05:32.578830 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9abd5e25-eac6-4cbb-8d6e-781a8c657250 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 11:05:32.588080 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9abd5e25-eac6-4cbb-8d6e-781a8c657250 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 11:05:32.589487 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9abd5e25-eac6-4cbb-8d6e-781a8c657250 service cinder] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 11:05:32.599563 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1290/2609] 10.222.0.25 () {60 vars in 1365 bytes} [Sun Aug 7 11:05:32 2022] GET /identity/v3/auth/tokens => generated 3349 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 07 11:05:33.896389 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b235cd1-cd7d-48fc-b062-2dcde084e1c9 None None] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 11:05:33.957430 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b235cd1-cd7d-48fc-b062-2dcde084e1c9 None None] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 11:05:33.961966 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebd39ac9-aed1-4fef-b11b-d0463d51267a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5ac47e3b09f43d19a23a0f457fe9e23', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 11:05:33.962585 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebd39ac9-aed1-4fef-b11b-d0463d51267a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 11:05:33.962861 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebd39ac9-aed1-4fef-b11b-d0463d51267a admin admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 11:05:33.963163 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebd39ac9-aed1-4fef-b11b-d0463d51267a admin admin] PATH_INFO: `/v3/users/b179c81f8dc24221a600ca718fe3a09e` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 11:05:33.964269 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebd39ac9-aed1-4fef-b11b-d0463d51267a admin admin] RBAC: Authorizing `identity:delete_user(user_id=b179c81f8dc24221a600ca718fe3a09e)` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 11:05:33.966157 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebd39ac9-aed1-4fef-b11b-d0463d51267a admin admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 11:05:33.984179 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=99,keystone:DELETE=1 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 11:05:34.013055 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.notifications [None req-ebd39ac9-aed1-4fef-b11b-d0463d51267a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b179c81f8dc24221a600ca718fe3a09e'} {{(pid=80144) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 11:05:34.020513 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.notifications [None req-ebd39ac9-aed1-4fef-b11b-d0463d51267a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b179c81f8dc24221a600ca718fe3a09e'} {{(pid=80144) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 11:05:34.027413 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.notifications [None req-ebd39ac9-aed1-4fef-b11b-d0463d51267a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b179c81f8dc24221a600ca718fe3a09e'} {{(pid=80144) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 11:05:34.028010 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.notifications [None req-ebd39ac9-aed1-4fef-b11b-d0463d51267a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b179c81f8dc24221a600ca718fe3a09e'} {{(pid=80144) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 11:05:34.028630 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.notifications [None req-ebd39ac9-aed1-4fef-b11b-d0463d51267a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b179c81f8dc24221a600ca718fe3a09e'} {{(pid=80144) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 11:05:34.045266 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 1320/2610] 10.222.0.25 () {58 vars in 1224 bytes} [Sun Aug 7 11:05:33 2022] DELETE /identity/v3/users/b179c81f8dc24221a600ca718fe3a09e => generated 0 bytes in 149 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 07 11:05:34.049971 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75697a21-ef73-415b-8657-5ff364a763f5 None None] REQUEST_METHOD: `POST` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 11:05:34.050467 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75697a21-ef73-415b-8657-5ff364a763f5 None None] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 11:05:34.050574 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75697a21-ef73-415b-8657-5ff364a763f5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 11:05:34.069762 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.auth.core [None req-75697a21-ef73-415b-8657-5ff364a763f5 None None] MFA Rules not processed for user `bac4e6b76fe64217ac8fa44e71cffe6a`. Rule list: `[]` (Enabled: `True`). {{(pid=80143) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 11:05:34.070966 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-75697a21-ef73-415b-8657-5ff364a763f5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 11:05:34.131568 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1291/2611] 10.222.0.25 () {58 vars in 958 bytes} [Sun Aug 7 11:05:34 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 07 11:05:34.141319 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebd39ac9-aed1-4fef-b11b-d0463d51267a admin admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 11:05:34.142780 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.fernet_utils [None req-ebd39ac9-aed1-4fef-b11b-d0463d51267a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80144) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 11:05:34.201478 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebd39ac9-aed1-4fef-b11b-d0463d51267a admin admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 11:05:34.205167 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bad0b110-a408-4f43-b708-cdbd27bbc84c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be1283c9bf47424c9aa3ad27cab03027', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6de1f3cac02c4bcdb05dd2337176033d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 11:05:34.205778 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bad0b110-a408-4f43-b708-cdbd27bbc84c service neutron] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 11:05:34.206058 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bad0b110-a408-4f43-b708-cdbd27bbc84c service neutron] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 11:05:34.206429 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bad0b110-a408-4f43-b708-cdbd27bbc84c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 11:05:34.217945 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bad0b110-a408-4f43-b708-cdbd27bbc84c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 11:05:34.219412 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bad0b110-a408-4f43-b708-cdbd27bbc84c service neutron] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 11:05:34.229172 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 1321/2612] 10.222.0.25 () {60 vars in 1365 bytes} [Sun Aug 7 11:05:34 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 07 11:05:34.515334 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75697a21-ef73-415b-8657-5ff364a763f5 None None] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 11:05:34.516786 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-75697a21-ef73-415b-8657-5ff364a763f5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 11:05:34.528665 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75697a21-ef73-415b-8657-5ff364a763f5 None None] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 11:05:34.535960 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0857c8b5-0672-4b4a-a3f5-ae6c1086b005 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5ac47e3b09f43d19a23a0f457fe9e23', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 11:05:34.536613 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0857c8b5-0672-4b4a-a3f5-ae6c1086b005 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 11:05:34.536896 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0857c8b5-0672-4b4a-a3f5-ae6c1086b005 admin admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 11:05:34.537169 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0857c8b5-0672-4b4a-a3f5-ae6c1086b005 admin admin] PATH_INFO: `/v3/projects/0f10b51b1fb84a68b583978e1973a72c` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 11:05:34.538363 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0857c8b5-0672-4b4a-a3f5-ae6c1086b005 admin admin] RBAC: Authorizing `identity:delete_project(project_id=0f10b51b1fb84a68b583978e1973a72c)` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 11:05:34.540294 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0857c8b5-0672-4b4a-a3f5-ae6c1086b005 admin admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 11:05:34.589129 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-0857c8b5-0672-4b4a-a3f5-ae6c1086b005 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0f10b51b1fb84a68b583978e1973a72c', 'request_id': 'req-0857c8b5-0672-4b4a-a3f5-ae6c1086b005'} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 11:05:34.594547 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-0857c8b5-0672-4b4a-a3f5-ae6c1086b005 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0f10b51b1fb84a68b583978e1973a72c', 'request_id': 'req-0857c8b5-0672-4b4a-a3f5-ae6c1086b005'} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 11:05:34.601419 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-0857c8b5-0672-4b4a-a3f5-ae6c1086b005 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0f10b51b1fb84a68b583978e1973a72c', 'request_id': 'req-0857c8b5-0672-4b4a-a3f5-ae6c1086b005'} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 11:05:34.614254 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-0857c8b5-0672-4b4a-a3f5-ae6c1086b005 admin admin] The token cache is being invalidate because project 0f10b51b1fb84a68b583978e1973a72c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80143) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 07 11:05:34.614540 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-0857c8b5-0672-4b4a-a3f5-ae6c1086b005 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 11:05:34.615192 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-0857c8b5-0672-4b4a-a3f5-ae6c1086b005 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 11:05:34.616668 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1292/2613] 10.222.0.25 () {58 vars in 1233 bytes} [Sun Aug 7 11:05:34 2022] DELETE /identity/v3/projects/0f10b51b1fb84a68b583978e1973a72c => generated 0 bytes in 102 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 07 11:05:34.620630 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bad0b110-a408-4f43-b708-cdbd27bbc84c service neutron] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 11:05:34.631835 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bad0b110-a408-4f43-b708-cdbd27bbc84c service neutron] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 11:05:34.687822 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-660fcf2d-577a-4dd7-b05f-512afac85190 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5ac47e3b09f43d19a23a0f457fe9e23', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 11:05:34.688458 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-660fcf2d-577a-4dd7-b05f-512afac85190 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 11:05:34.688740 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-660fcf2d-577a-4dd7-b05f-512afac85190 admin admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 11:05:34.689014 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-660fcf2d-577a-4dd7-b05f-512afac85190 admin admin] PATH_INFO: `/v3/users/abc98252214f4d0baf2da6d6c655b3c6` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 11:05:34.690137 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-660fcf2d-577a-4dd7-b05f-512afac85190 admin admin] RBAC: Authorizing `identity:delete_user(user_id=abc98252214f4d0baf2da6d6c655b3c6)` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 11:05:34.692084 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-660fcf2d-577a-4dd7-b05f-512afac85190 admin admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 11:05:34.731488 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:DELETE=14,keystone:SELECT=85,keystone:INSERT=1 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 11:05:34.739823 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.notifications [None req-660fcf2d-577a-4dd7-b05f-512afac85190 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'abc98252214f4d0baf2da6d6c655b3c6'} {{(pid=80144) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 11:05:34.746954 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.notifications [None req-660fcf2d-577a-4dd7-b05f-512afac85190 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'abc98252214f4d0baf2da6d6c655b3c6'} {{(pid=80144) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 11:05:34.754486 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.notifications [None req-660fcf2d-577a-4dd7-b05f-512afac85190 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'abc98252214f4d0baf2da6d6c655b3c6'} {{(pid=80144) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 11:05:34.755154 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.notifications [None req-660fcf2d-577a-4dd7-b05f-512afac85190 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'abc98252214f4d0baf2da6d6c655b3c6'} {{(pid=80144) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 11:05:34.755837 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.notifications [None req-660fcf2d-577a-4dd7-b05f-512afac85190 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'abc98252214f4d0baf2da6d6c655b3c6'} {{(pid=80144) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 11:05:34.772928 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 1322/2614] 10.222.0.25 () {58 vars in 1224 bytes} [Sun Aug 7 11:05:34 2022] DELETE /identity/v3/users/abc98252214f4d0baf2da6d6c655b3c6 => generated 0 bytes in 153 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 07 11:05:35.049660 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0857c8b5-0672-4b4a-a3f5-ae6c1086b005 admin admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 11:05:35.051563 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-0857c8b5-0672-4b4a-a3f5-ae6c1086b005 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 11:05:35.060650 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=90,keystone:INSERT=3,keystone:DELETE=7 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 11:05:35.071557 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0857c8b5-0672-4b4a-a3f5-ae6c1086b005 admin admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 11:05:35.129296 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9f6ee6e-f51b-4d57-abbf-191092a4668a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5ac47e3b09f43d19a23a0f457fe9e23', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 11:05:35.129918 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9f6ee6e-f51b-4d57-abbf-191092a4668a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 11:05:35.130198 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9f6ee6e-f51b-4d57-abbf-191092a4668a admin admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 11:05:35.130469 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9f6ee6e-f51b-4d57-abbf-191092a4668a admin admin] PATH_INFO: `/v3/projects/8073eb814a2149d685c79aaf6a25a118` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 11:05:35.131726 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9f6ee6e-f51b-4d57-abbf-191092a4668a admin admin] RBAC: Authorizing `identity:delete_project(project_id=8073eb814a2149d685c79aaf6a25a118)` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 11:05:35.133593 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9f6ee6e-f51b-4d57-abbf-191092a4668a admin admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 11:05:35.178651 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-c9f6ee6e-f51b-4d57-abbf-191092a4668a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8073eb814a2149d685c79aaf6a25a118', 'request_id': 'req-c9f6ee6e-f51b-4d57-abbf-191092a4668a'} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 11:05:35.183931 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-c9f6ee6e-f51b-4d57-abbf-191092a4668a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8073eb814a2149d685c79aaf6a25a118', 'request_id': 'req-c9f6ee6e-f51b-4d57-abbf-191092a4668a'} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 11:05:35.189866 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-c9f6ee6e-f51b-4d57-abbf-191092a4668a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8073eb814a2149d685c79aaf6a25a118', 'request_id': 'req-c9f6ee6e-f51b-4d57-abbf-191092a4668a'} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 11:05:35.203226 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-c9f6ee6e-f51b-4d57-abbf-191092a4668a admin admin] The token cache is being invalidate because project 8073eb814a2149d685c79aaf6a25a118 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80143) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 07 11:05:35.203512 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-c9f6ee6e-f51b-4d57-abbf-191092a4668a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 11:05:35.204118 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-c9f6ee6e-f51b-4d57-abbf-191092a4668a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 11:05:35.205584 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1293/2615] 10.222.0.25 () {58 vars in 1233 bytes} [Sun Aug 7 11:05:35 2022] DELETE /identity/v3/projects/8073eb814a2149d685c79aaf6a25a118 => generated 0 bytes in 156 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 07 11:05:35.211370 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17d4be63-c116-47cd-b4c8-769222561eb6 None None] REQUEST_METHOD: `POST` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 11:05:35.211864 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17d4be63-c116-47cd-b4c8-769222561eb6 None None] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 11:05:35.212141 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17d4be63-c116-47cd-b4c8-769222561eb6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 11:05:35.231923 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.auth.core [None req-17d4be63-c116-47cd-b4c8-769222561eb6 None None] MFA Rules not processed for user `bac4e6b76fe64217ac8fa44e71cffe6a`. Rule list: `[]` (Enabled: `True`). {{(pid=80144) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 11:05:35.233123 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.fernet_utils [None req-17d4be63-c116-47cd-b4c8-769222561eb6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80144) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 11:05:35.292970 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 1323/2616] 10.222.0.25 () {58 vars in 958 bytes} [Sun Aug 7 11:05:35 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 07 11:05:35.302622 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90394c25-d660-472a-be8d-7d7c4f7e73cc None None] REQUEST_METHOD: `POST` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 11:05:35.302892 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90394c25-d660-472a-be8d-7d7c4f7e73cc None None] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 11:05:35.303208 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90394c25-d660-472a-be8d-7d7c4f7e73cc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 11:05:35.322872 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.auth.core [None req-90394c25-d660-472a-be8d-7d7c4f7e73cc None None] MFA Rules not processed for user `bac4e6b76fe64217ac8fa44e71cffe6a`. Rule list: `[]` (Enabled: `True`). {{(pid=80143) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 11:05:35.324084 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-90394c25-d660-472a-be8d-7d7c4f7e73cc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 11:05:35.332457 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1294/2617] 10.222.0.25 () {58 vars in 958 bytes} [Sun Aug 7 11:05:35 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 07 11:05:35.336607 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17d4be63-c116-47cd-b4c8-769222561eb6 None None] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 11:05:35.346451 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17d4be63-c116-47cd-b4c8-769222561eb6 None None] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 11:05:35.351442 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c2cc4e6-b997-4f68-b1df-3e5c42e801cb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5ac47e3b09f43d19a23a0f457fe9e23', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 11:05:35.352043 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c2cc4e6-b997-4f68-b1df-3e5c42e801cb admin admin] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 11:05:35.352324 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c2cc4e6-b997-4f68-b1df-3e5c42e801cb admin admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 11:05:35.352626 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c2cc4e6-b997-4f68-b1df-3e5c42e801cb admin admin] PATH_INFO: `/v3/domains` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 11:05:35.353270 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c2cc4e6-b997-4f68-b1df-3e5c42e801cb admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80144) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 11:05:35.353572 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c2cc4e6-b997-4f68-b1df-3e5c42e801cb admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 11:05:35.355195 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c2cc4e6-b997-4f68-b1df-3e5c42e801cb admin admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 11:05:35.367915 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 1324/2618] 10.222.0.25 () {58 vars in 1153 bytes} [Sun Aug 7 11:05:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 11:05:35.372139 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90394c25-d660-472a-be8d-7d7c4f7e73cc None None] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 11:05:35.381963 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90394c25-d660-472a-be8d-7d7c4f7e73cc None None] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 11:05:35.386529 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1fc6ea9-b3e4-4fdc-8634-df72d69188f0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5ac47e3b09f43d19a23a0f457fe9e23', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 11:05:35.387156 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1fc6ea9-b3e4-4fdc-8634-df72d69188f0 admin admin] REQUEST_METHOD: `POST` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 11:05:35.387436 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1fc6ea9-b3e4-4fdc-8634-df72d69188f0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 11:05:35.387706 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1fc6ea9-b3e4-4fdc-8634-df72d69188f0 admin admin] PATH_INFO: `/v3/projects` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 11:05:35.388327 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1fc6ea9-b3e4-4fdc-8634-df72d69188f0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 11:05:35.390209 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1fc6ea9-b3e4-4fdc-8634-df72d69188f0 admin admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 11:05:35.422263 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1295/2619] 10.222.0.25 () {60 vars in 1153 bytes} [Sun Aug 7 11:05:35 2022] POST /identity/v3/projects => generated 371 bytes in 48 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 07 11:05:35.423641 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c2cc4e6-b997-4f68-b1df-3e5c42e801cb admin admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 11:05:35.435970 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c2cc4e6-b997-4f68-b1df-3e5c42e801cb admin admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 11:05:35.501145 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2480e02-8d83-4a88-aae2-6cd060d64b8b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5ac47e3b09f43d19a23a0f457fe9e23', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 11:05:35.501761 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2480e02-8d83-4a88-aae2-6cd060d64b8b admin admin] REQUEST_METHOD: `POST` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 11:05:35.502037 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2480e02-8d83-4a88-aae2-6cd060d64b8b admin admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 11:05:35.502310 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2480e02-8d83-4a88-aae2-6cd060d64b8b admin admin] PATH_INFO: `/v3/users` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 11:05:35.503702 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2480e02-8d83-4a88-aae2-6cd060d64b8b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 11:05:35.505951 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2480e02-8d83-4a88-aae2-6cd060d64b8b admin admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 11:05:35.529317 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 1325/2620] 10.222.0.25 () {60 vars in 1144 bytes} [Sun Aug 7 11:05:35 2022] POST /identity/v3/users => generated 286 bytes in 106 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 07 11:05:35.533687 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1fc6ea9-b3e4-4fdc-8634-df72d69188f0 admin admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 11:05:35.544163 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1fc6ea9-b3e4-4fdc-8634-df72d69188f0 admin admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 11:05:35.548745 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9751401-2d69-4fdf-860d-029d90e2de3c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5ac47e3b09f43d19a23a0f457fe9e23', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 11:05:35.549340 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9751401-2d69-4fdf-860d-029d90e2de3c admin admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 11:05:35.549621 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9751401-2d69-4fdf-860d-029d90e2de3c admin admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 11:05:35.549897 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9751401-2d69-4fdf-860d-029d90e2de3c admin admin] PATH_INFO: `/v3/roles` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 11:05:35.550642 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9751401-2d69-4fdf-860d-029d90e2de3c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 11:05:35.552316 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9751401-2d69-4fdf-860d-029d90e2de3c admin admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 11:05:35.562312 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1296/2621] 10.222.0.25 () {58 vars in 1122 bytes} [Sun Aug 7 11:05:35 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 07 11:05:35.566852 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2480e02-8d83-4a88-aae2-6cd060d64b8b admin admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 11:05:35.577093 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2480e02-8d83-4a88-aae2-6cd060d64b8b admin admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 11:05:35.582455 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5aedbdf-9e14-4727-badf-f9490ac1a89b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5ac47e3b09f43d19a23a0f457fe9e23', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 11:05:35.583907 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5aedbdf-9e14-4727-badf-f9490ac1a89b admin admin] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 11:05:35.584204 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5aedbdf-9e14-4727-badf-f9490ac1a89b admin admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 11:05:35.584476 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5aedbdf-9e14-4727-badf-f9490ac1a89b admin admin] PATH_INFO: `/v3/roles` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 11:05:35.585427 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5aedbdf-9e14-4727-badf-f9490ac1a89b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 11:05:35.588105 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5aedbdf-9e14-4727-badf-f9490ac1a89b admin admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 11:05:35.599204 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 1326/2622] 10.222.0.25 () {58 vars in 1122 bytes} [Sun Aug 7 11:05:35 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 07 11:05:35.603499 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9751401-2d69-4fdf-860d-029d90e2de3c admin admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 11:05:35.614271 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9751401-2d69-4fdf-860d-029d90e2de3c admin admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 11:05:35.618816 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b917b3b-3268-4f7c-b194-d7af81c91181 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5ac47e3b09f43d19a23a0f457fe9e23', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 11:05:35.619500 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b917b3b-3268-4f7c-b194-d7af81c91181 admin admin] REQUEST_METHOD: `PUT` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 11:05:35.619783 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b917b3b-3268-4f7c-b194-d7af81c91181 admin admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 11:05:35.620062 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b917b3b-3268-4f7c-b194-d7af81c91181 admin admin] PATH_INFO: `/v3/projects/a73187620ecd4be78ab17e15d0807334/users/84ad97d05cc14e88902772e8157df2a3/roles/b98dfb3b9fee4644af2887a3ac3fd2e6` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 11:05:35.634696 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b917b3b-3268-4f7c-b194-d7af81c91181 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a73187620ecd4be78ab17e15d0807334, user_id=84ad97d05cc14e88902772e8157df2a3, role_id=b98dfb3b9fee4644af2887a3ac3fd2e6)` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 11:05:35.637372 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b917b3b-3268-4f7c-b194-d7af81c91181 admin admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 11:05:35.649975 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1297/2623] 10.222.0.25 () {60 vars in 1483 bytes} [Sun Aug 7 11:05:35 2022] PUT /identity/v3/projects/a73187620ecd4be78ab17e15d0807334/users/84ad97d05cc14e88902772e8157df2a3/roles/b98dfb3b9fee4644af2887a3ac3fd2e6 => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 07 11:05:35.660180 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94ae1e7f-7edf-48f6-99f2-a010c1469859 None None] REQUEST_METHOD: `POST` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 11:05:35.660465 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94ae1e7f-7edf-48f6-99f2-a010c1469859 None None] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 11:05:35.660742 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94ae1e7f-7edf-48f6-99f2-a010c1469859 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 11:05:35.688509 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.auth.core [None req-94ae1e7f-7edf-48f6-99f2-a010c1469859 None None] MFA Rules not processed for user `84ad97d05cc14e88902772e8157df2a3`. Rule list: `[]` (Enabled: `True`). {{(pid=80144) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 11:05:35.689659 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.fernet_utils [None req-94ae1e7f-7edf-48f6-99f2-a010c1469859 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80144) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 11:05:35.699080 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:DELETE=5,keystone:SELECT=91,keystone:INSERT=4 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 11:05:35.760145 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 1327/2624] 10.222.0.25 () {58 vars in 958 bytes} [Sun Aug 7 11:05:35 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 07 11:05:35.764801 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b917b3b-3268-4f7c-b194-d7af81c91181 admin admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 11:05:35.804187 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 11:05:35.837932 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b917b3b-3268-4f7c-b194-d7af81c91181 admin admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 11:05:35.842692 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-572bbf9f-b104-4751-aa17-93d92f8be15e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5ac47e3b09f43d19a23a0f457fe9e23', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 11:05:35.843340 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-572bbf9f-b104-4751-aa17-93d92f8be15e admin admin] REQUEST_METHOD: `POST` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 11:05:35.843641 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-572bbf9f-b104-4751-aa17-93d92f8be15e admin admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 11:05:35.843899 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-572bbf9f-b104-4751-aa17-93d92f8be15e admin admin] PATH_INFO: `/v3/projects` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 11:05:35.844523 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-572bbf9f-b104-4751-aa17-93d92f8be15e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 11:05:35.846429 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-572bbf9f-b104-4751-aa17-93d92f8be15e admin admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 11:05:35.875021 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1298/2625] 10.222.0.25 () {60 vars in 1153 bytes} [Sun Aug 7 11:05:35 2022] POST /identity/v3/projects => generated 373 bytes in 108 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 07 11:05:35.876942 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94ae1e7f-7edf-48f6-99f2-a010c1469859 None None] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 11:05:35.887992 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94ae1e7f-7edf-48f6-99f2-a010c1469859 None None] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 11:05:35.944561 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48cdc1bc-8b22-4ce5-8286-f2e205423988 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5ac47e3b09f43d19a23a0f457fe9e23', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 11:05:35.945091 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48cdc1bc-8b22-4ce5-8286-f2e205423988 admin admin] REQUEST_METHOD: `POST` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 11:05:35.945376 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48cdc1bc-8b22-4ce5-8286-f2e205423988 admin admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 11:05:35.945653 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48cdc1bc-8b22-4ce5-8286-f2e205423988 admin admin] PATH_INFO: `/v3/users` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 11:05:35.946286 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48cdc1bc-8b22-4ce5-8286-f2e205423988 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 11:05:35.948220 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48cdc1bc-8b22-4ce5-8286-f2e205423988 admin admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 11:05:35.969046 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 1328/2626] 10.222.0.25 () {60 vars in 1144 bytes} [Sun Aug 7 11:05:35 2022] POST /identity/v3/users => generated 293 bytes in 93 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 07 11:05:35.973430 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-572bbf9f-b104-4751-aa17-93d92f8be15e admin admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 11:05:35.983646 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-572bbf9f-b104-4751-aa17-93d92f8be15e admin admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 11:05:35.988482 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1f0f82f-2ecb-4bd8-979f-4fbf048b57f1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5ac47e3b09f43d19a23a0f457fe9e23', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 11:05:35.989080 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1f0f82f-2ecb-4bd8-979f-4fbf048b57f1 admin admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 11:05:35.989359 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1f0f82f-2ecb-4bd8-979f-4fbf048b57f1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 11:05:35.989633 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1f0f82f-2ecb-4bd8-979f-4fbf048b57f1 admin admin] PATH_INFO: `/v3/roles` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 11:05:35.990269 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1f0f82f-2ecb-4bd8-979f-4fbf048b57f1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 11:05:35.991880 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1f0f82f-2ecb-4bd8-979f-4fbf048b57f1 admin admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 11:05:36.002076 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1299/2627] 10.222.0.25 () {58 vars in 1122 bytes} [Sun Aug 7 11:05:35 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 07 11:05:36.006548 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48cdc1bc-8b22-4ce5-8286-f2e205423988 admin admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 11:05:36.016470 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48cdc1bc-8b22-4ce5-8286-f2e205423988 admin admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 11:05:36.021776 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22d7d2dd-3ee9-42b1-bcf5-d7b1b87745f7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5ac47e3b09f43d19a23a0f457fe9e23', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 11:05:36.022434 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22d7d2dd-3ee9-42b1-bcf5-d7b1b87745f7 admin admin] REQUEST_METHOD: `PUT` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 11:05:36.022777 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22d7d2dd-3ee9-42b1-bcf5-d7b1b87745f7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 11:05:36.023030 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22d7d2dd-3ee9-42b1-bcf5-d7b1b87745f7 admin admin] PATH_INFO: `/v3/projects/e2f1e70d74b94091b3504a72b06feb9b/users/de64ccfe4b6b41a083d663a98c0ed7a9/roles/81b1f1bcee2a473088fbbd327c1c5b23` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 11:05:36.037510 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22d7d2dd-3ee9-42b1-bcf5-d7b1b87745f7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e2f1e70d74b94091b3504a72b06feb9b, user_id=de64ccfe4b6b41a083d663a98c0ed7a9, role_id=81b1f1bcee2a473088fbbd327c1c5b23)` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 11:05:36.040317 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22d7d2dd-3ee9-42b1-bcf5-d7b1b87745f7 admin admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 11:05:36.052869 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 1329/2628] 10.222.0.25 () {60 vars in 1483 bytes} [Sun Aug 7 11:05:36 2022] PUT /identity/v3/projects/e2f1e70d74b94091b3504a72b06feb9b/users/de64ccfe4b6b41a083d663a98c0ed7a9/roles/81b1f1bcee2a473088fbbd327c1c5b23 => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 07 11:05:36.056804 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1f0f82f-2ecb-4bd8-979f-4fbf048b57f1 admin admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 11:05:36.067391 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1f0f82f-2ecb-4bd8-979f-4fbf048b57f1 admin admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 11:05:36.122121 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3a93dcd-3179-4a57-8f32-e0d991515fae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5ac47e3b09f43d19a23a0f457fe9e23', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 11:05:36.122720 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3a93dcd-3179-4a57-8f32-e0d991515fae admin admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 11:05:36.123034 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3a93dcd-3179-4a57-8f32-e0d991515fae admin admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 11:05:36.123307 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3a93dcd-3179-4a57-8f32-e0d991515fae admin admin] PATH_INFO: `/v3/roles` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 11:05:36.123943 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3a93dcd-3179-4a57-8f32-e0d991515fae admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 11:05:36.125506 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3a93dcd-3179-4a57-8f32-e0d991515fae admin admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 11:05:36.135513 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1300/2629] 10.222.0.25 () {58 vars in 1122 bytes} [Sun Aug 7 11:05:36 2022] GET /identity/v3/roles => generated 1420 bytes in 79 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 07 11:05:36.139672 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22d7d2dd-3ee9-42b1-bcf5-d7b1b87745f7 admin admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 11:05:36.150786 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22d7d2dd-3ee9-42b1-bcf5-d7b1b87745f7 admin admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 11:05:36.155421 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b476d404-a4a1-44fb-8890-d130b0172f6d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5ac47e3b09f43d19a23a0f457fe9e23', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 11:05:36.156068 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b476d404-a4a1-44fb-8890-d130b0172f6d admin admin] REQUEST_METHOD: `PUT` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 11:05:36.156350 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b476d404-a4a1-44fb-8890-d130b0172f6d admin admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 11:05:36.156625 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b476d404-a4a1-44fb-8890-d130b0172f6d admin admin] PATH_INFO: `/v3/projects/e2f1e70d74b94091b3504a72b06feb9b/users/de64ccfe4b6b41a083d663a98c0ed7a9/roles/81b1f1bcee2a473088fbbd327c1c5b23` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 11:05:36.158551 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b476d404-a4a1-44fb-8890-d130b0172f6d admin admin] RBAC: Authorizing `identity:create_grant(project_id=e2f1e70d74b94091b3504a72b06feb9b, user_id=de64ccfe4b6b41a083d663a98c0ed7a9, role_id=81b1f1bcee2a473088fbbd327c1c5b23)` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 11:05:36.161221 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b476d404-a4a1-44fb-8890-d130b0172f6d admin admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 11:05:36.171269 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 1330/2630] 10.222.0.25 () {60 vars in 1483 bytes} [Sun Aug 7 11:05:36 2022] PUT /identity/v3/projects/e2f1e70d74b94091b3504a72b06feb9b/users/de64ccfe4b6b41a083d663a98c0ed7a9/roles/81b1f1bcee2a473088fbbd327c1c5b23 => generated 0 bytes in 32 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 07 11:05:36.182665 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fd3a0c0-8ad4-43ef-b9fc-a591f1022ffe None None] REQUEST_METHOD: `POST` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 11:05:36.182980 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fd3a0c0-8ad4-43ef-b9fc-a591f1022ffe None None] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 11:05:36.183258 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fd3a0c0-8ad4-43ef-b9fc-a591f1022ffe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 11:05:36.212490 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.auth.core [None req-1fd3a0c0-8ad4-43ef-b9fc-a591f1022ffe None None] MFA Rules not processed for user `de64ccfe4b6b41a083d663a98c0ed7a9`. Rule list: `[]` (Enabled: `True`). {{(pid=80143) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 11:05:36.213610 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-1fd3a0c0-8ad4-43ef-b9fc-a591f1022ffe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 11:05:36.280759 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1301/2631] 10.222.0.25 () {58 vars in 958 bytes} [Sun Aug 7 11:05:36 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 07 11:05:36.291958 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b476d404-a4a1-44fb-8890-d130b0172f6d admin admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 11:05:36.293640 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.fernet_utils [None req-b476d404-a4a1-44fb-8890-d130b0172f6d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80144) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 11:05:36.356609 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b476d404-a4a1-44fb-8890-d130b0172f6d admin admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 11:05:36.360839 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fc8e146-550f-4d01-9e03-652b04337492 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88dbd6dfd1164994a3e16da553561910', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6de1f3cac02c4bcdb05dd2337176033d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 11:05:36.361459 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fc8e146-550f-4d01-9e03-652b04337492 service cinder] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 11:05:36.361739 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fc8e146-550f-4d01-9e03-652b04337492 service cinder] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 11:05:36.362012 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fc8e146-550f-4d01-9e03-652b04337492 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 11:05:36.372533 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fc8e146-550f-4d01-9e03-652b04337492 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 11:05:36.374202 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fc8e146-550f-4d01-9e03-652b04337492 service cinder] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 11:05:36.384763 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 1331/2632] 10.222.0.25 () {60 vars in 1365 bytes} [Sun Aug 7 11:05:36 2022] GET /identity/v3/auth/tokens => generated 3317 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 07 11:05:36.778499 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fd3a0c0-8ad4-43ef-b9fc-a591f1022ffe None None] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 11:05:36.807652 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 11:05:36.840631 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fd3a0c0-8ad4-43ef-b9fc-a591f1022ffe None None] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 11:05:36.845048 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-173f33ae-d7df-4778-bc50-06219eab5dc7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5ac47e3b09f43d19a23a0f457fe9e23', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 11:05:36.845669 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-173f33ae-d7df-4778-bc50-06219eab5dc7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 11:05:36.845945 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-173f33ae-d7df-4778-bc50-06219eab5dc7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 11:05:36.846214 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-173f33ae-d7df-4778-bc50-06219eab5dc7 admin admin] PATH_INFO: `/v3/users/84ad97d05cc14e88902772e8157df2a3` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 11:05:36.847457 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-173f33ae-d7df-4778-bc50-06219eab5dc7 admin admin] RBAC: Authorizing `identity:delete_user(user_id=84ad97d05cc14e88902772e8157df2a3)` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 11:05:36.849346 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-173f33ae-d7df-4778-bc50-06219eab5dc7 admin admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 11:05:36.896316 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-173f33ae-d7df-4778-bc50-06219eab5dc7 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '84ad97d05cc14e88902772e8157df2a3'} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 11:05:36.903373 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-173f33ae-d7df-4778-bc50-06219eab5dc7 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '84ad97d05cc14e88902772e8157df2a3'} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 11:05:36.915224 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-173f33ae-d7df-4778-bc50-06219eab5dc7 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '84ad97d05cc14e88902772e8157df2a3'} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 11:05:36.915898 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-173f33ae-d7df-4778-bc50-06219eab5dc7 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '84ad97d05cc14e88902772e8157df2a3'} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 11:05:36.923282 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-173f33ae-d7df-4778-bc50-06219eab5dc7 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '84ad97d05cc14e88902772e8157df2a3'} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 11:05:36.929692 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1302/2633] 10.222.0.25 () {58 vars in 1224 bytes} [Sun Aug 7 11:05:36 2022] DELETE /identity/v3/users/84ad97d05cc14e88902772e8157df2a3 => generated 0 bytes in 152 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 07 11:05:36.934341 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af312b66-9446-41b1-9452-0c1419e35420 None None] REQUEST_METHOD: `POST` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 11:05:36.934635 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af312b66-9446-41b1-9452-0c1419e35420 None None] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 11:05:36.934945 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af312b66-9446-41b1-9452-0c1419e35420 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 11:05:36.948825 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 11:05:36.954876 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.auth.core [None req-af312b66-9446-41b1-9452-0c1419e35420 None None] MFA Rules not processed for user `bac4e6b76fe64217ac8fa44e71cffe6a`. Rule list: `[]` (Enabled: `True`). {{(pid=80144) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 11:05:36.956167 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.fernet_utils [None req-af312b66-9446-41b1-9452-0c1419e35420 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80144) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 11:05:37.022117 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 1332/2634] 10.222.0.25 () {58 vars in 958 bytes} [Sun Aug 7 11:05:36 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 07 11:05:37.032213 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-173f33ae-d7df-4778-bc50-06219eab5dc7 admin admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 11:05:37.033667 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-173f33ae-d7df-4778-bc50-06219eab5dc7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 11:05:37.090411 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-173f33ae-d7df-4778-bc50-06219eab5dc7 admin admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 11:05:37.094157 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad6f78fb-2af1-4467-9eda-5ac934e1e59f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be1283c9bf47424c9aa3ad27cab03027', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6de1f3cac02c4bcdb05dd2337176033d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 11:05:37.094756 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad6f78fb-2af1-4467-9eda-5ac934e1e59f service neutron] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 11:05:37.095085 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad6f78fb-2af1-4467-9eda-5ac934e1e59f service neutron] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 11:05:37.095366 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad6f78fb-2af1-4467-9eda-5ac934e1e59f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 11:05:37.104557 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad6f78fb-2af1-4467-9eda-5ac934e1e59f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 11:05:37.105976 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad6f78fb-2af1-4467-9eda-5ac934e1e59f service neutron] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 11:05:37.116261 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1303/2635] 10.222.0.25 () {60 vars in 1365 bytes} [Sun Aug 7 11:05:37 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 07 11:05:37.385522 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af312b66-9446-41b1-9452-0c1419e35420 None None] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 11:05:37.387028 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.fernet_utils [None req-af312b66-9446-41b1-9452-0c1419e35420 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80144) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 11:05:37.400045 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af312b66-9446-41b1-9452-0c1419e35420 None None] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 11:05:37.404795 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b6a838f-5ec9-4477-a932-0d5c5f9f36b2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5ac47e3b09f43d19a23a0f457fe9e23', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 11:05:37.405458 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b6a838f-5ec9-4477-a932-0d5c5f9f36b2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 11:05:37.405735 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b6a838f-5ec9-4477-a932-0d5c5f9f36b2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 11:05:37.406007 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b6a838f-5ec9-4477-a932-0d5c5f9f36b2 admin admin] PATH_INFO: `/v3/projects/a73187620ecd4be78ab17e15d0807334` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 11:05:37.407265 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b6a838f-5ec9-4477-a932-0d5c5f9f36b2 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a73187620ecd4be78ab17e15d0807334)` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 11:05:37.412881 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b6a838f-5ec9-4477-a932-0d5c5f9f36b2 admin admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 11:05:37.460293 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.notifications [None req-3b6a838f-5ec9-4477-a932-0d5c5f9f36b2 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a73187620ecd4be78ab17e15d0807334', 'request_id': 'req-3b6a838f-5ec9-4477-a932-0d5c5f9f36b2'} {{(pid=80144) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 11:05:37.465852 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.notifications [None req-3b6a838f-5ec9-4477-a932-0d5c5f9f36b2 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a73187620ecd4be78ab17e15d0807334', 'request_id': 'req-3b6a838f-5ec9-4477-a932-0d5c5f9f36b2'} {{(pid=80144) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 11:05:37.474043 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.notifications [None req-3b6a838f-5ec9-4477-a932-0d5c5f9f36b2 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a73187620ecd4be78ab17e15d0807334', 'request_id': 'req-3b6a838f-5ec9-4477-a932-0d5c5f9f36b2'} {{(pid=80144) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 11:05:37.485427 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.notifications [None req-3b6a838f-5ec9-4477-a932-0d5c5f9f36b2 admin admin] The token cache is being invalidate because project a73187620ecd4be78ab17e15d0807334 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80144) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 07 11:05:37.485720 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.notifications [None req-3b6a838f-5ec9-4477-a932-0d5c5f9f36b2 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80144) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 11:05:37.486363 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.notifications [None req-3b6a838f-5ec9-4477-a932-0d5c5f9f36b2 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80144) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 11:05:37.489074 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 1333/2636] 10.222.0.25 () {58 vars in 1233 bytes} [Sun Aug 7 11:05:37 2022] DELETE /identity/v3/projects/a73187620ecd4be78ab17e15d0807334 => generated 0 bytes in 104 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 07 11:05:37.493094 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad6f78fb-2af1-4467-9eda-5ac934e1e59f service neutron] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 11:05:37.502968 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad6f78fb-2af1-4467-9eda-5ac934e1e59f service neutron] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 11:05:37.546769 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 11:05:37.560957 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37dd1250-5284-4fa3-9486-e702ed2e675e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5ac47e3b09f43d19a23a0f457fe9e23', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 11:05:37.561528 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37dd1250-5284-4fa3-9486-e702ed2e675e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 11:05:37.561817 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37dd1250-5284-4fa3-9486-e702ed2e675e admin admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 11:05:37.562095 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37dd1250-5284-4fa3-9486-e702ed2e675e admin admin] PATH_INFO: `/v3/users/de64ccfe4b6b41a083d663a98c0ed7a9` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 11:05:37.563257 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37dd1250-5284-4fa3-9486-e702ed2e675e admin admin] RBAC: Authorizing `identity:delete_user(user_id=de64ccfe4b6b41a083d663a98c0ed7a9)` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 11:05:37.565199 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37dd1250-5284-4fa3-9486-e702ed2e675e admin admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 11:05:37.614459 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-37dd1250-5284-4fa3-9486-e702ed2e675e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'de64ccfe4b6b41a083d663a98c0ed7a9'} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 11:05:37.621344 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-37dd1250-5284-4fa3-9486-e702ed2e675e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'de64ccfe4b6b41a083d663a98c0ed7a9'} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 11:05:37.633139 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-37dd1250-5284-4fa3-9486-e702ed2e675e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'de64ccfe4b6b41a083d663a98c0ed7a9'} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 11:05:37.633901 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-37dd1250-5284-4fa3-9486-e702ed2e675e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'de64ccfe4b6b41a083d663a98c0ed7a9'} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 11:05:37.640966 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-37dd1250-5284-4fa3-9486-e702ed2e675e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'de64ccfe4b6b41a083d663a98c0ed7a9'} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 11:05:37.647939 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1304/2637] 10.222.0.25 () {58 vars in 1224 bytes} [Sun Aug 7 11:05:37 2022] DELETE /identity/v3/users/de64ccfe4b6b41a083d663a98c0ed7a9 => generated 0 bytes in 155 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 07 11:05:37.936745 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b6a838f-5ec9-4477-a932-0d5c5f9f36b2 admin admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 11:05:37.938357 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.fernet_utils [None req-3b6a838f-5ec9-4477-a932-0d5c5f9f36b2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80144) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 11:05:37.952408 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b6a838f-5ec9-4477-a932-0d5c5f9f36b2 admin admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 11:05:38.028158 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2ada4c3-e9a0-4e79-bf4e-9008df605b11 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5ac47e3b09f43d19a23a0f457fe9e23', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 11:05:38.028804 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2ada4c3-e9a0-4e79-bf4e-9008df605b11 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 11:05:38.029117 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2ada4c3-e9a0-4e79-bf4e-9008df605b11 admin admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 11:05:38.029467 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2ada4c3-e9a0-4e79-bf4e-9008df605b11 admin admin] PATH_INFO: `/v3/projects/e2f1e70d74b94091b3504a72b06feb9b` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 11:05:38.031080 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2ada4c3-e9a0-4e79-bf4e-9008df605b11 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e2f1e70d74b94091b3504a72b06feb9b)` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 11:05:38.032978 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2ada4c3-e9a0-4e79-bf4e-9008df605b11 admin admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 11:05:38.068224 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 11:05:38.081571 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.notifications [None req-b2ada4c3-e9a0-4e79-bf4e-9008df605b11 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e2f1e70d74b94091b3504a72b06feb9b', 'request_id': 'req-b2ada4c3-e9a0-4e79-bf4e-9008df605b11'} {{(pid=80144) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 11:05:38.087083 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.notifications [None req-b2ada4c3-e9a0-4e79-bf4e-9008df605b11 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e2f1e70d74b94091b3504a72b06feb9b', 'request_id': 'req-b2ada4c3-e9a0-4e79-bf4e-9008df605b11'} {{(pid=80144) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 11:05:38.094133 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.notifications [None req-b2ada4c3-e9a0-4e79-bf4e-9008df605b11 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e2f1e70d74b94091b3504a72b06feb9b', 'request_id': 'req-b2ada4c3-e9a0-4e79-bf4e-9008df605b11'} {{(pid=80144) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 11:05:38.105939 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.notifications [None req-b2ada4c3-e9a0-4e79-bf4e-9008df605b11 admin admin] The token cache is being invalidate because project e2f1e70d74b94091b3504a72b06feb9b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80144) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 07 11:05:38.106227 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.notifications [None req-b2ada4c3-e9a0-4e79-bf4e-9008df605b11 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80144) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 11:05:38.106878 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.notifications [None req-b2ada4c3-e9a0-4e79-bf4e-9008df605b11 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80144) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 11:05:38.108383 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 1334/2638] 10.222.0.25 () {58 vars in 1233 bytes} [Sun Aug 7 11:05:37 2022] DELETE /identity/v3/projects/e2f1e70d74b94091b3504a72b06feb9b => generated 0 bytes in 172 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 07 11:05:38.114495 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdd7ae9c-296e-4db0-be68-500e1754a774 None None] REQUEST_METHOD: `POST` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 11:05:38.114992 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdd7ae9c-296e-4db0-be68-500e1754a774 None None] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 11:05:38.115104 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdd7ae9c-296e-4db0-be68-500e1754a774 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 11:05:38.134411 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.auth.core [None req-bdd7ae9c-296e-4db0-be68-500e1754a774 None None] MFA Rules not processed for user `bac4e6b76fe64217ac8fa44e71cffe6a`. Rule list: `[]` (Enabled: `True`). {{(pid=80143) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 11:05:38.135636 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-bdd7ae9c-296e-4db0-be68-500e1754a774 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 11:05:38.194943 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1305/2639] 10.222.0.25 () {58 vars in 958 bytes} [Sun Aug 7 11:05:38 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 07 11:05:38.202385 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fc1d2d8-e5c9-49ce-8d48-92c820ab4f9e None None] REQUEST_METHOD: `POST` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 11:05:38.202670 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fc1d2d8-e5c9-49ce-8d48-92c820ab4f9e None None] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 11:05:38.202969 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fc1d2d8-e5c9-49ce-8d48-92c820ab4f9e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 11:05:38.224064 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.auth.core [None req-6fc1d2d8-e5c9-49ce-8d48-92c820ab4f9e None None] MFA Rules not processed for user `bac4e6b76fe64217ac8fa44e71cffe6a`. Rule list: `[]` (Enabled: `True`). {{(pid=80144) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 11:05:38.225260 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.fernet_utils [None req-6fc1d2d8-e5c9-49ce-8d48-92c820ab4f9e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80144) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 11:05:38.232991 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 1335/2640] 10.222.0.25 () {58 vars in 958 bytes} [Sun Aug 7 11:05:38 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 07 11:05:38.237624 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdd7ae9c-296e-4db0-be68-500e1754a774 None None] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 11:05:38.247487 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdd7ae9c-296e-4db0-be68-500e1754a774 None None] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 11:05:38.252029 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c84ccdf1-d287-44e1-9605-fe8f345404f4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5ac47e3b09f43d19a23a0f457fe9e23', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 11:05:38.252629 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c84ccdf1-d287-44e1-9605-fe8f345404f4 admin admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 11:05:38.253401 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c84ccdf1-d287-44e1-9605-fe8f345404f4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 11:05:38.253675 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c84ccdf1-d287-44e1-9605-fe8f345404f4 admin admin] PATH_INFO: `/v3/domains` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 11:05:38.254318 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c84ccdf1-d287-44e1-9605-fe8f345404f4 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80143) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 11:05:38.254621 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c84ccdf1-d287-44e1-9605-fe8f345404f4 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 11:05:38.256258 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c84ccdf1-d287-44e1-9605-fe8f345404f4 admin admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 11:05:38.267497 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1306/2641] 10.222.0.25 () {58 vars in 1153 bytes} [Sun Aug 7 11:05:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 11:05:38.272522 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fc1d2d8-e5c9-49ce-8d48-92c820ab4f9e None None] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 11:05:38.281522 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fc1d2d8-e5c9-49ce-8d48-92c820ab4f9e None None] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 11:05:38.286520 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f64612a0-918e-46af-b48b-64a2d42cedfb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5ac47e3b09f43d19a23a0f457fe9e23', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 11:05:38.287136 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f64612a0-918e-46af-b48b-64a2d42cedfb admin admin] REQUEST_METHOD: `POST` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 11:05:38.287413 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f64612a0-918e-46af-b48b-64a2d42cedfb admin admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 11:05:38.287684 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f64612a0-918e-46af-b48b-64a2d42cedfb admin admin] PATH_INFO: `/v3/projects` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 11:05:38.288300 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f64612a0-918e-46af-b48b-64a2d42cedfb admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 11:05:38.290152 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f64612a0-918e-46af-b48b-64a2d42cedfb admin admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 11:05:38.318140 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 1336/2642] 10.222.0.25 () {60 vars in 1153 bytes} [Sun Aug 7 11:05:38 2022] POST /identity/v3/projects => generated 385 bytes in 46 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 07 11:05:38.322301 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c84ccdf1-d287-44e1-9605-fe8f345404f4 admin admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 11:05:38.332831 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c84ccdf1-d287-44e1-9605-fe8f345404f4 admin admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 11:05:38.382749 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 11:05:38.390425 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c1f8b38-043f-41b0-9900-6d51c033e740 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5ac47e3b09f43d19a23a0f457fe9e23', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 11:05:38.391263 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c1f8b38-043f-41b0-9900-6d51c033e740 admin admin] REQUEST_METHOD: `POST` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 11:05:38.391715 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c1f8b38-043f-41b0-9900-6d51c033e740 admin admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 11:05:38.392040 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c1f8b38-043f-41b0-9900-6d51c033e740 admin admin] PATH_INFO: `/v3/users` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 11:05:38.392813 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c1f8b38-043f-41b0-9900-6d51c033e740 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 11:05:38.395386 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c1f8b38-043f-41b0-9900-6d51c033e740 admin admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 11:05:38.418393 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1307/2643] 10.222.0.25 () {60 vars in 1144 bytes} [Sun Aug 7 11:05:38 2022] POST /identity/v3/users => generated 293 bytes in 94 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 07 11:05:38.421279 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f64612a0-918e-46af-b48b-64a2d42cedfb admin admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 11:05:38.431570 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f64612a0-918e-46af-b48b-64a2d42cedfb admin admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 11:05:38.436129 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a283197-0d71-4b16-93b4-d90dd6e06ac9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5ac47e3b09f43d19a23a0f457fe9e23', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 11:05:38.436747 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a283197-0d71-4b16-93b4-d90dd6e06ac9 admin admin] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 11:05:38.437023 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a283197-0d71-4b16-93b4-d90dd6e06ac9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 11:05:38.437295 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a283197-0d71-4b16-93b4-d90dd6e06ac9 admin admin] PATH_INFO: `/v3/roles` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 11:05:38.437935 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a283197-0d71-4b16-93b4-d90dd6e06ac9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 11:05:38.439572 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a283197-0d71-4b16-93b4-d90dd6e06ac9 admin admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 11:05:38.449923 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 1337/2644] 10.222.0.25 () {58 vars in 1122 bytes} [Sun Aug 7 11:05:38 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 07 11:05:38.454094 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c1f8b38-043f-41b0-9900-6d51c033e740 admin admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 11:05:38.464882 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c1f8b38-043f-41b0-9900-6d51c033e740 admin admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 11:05:38.469485 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82108882-335b-414e-8198-d3c421a51ea1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5ac47e3b09f43d19a23a0f457fe9e23', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 11:05:38.470074 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82108882-335b-414e-8198-d3c421a51ea1 admin admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 11:05:38.470349 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82108882-335b-414e-8198-d3c421a51ea1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 11:05:38.470616 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82108882-335b-414e-8198-d3c421a51ea1 admin admin] PATH_INFO: `/v3/roles` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 11:05:38.471289 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82108882-335b-414e-8198-d3c421a51ea1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 11:05:38.473010 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82108882-335b-414e-8198-d3c421a51ea1 admin admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 11:05:38.484242 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1308/2645] 10.222.0.25 () {58 vars in 1122 bytes} [Sun Aug 7 11:05:38 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 07 11:05:38.488410 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a283197-0d71-4b16-93b4-d90dd6e06ac9 admin admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 11:05:38.498330 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a283197-0d71-4b16-93b4-d90dd6e06ac9 admin admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 11:05:38.502839 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a219687b-e7a2-456c-bbc8-000e8ce00fbb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5ac47e3b09f43d19a23a0f457fe9e23', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 11:05:38.504030 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a219687b-e7a2-456c-bbc8-000e8ce00fbb admin admin] REQUEST_METHOD: `PUT` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 11:05:38.504308 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a219687b-e7a2-456c-bbc8-000e8ce00fbb admin admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 11:05:38.504579 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a219687b-e7a2-456c-bbc8-000e8ce00fbb admin admin] PATH_INFO: `/v3/projects/575a9f90c7674a0faea1b812e8ac4f71/users/2df8bfade7fa4df5a87c1367c6263ec5/roles/b98dfb3b9fee4644af2887a3ac3fd2e6` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 11:05:38.518438 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a219687b-e7a2-456c-bbc8-000e8ce00fbb admin admin] RBAC: Authorizing `identity:create_grant(project_id=575a9f90c7674a0faea1b812e8ac4f71, user_id=2df8bfade7fa4df5a87c1367c6263ec5, role_id=b98dfb3b9fee4644af2887a3ac3fd2e6)` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 11:05:38.521655 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a219687b-e7a2-456c-bbc8-000e8ce00fbb admin admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 11:05:38.533801 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 1338/2646] 10.222.0.25 () {60 vars in 1483 bytes} [Sun Aug 7 11:05:38 2022] PUT /identity/v3/projects/575a9f90c7674a0faea1b812e8ac4f71/users/2df8bfade7fa4df5a87c1367c6263ec5/roles/b98dfb3b9fee4644af2887a3ac3fd2e6 => generated 0 bytes in 46 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 07 11:05:38.544008 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42828d17-087e-4049-8b68-a1cd0d785a1a None None] REQUEST_METHOD: `POST` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 11:05:38.544299 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42828d17-087e-4049-8b68-a1cd0d785a1a None None] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 11:05:38.544578 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42828d17-087e-4049-8b68-a1cd0d785a1a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 11:05:38.573022 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.auth.core [None req-42828d17-087e-4049-8b68-a1cd0d785a1a None None] MFA Rules not processed for user `2df8bfade7fa4df5a87c1367c6263ec5`. Rule list: `[]` (Enabled: `True`). {{(pid=80143) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 11:05:38.574169 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-42828d17-087e-4049-8b68-a1cd0d785a1a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 11:05:38.642141 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1309/2647] 10.222.0.25 () {58 vars in 958 bytes} [Sun Aug 7 11:05:38 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 07 11:05:38.646762 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a219687b-e7a2-456c-bbc8-000e8ce00fbb admin admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 11:05:38.708732 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a219687b-e7a2-456c-bbc8-000e8ce00fbb admin admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 11:05:38.713314 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28cd7e35-87a3-4946-b5a7-34c9c207c9da admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5ac47e3b09f43d19a23a0f457fe9e23', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 11:05:38.713953 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28cd7e35-87a3-4946-b5a7-34c9c207c9da admin admin] REQUEST_METHOD: `POST` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 11:05:38.714232 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28cd7e35-87a3-4946-b5a7-34c9c207c9da admin admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 11:05:38.714505 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28cd7e35-87a3-4946-b5a7-34c9c207c9da admin admin] PATH_INFO: `/v3/projects` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 11:05:38.715149 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28cd7e35-87a3-4946-b5a7-34c9c207c9da admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 11:05:38.717045 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28cd7e35-87a3-4946-b5a7-34c9c207c9da admin admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 11:05:38.743734 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 1339/2648] 10.222.0.25 () {60 vars in 1153 bytes} [Sun Aug 7 11:05:38 2022] POST /identity/v3/projects => generated 387 bytes in 97 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 07 11:05:38.747965 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42828d17-087e-4049-8b68-a1cd0d785a1a None None] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 11:05:38.759077 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42828d17-087e-4049-8b68-a1cd0d785a1a None None] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 11:05:38.814913 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d39caed7-ae21-4c56-b5cb-d4da13f89cce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5ac47e3b09f43d19a23a0f457fe9e23', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 11:05:38.816048 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d39caed7-ae21-4c56-b5cb-d4da13f89cce admin admin] REQUEST_METHOD: `POST` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 11:05:38.816335 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d39caed7-ae21-4c56-b5cb-d4da13f89cce admin admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 11:05:38.816614 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d39caed7-ae21-4c56-b5cb-d4da13f89cce admin admin] PATH_INFO: `/v3/users` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 11:05:38.817230 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d39caed7-ae21-4c56-b5cb-d4da13f89cce admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 11:05:38.819157 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d39caed7-ae21-4c56-b5cb-d4da13f89cce admin admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 11:05:38.841029 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1310/2649] 10.222.0.25 () {60 vars in 1144 bytes} [Sun Aug 7 11:05:38 2022] POST /identity/v3/users => generated 294 bytes in 94 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 07 11:05:38.845097 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28cd7e35-87a3-4946-b5a7-34c9c207c9da admin admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 11:05:38.856195 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28cd7e35-87a3-4946-b5a7-34c9c207c9da admin admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 11:05:38.861274 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41b6d36b-a9e7-47b9-9bc1-d1a988dee251 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5ac47e3b09f43d19a23a0f457fe9e23', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 11:05:38.861874 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41b6d36b-a9e7-47b9-9bc1-d1a988dee251 admin admin] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 11:05:38.862157 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41b6d36b-a9e7-47b9-9bc1-d1a988dee251 admin admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 11:05:38.863051 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41b6d36b-a9e7-47b9-9bc1-d1a988dee251 admin admin] PATH_INFO: `/v3/roles` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 11:05:38.863708 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41b6d36b-a9e7-47b9-9bc1-d1a988dee251 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 11:05:38.865291 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41b6d36b-a9e7-47b9-9bc1-d1a988dee251 admin admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 11:05:38.875863 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 1340/2650] 10.222.0.25 () {58 vars in 1122 bytes} [Sun Aug 7 11:05:38 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 07 11:05:38.880535 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d39caed7-ae21-4c56-b5cb-d4da13f89cce admin admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 11:05:38.890366 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d39caed7-ae21-4c56-b5cb-d4da13f89cce admin admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 11:05:38.895935 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-271b76aa-09b4-4c65-9232-01023c1391d6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5ac47e3b09f43d19a23a0f457fe9e23', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 11:05:38.896532 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-271b76aa-09b4-4c65-9232-01023c1391d6 admin admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 11:05:38.896815 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-271b76aa-09b4-4c65-9232-01023c1391d6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 11:05:38.897089 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-271b76aa-09b4-4c65-9232-01023c1391d6 admin admin] PATH_INFO: `/v3/roles` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 11:05:38.897729 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-271b76aa-09b4-4c65-9232-01023c1391d6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 11:05:38.899547 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-271b76aa-09b4-4c65-9232-01023c1391d6 admin admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 11:05:38.910286 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1311/2651] 10.222.0.25 () {58 vars in 1122 bytes} [Sun Aug 7 11:05:38 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 07 11:05:38.913840 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41b6d36b-a9e7-47b9-9bc1-d1a988dee251 admin admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 11:05:38.923766 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41b6d36b-a9e7-47b9-9bc1-d1a988dee251 admin admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 11:05:38.928373 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ca8e986-169f-4741-9d84-1c79e76de8b2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5ac47e3b09f43d19a23a0f457fe9e23', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 11:05:38.929015 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ca8e986-169f-4741-9d84-1c79e76de8b2 admin admin] REQUEST_METHOD: `PUT` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 11:05:38.929297 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ca8e986-169f-4741-9d84-1c79e76de8b2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 11:05:38.929573 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ca8e986-169f-4741-9d84-1c79e76de8b2 admin admin] PATH_INFO: `/v3/projects/86846851dd8841588845fa41831684ad/users/98f674549c524f949ca06044f3e2b23b/roles/b98dfb3b9fee4644af2887a3ac3fd2e6` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 11:05:38.945831 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ca8e986-169f-4741-9d84-1c79e76de8b2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=86846851dd8841588845fa41831684ad, user_id=98f674549c524f949ca06044f3e2b23b, role_id=b98dfb3b9fee4644af2887a3ac3fd2e6)` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 11:05:38.948499 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ca8e986-169f-4741-9d84-1c79e76de8b2 admin admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 11:05:38.961705 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 1341/2652] 10.222.0.25 () {60 vars in 1483 bytes} [Sun Aug 7 11:05:38 2022] PUT /identity/v3/projects/86846851dd8841588845fa41831684ad/users/98f674549c524f949ca06044f3e2b23b/roles/b98dfb3b9fee4644af2887a3ac3fd2e6 => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 07 11:05:38.972867 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00b55249-2ea9-4f00-8085-5ac0847506e0 None None] REQUEST_METHOD: `POST` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 11:05:38.973155 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00b55249-2ea9-4f00-8085-5ac0847506e0 None None] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 11:05:38.973430 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00b55249-2ea9-4f00-8085-5ac0847506e0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 11:05:39.001207 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.auth.core [None req-00b55249-2ea9-4f00-8085-5ac0847506e0 None None] MFA Rules not processed for user `98f674549c524f949ca06044f3e2b23b`. Rule list: `[]` (Enabled: `True`). {{(pid=80143) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 11:05:39.002399 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-00b55249-2ea9-4f00-8085-5ac0847506e0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 11:05:39.033151 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 11:05:39.072642 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1312/2653] 10.222.0.25 () {58 vars in 958 bytes} [Sun Aug 7 11:05:38 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 07 11:05:39.077166 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ca8e986-169f-4741-9d84-1c79e76de8b2 admin admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 11:05:39.087140 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=90,keystone:DELETE=5,keystone:INSERT=5 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 11:05:39.145087 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ca8e986-169f-4741-9d84-1c79e76de8b2 admin admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 11:05:39.150552 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f6b7f10-f4bd-48eb-8a1a-9732730f1a9a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5ac47e3b09f43d19a23a0f457fe9e23', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 11:05:39.151232 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f6b7f10-f4bd-48eb-8a1a-9732730f1a9a admin admin] REQUEST_METHOD: `POST` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 11:05:39.151561 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f6b7f10-f4bd-48eb-8a1a-9732730f1a9a admin admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 11:05:39.151867 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f6b7f10-f4bd-48eb-8a1a-9732730f1a9a admin admin] PATH_INFO: `/v3/projects` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 11:05:39.152510 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f6b7f10-f4bd-48eb-8a1a-9732730f1a9a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 11:05:39.154585 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f6b7f10-f4bd-48eb-8a1a-9732730f1a9a admin admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 11:05:39.190220 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 1342/2654] 10.222.0.25 () {60 vars in 1153 bytes} [Sun Aug 7 11:05:39 2022] POST /identity/v3/projects => generated 387 bytes in 114 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 07 11:05:39.198786 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00b55249-2ea9-4f00-8085-5ac0847506e0 None None] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 11:05:39.212134 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00b55249-2ea9-4f00-8085-5ac0847506e0 None None] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 11:05:39.269728 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39df937a-8b9d-42c3-ba56-56022e412d80 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5ac47e3b09f43d19a23a0f457fe9e23', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 11:05:39.270342 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39df937a-8b9d-42c3-ba56-56022e412d80 admin admin] REQUEST_METHOD: `POST` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 11:05:39.270621 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39df937a-8b9d-42c3-ba56-56022e412d80 admin admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 11:05:39.270917 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39df937a-8b9d-42c3-ba56-56022e412d80 admin admin] PATH_INFO: `/v3/users` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 11:05:39.271556 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39df937a-8b9d-42c3-ba56-56022e412d80 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 11:05:39.273487 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39df937a-8b9d-42c3-ba56-56022e412d80 admin admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 11:05:39.293615 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1313/2655] 10.222.0.25 () {60 vars in 1144 bytes} [Sun Aug 7 11:05:39 2022] POST /identity/v3/users => generated 300 bytes in 95 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 07 11:05:39.297664 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f6b7f10-f4bd-48eb-8a1a-9732730f1a9a admin admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 11:05:39.307905 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f6b7f10-f4bd-48eb-8a1a-9732730f1a9a admin admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 11:05:39.312406 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2188741-e9f1-462c-856e-fa0f3a4f366b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5ac47e3b09f43d19a23a0f457fe9e23', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 11:05:39.313000 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2188741-e9f1-462c-856e-fa0f3a4f366b admin admin] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 11:05:39.313275 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2188741-e9f1-462c-856e-fa0f3a4f366b admin admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 11:05:39.313542 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2188741-e9f1-462c-856e-fa0f3a4f366b admin admin] PATH_INFO: `/v3/roles` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 11:05:39.314172 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2188741-e9f1-462c-856e-fa0f3a4f366b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 11:05:39.316574 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2188741-e9f1-462c-856e-fa0f3a4f366b admin admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 11:05:39.327140 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 1343/2656] 10.222.0.25 () {58 vars in 1122 bytes} [Sun Aug 7 11:05:39 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 07 11:05:39.331474 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39df937a-8b9d-42c3-ba56-56022e412d80 admin admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 11:05:39.341393 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39df937a-8b9d-42c3-ba56-56022e412d80 admin admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 11:05:39.346032 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd5c266e-e981-4710-b821-33af0fd650ec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5ac47e3b09f43d19a23a0f457fe9e23', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 11:05:39.346712 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd5c266e-e981-4710-b821-33af0fd650ec admin admin] REQUEST_METHOD: `PUT` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 11:05:39.347095 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd5c266e-e981-4710-b821-33af0fd650ec admin admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 11:05:39.347377 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd5c266e-e981-4710-b821-33af0fd650ec admin admin] PATH_INFO: `/v3/projects/d121077dda014d9aa4b0409aa1869b85/users/b9779bac8dcc460d9990e98733ac72eb/roles/81b1f1bcee2a473088fbbd327c1c5b23` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 11:05:39.362104 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd5c266e-e981-4710-b821-33af0fd650ec admin admin] RBAC: Authorizing `identity:create_grant(project_id=d121077dda014d9aa4b0409aa1869b85, user_id=b9779bac8dcc460d9990e98733ac72eb, role_id=81b1f1bcee2a473088fbbd327c1c5b23)` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 11:05:39.365386 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd5c266e-e981-4710-b821-33af0fd650ec admin admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 11:05:39.377855 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1314/2657] 10.222.0.25 () {60 vars in 1483 bytes} [Sun Aug 7 11:05:39 2022] PUT /identity/v3/projects/d121077dda014d9aa4b0409aa1869b85/users/b9779bac8dcc460d9990e98733ac72eb/roles/81b1f1bcee2a473088fbbd327c1c5b23 => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 07 11:05:39.382458 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2188741-e9f1-462c-856e-fa0f3a4f366b admin admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 11:05:39.392345 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2188741-e9f1-462c-856e-fa0f3a4f366b admin admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 11:05:39.449515 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d02ee5a-0550-413a-a4f6-40d980e5cb71 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5ac47e3b09f43d19a23a0f457fe9e23', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 11:05:39.450110 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d02ee5a-0550-413a-a4f6-40d980e5cb71 admin admin] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 11:05:39.450394 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d02ee5a-0550-413a-a4f6-40d980e5cb71 admin admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 11:05:39.450667 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d02ee5a-0550-413a-a4f6-40d980e5cb71 admin admin] PATH_INFO: `/v3/roles` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 11:05:39.451352 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d02ee5a-0550-413a-a4f6-40d980e5cb71 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 11:05:39.452932 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d02ee5a-0550-413a-a4f6-40d980e5cb71 admin admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 11:05:39.466580 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 1344/2658] 10.222.0.25 () {58 vars in 1122 bytes} [Sun Aug 7 11:05:39 2022] GET /identity/v3/roles => generated 1420 bytes in 85 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 07 11:05:39.470938 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd5c266e-e981-4710-b821-33af0fd650ec admin admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 11:05:39.481092 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd5c266e-e981-4710-b821-33af0fd650ec admin admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 11:05:39.485844 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-469b6218-61ab-4193-aa4b-410490fbbd4d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5ac47e3b09f43d19a23a0f457fe9e23', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 11:05:39.486485 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-469b6218-61ab-4193-aa4b-410490fbbd4d admin admin] REQUEST_METHOD: `PUT` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 11:05:39.486767 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-469b6218-61ab-4193-aa4b-410490fbbd4d admin admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 11:05:39.487135 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-469b6218-61ab-4193-aa4b-410490fbbd4d admin admin] PATH_INFO: `/v3/projects/d121077dda014d9aa4b0409aa1869b85/users/b9779bac8dcc460d9990e98733ac72eb/roles/81b1f1bcee2a473088fbbd327c1c5b23` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 11:05:39.489564 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-469b6218-61ab-4193-aa4b-410490fbbd4d admin admin] RBAC: Authorizing `identity:create_grant(project_id=d121077dda014d9aa4b0409aa1869b85, user_id=b9779bac8dcc460d9990e98733ac72eb, role_id=81b1f1bcee2a473088fbbd327c1c5b23)` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 11:05:39.492269 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-469b6218-61ab-4193-aa4b-410490fbbd4d admin admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 11:05:39.507352 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1315/2659] 10.222.0.25 () {60 vars in 1483 bytes} [Sun Aug 7 11:05:39 2022] PUT /identity/v3/projects/d121077dda014d9aa4b0409aa1869b85/users/b9779bac8dcc460d9990e98733ac72eb/roles/81b1f1bcee2a473088fbbd327c1c5b23 => generated 0 bytes in 37 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 07 11:05:39.521923 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-530f17da-77e8-43d3-9f06-55a8fe9e7c59 None None] REQUEST_METHOD: `POST` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 11:05:39.522209 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-530f17da-77e8-43d3-9f06-55a8fe9e7c59 None None] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 11:05:39.522485 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-530f17da-77e8-43d3-9f06-55a8fe9e7c59 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 11:05:39.549571 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.auth.core [None req-530f17da-77e8-43d3-9f06-55a8fe9e7c59 None None] MFA Rules not processed for user `b9779bac8dcc460d9990e98733ac72eb`. Rule list: `[]` (Enabled: `True`). {{(pid=80144) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 11:05:39.551421 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.fernet_utils [None req-530f17da-77e8-43d3-9f06-55a8fe9e7c59 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80144) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 11:05:39.617537 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 1345/2660] 10.222.0.25 () {58 vars in 958 bytes} [Sun Aug 7 11:05:39 2022] POST /identity/v3/auth/tokens => generated 3331 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 07 11:05:39.629336 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-469b6218-61ab-4193-aa4b-410490fbbd4d admin admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 11:05:39.631100 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-469b6218-61ab-4193-aa4b-410490fbbd4d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 11:05:39.687402 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-469b6218-61ab-4193-aa4b-410490fbbd4d admin admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 11:05:39.691970 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d06d377-b25e-494b-bb8b-cf34476e9986 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88dbd6dfd1164994a3e16da553561910', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6de1f3cac02c4bcdb05dd2337176033d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 11:05:39.692573 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d06d377-b25e-494b-bb8b-cf34476e9986 service cinder] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 11:05:39.692853 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d06d377-b25e-494b-bb8b-cf34476e9986 service cinder] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 11:05:39.693126 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d06d377-b25e-494b-bb8b-cf34476e9986 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 11:05:39.702072 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d06d377-b25e-494b-bb8b-cf34476e9986 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 11:05:39.703518 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d06d377-b25e-494b-bb8b-cf34476e9986 service cinder] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 11:05:39.712874 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1316/2661] 10.222.0.25 () {60 vars in 1365 bytes} [Sun Aug 7 11:05:39 2022] GET /identity/v3/auth/tokens => generated 3331 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 07 11:05:40.202718 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-530f17da-77e8-43d3-9f06-55a8fe9e7c59 None None] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 11:05:40.204115 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.fernet_utils [None req-530f17da-77e8-43d3-9f06-55a8fe9e7c59 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80144) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 11:05:40.211543 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 11:05:40.218423 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-530f17da-77e8-43d3-9f06-55a8fe9e7c59 None None] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 11:05:40.223312 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6866627c-bf5b-4bb8-81fb-517c69ddc149 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88dbd6dfd1164994a3e16da553561910', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6de1f3cac02c4bcdb05dd2337176033d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 11:05:40.224920 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6866627c-bf5b-4bb8-81fb-517c69ddc149 service cinder] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 11:05:40.225200 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6866627c-bf5b-4bb8-81fb-517c69ddc149 service cinder] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 11:05:40.225474 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6866627c-bf5b-4bb8-81fb-517c69ddc149 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 11:05:40.280533 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6866627c-bf5b-4bb8-81fb-517c69ddc149 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 11:05:40.281942 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6866627c-bf5b-4bb8-81fb-517c69ddc149 service cinder] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 11:05:40.291167 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 1346/2662] 10.222.0.25 () {60 vars in 1365 bytes} [Sun Aug 7 11:05:40 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 07 11:05:46.391144 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d06d377-b25e-494b-bb8b-cf34476e9986 service cinder] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 11:05:46.400357 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d06d377-b25e-494b-bb8b-cf34476e9986 service cinder] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 11:05:46.403963 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbb18c47-8fea-4a3c-94ec-e7233df6158c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88dbd6dfd1164994a3e16da553561910', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6de1f3cac02c4bcdb05dd2337176033d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 11:05:46.404559 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbb18c47-8fea-4a3c-94ec-e7233df6158c service cinder] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 11:05:46.404840 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbb18c47-8fea-4a3c-94ec-e7233df6158c service cinder] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 11:05:46.405115 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbb18c47-8fea-4a3c-94ec-e7233df6158c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 11:05:46.430125 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 11:05:46.464202 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbb18c47-8fea-4a3c-94ec-e7233df6158c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 11:05:46.465611 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbb18c47-8fea-4a3c-94ec-e7233df6158c service cinder] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 11:05:46.474599 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1317/2663] 10.222.0.25 () {60 vars in 1365 bytes} [Sun Aug 7 11:05:46 2022] GET /identity/v3/auth/tokens => generated 3264 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 07 11:05:48.302110 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6866627c-bf5b-4bb8-81fb-517c69ddc149 service cinder] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 11:05:48.303816 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.fernet_utils [None req-6866627c-bf5b-4bb8-81fb-517c69ddc149 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80144) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 11:05:48.373348 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6866627c-bf5b-4bb8-81fb-517c69ddc149 service cinder] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 11:05:48.378051 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6245160-5308-495c-8411-f6cfeccf6924 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5ac47e3b09f43d19a23a0f457fe9e23', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 11:05:48.379646 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6245160-5308-495c-8411-f6cfeccf6924 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 11:05:48.379924 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6245160-5308-495c-8411-f6cfeccf6924 admin admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 11:05:48.380194 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6245160-5308-495c-8411-f6cfeccf6924 admin admin] PATH_INFO: `/v3/users/54ce608a2b2d467fb0b3ac19abd41acf` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 11:05:48.381341 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6245160-5308-495c-8411-f6cfeccf6924 admin admin] RBAC: Authorizing `identity:delete_user(user_id=54ce608a2b2d467fb0b3ac19abd41acf)` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 11:05:48.383259 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6245160-5308-495c-8411-f6cfeccf6924 admin admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 11:05:48.433348 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.notifications [None req-e6245160-5308-495c-8411-f6cfeccf6924 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '54ce608a2b2d467fb0b3ac19abd41acf'} {{(pid=80144) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 11:05:48.441671 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.notifications [None req-e6245160-5308-495c-8411-f6cfeccf6924 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '54ce608a2b2d467fb0b3ac19abd41acf'} {{(pid=80144) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 11:05:48.449739 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.notifications [None req-e6245160-5308-495c-8411-f6cfeccf6924 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '54ce608a2b2d467fb0b3ac19abd41acf'} {{(pid=80144) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 11:05:48.450407 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.notifications [None req-e6245160-5308-495c-8411-f6cfeccf6924 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '54ce608a2b2d467fb0b3ac19abd41acf'} {{(pid=80144) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 11:05:48.451098 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.notifications [None req-e6245160-5308-495c-8411-f6cfeccf6924 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '54ce608a2b2d467fb0b3ac19abd41acf'} {{(pid=80144) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 11:05:48.469885 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 1347/2664] 10.222.0.25 () {58 vars in 1224 bytes} [Sun Aug 7 11:05:48 2022] DELETE /identity/v3/users/54ce608a2b2d467fb0b3ac19abd41acf => generated 0 bytes in 170 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 07 11:05:48.475658 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ff78b70-1734-4d35-94ed-eeae4dbdef90 None None] REQUEST_METHOD: `POST` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 11:05:48.476189 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ff78b70-1734-4d35-94ed-eeae4dbdef90 None None] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 11:05:48.476189 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ff78b70-1734-4d35-94ed-eeae4dbdef90 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 11:05:48.499042 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.auth.core [None req-8ff78b70-1734-4d35-94ed-eeae4dbdef90 None None] MFA Rules not processed for user `bac4e6b76fe64217ac8fa44e71cffe6a`. Rule list: `[]` (Enabled: `True`). {{(pid=80143) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 11:05:48.500237 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-8ff78b70-1734-4d35-94ed-eeae4dbdef90 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 11:05:48.554594 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6245160-5308-495c-8411-f6cfeccf6924 admin admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 11:05:48.556367 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.fernet_utils [None req-e6245160-5308-495c-8411-f6cfeccf6924 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80144) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 11:05:48.572152 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1318/2665] 10.222.0.25 () {58 vars in 958 bytes} [Sun Aug 7 11:05:48 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 07 11:05:48.580747 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 11:05:48.582371 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ff78b70-1734-4d35-94ed-eeae4dbdef90 None None] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 11:05:48.583914 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-8ff78b70-1734-4d35-94ed-eeae4dbdef90 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 11:05:48.632755 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6245160-5308-495c-8411-f6cfeccf6924 admin admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 11:05:48.637497 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40645270-e063-475e-900d-bf8c31765694 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5ac47e3b09f43d19a23a0f457fe9e23', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 11:05:48.638127 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40645270-e063-475e-900d-bf8c31765694 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 11:05:48.638412 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40645270-e063-475e-900d-bf8c31765694 admin admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 11:05:48.638691 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40645270-e063-475e-900d-bf8c31765694 admin admin] PATH_INFO: `/v3/users/2df8bfade7fa4df5a87c1367c6263ec5` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 11:05:48.639863 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40645270-e063-475e-900d-bf8c31765694 admin admin] RBAC: Authorizing `identity:delete_user(user_id=2df8bfade7fa4df5a87c1367c6263ec5)` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 11:05:48.641963 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40645270-e063-475e-900d-bf8c31765694 admin admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 11:05:48.644301 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ff78b70-1734-4d35-94ed-eeae4dbdef90 None None] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 11:05:48.648098 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c4c080f-cdbf-4c11-986c-b3882a940c75 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be1283c9bf47424c9aa3ad27cab03027', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6de1f3cac02c4bcdb05dd2337176033d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 11:05:48.648723 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c4c080f-cdbf-4c11-986c-b3882a940c75 service neutron] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 11:05:48.649018 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c4c080f-cdbf-4c11-986c-b3882a940c75 service neutron] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 11:05:48.649288 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c4c080f-cdbf-4c11-986c-b3882a940c75 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 11:05:48.658333 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c4c080f-cdbf-4c11-986c-b3882a940c75 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 11:05:48.659829 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c4c080f-cdbf-4c11-986c-b3882a940c75 service neutron] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 11:05:48.669948 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1319/2666] 10.222.0.25 () {60 vars in 1365 bytes} [Sun Aug 7 11:05:48 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 07 11:05:48.694750 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.notifications [None req-40645270-e063-475e-900d-bf8c31765694 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2df8bfade7fa4df5a87c1367c6263ec5'} {{(pid=80144) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 11:05:48.701902 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.notifications [None req-40645270-e063-475e-900d-bf8c31765694 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2df8bfade7fa4df5a87c1367c6263ec5'} {{(pid=80144) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 11:05:48.709705 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.notifications [None req-40645270-e063-475e-900d-bf8c31765694 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2df8bfade7fa4df5a87c1367c6263ec5'} {{(pid=80144) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 11:05:48.710561 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.notifications [None req-40645270-e063-475e-900d-bf8c31765694 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2df8bfade7fa4df5a87c1367c6263ec5'} {{(pid=80144) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 11:05:48.711363 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.notifications [None req-40645270-e063-475e-900d-bf8c31765694 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2df8bfade7fa4df5a87c1367c6263ec5'} {{(pid=80144) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 11:05:48.730497 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 1348/2667] 10.222.0.25 () {58 vars in 1224 bytes} [Sun Aug 7 11:05:48 2022] DELETE /identity/v3/users/2df8bfade7fa4df5a87c1367c6263ec5 => generated 0 bytes in 176 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 07 11:05:48.735293 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d06f21f-6260-4871-ae43-758748848649 None None] REQUEST_METHOD: `POST` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 11:05:48.735584 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d06f21f-6260-4871-ae43-758748848649 None None] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 11:05:48.735860 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d06f21f-6260-4871-ae43-758748848649 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 11:05:48.755972 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.auth.core [None req-6d06f21f-6260-4871-ae43-758748848649 None None] MFA Rules not processed for user `bac4e6b76fe64217ac8fa44e71cffe6a`. Rule list: `[]` (Enabled: `True`). {{(pid=80143) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 11:05:48.757194 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-6d06f21f-6260-4871-ae43-758748848649 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 11:05:48.818565 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1320/2668] 10.222.0.25 () {58 vars in 958 bytes} [Sun Aug 7 11:05:48 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 07 11:05:48.828878 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40645270-e063-475e-900d-bf8c31765694 admin admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 11:05:48.830230 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.fernet_utils [None req-40645270-e063-475e-900d-bf8c31765694 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80144) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 11:05:48.890084 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40645270-e063-475e-900d-bf8c31765694 admin admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 11:05:48.893904 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-858a7fd4-2fe0-4e85-990c-f0adaf4017c0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be1283c9bf47424c9aa3ad27cab03027', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6de1f3cac02c4bcdb05dd2337176033d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 11:05:48.895058 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-858a7fd4-2fe0-4e85-990c-f0adaf4017c0 service neutron] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 11:05:48.895402 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-858a7fd4-2fe0-4e85-990c-f0adaf4017c0 service neutron] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 11:05:48.895582 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-858a7fd4-2fe0-4e85-990c-f0adaf4017c0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 11:05:48.905177 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-858a7fd4-2fe0-4e85-990c-f0adaf4017c0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 11:05:48.906221 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-858a7fd4-2fe0-4e85-990c-f0adaf4017c0 service neutron] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 11:05:48.915952 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 1349/2669] 10.222.0.25 () {60 vars in 1365 bytes} [Sun Aug 7 11:05:48 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 07 11:05:48.951701 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d06f21f-6260-4871-ae43-758748848649 None None] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 11:05:48.953432 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-6d06f21f-6260-4871-ae43-758748848649 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 11:05:48.967492 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d06f21f-6260-4871-ae43-758748848649 None None] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 11:05:48.973298 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2eb50c92-b818-4c65-b3fe-2af99662e7fd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5ac47e3b09f43d19a23a0f457fe9e23', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 11:05:48.974020 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eb50c92-b818-4c65-b3fe-2af99662e7fd admin admin] REQUEST_METHOD: `DELETE` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 11:05:48.974301 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eb50c92-b818-4c65-b3fe-2af99662e7fd admin admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 11:05:48.974698 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eb50c92-b818-4c65-b3fe-2af99662e7fd admin admin] PATH_INFO: `/v3/projects/3c41ff204b0049a29bf079b3d8725b83` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 11:05:48.976359 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2eb50c92-b818-4c65-b3fe-2af99662e7fd admin admin] RBAC: Authorizing `identity:delete_project(project_id=3c41ff204b0049a29bf079b3d8725b83)` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 11:05:48.978116 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2eb50c92-b818-4c65-b3fe-2af99662e7fd admin admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 11:05:48.981499 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=100 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 11:05:49.031445 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-2eb50c92-b818-4c65-b3fe-2af99662e7fd admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3c41ff204b0049a29bf079b3d8725b83', 'request_id': 'req-2eb50c92-b818-4c65-b3fe-2af99662e7fd'} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 11:05:49.037158 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-2eb50c92-b818-4c65-b3fe-2af99662e7fd admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3c41ff204b0049a29bf079b3d8725b83', 'request_id': 'req-2eb50c92-b818-4c65-b3fe-2af99662e7fd'} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 11:05:49.043634 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-2eb50c92-b818-4c65-b3fe-2af99662e7fd admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3c41ff204b0049a29bf079b3d8725b83', 'request_id': 'req-2eb50c92-b818-4c65-b3fe-2af99662e7fd'} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 11:05:49.057633 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-2eb50c92-b818-4c65-b3fe-2af99662e7fd admin admin] The token cache is being invalidate because project 3c41ff204b0049a29bf079b3d8725b83 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80143) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 07 11:05:49.058063 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-2eb50c92-b818-4c65-b3fe-2af99662e7fd admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 11:05:49.058789 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-2eb50c92-b818-4c65-b3fe-2af99662e7fd admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 11:05:49.060638 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1321/2670] 10.222.0.25 () {58 vars in 1233 bytes} [Sun Aug 7 11:05:48 2022] DELETE /identity/v3/projects/3c41ff204b0049a29bf079b3d8725b83 => generated 0 bytes in 110 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 07 11:05:49.065682 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03844ac1-cb71-43e9-bc38-03013c1000f9 None None] REQUEST_METHOD: `POST` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 11:05:49.066687 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03844ac1-cb71-43e9-bc38-03013c1000f9 None None] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 11:05:49.067076 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03844ac1-cb71-43e9-bc38-03013c1000f9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 11:05:49.093935 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.auth.core [None req-03844ac1-cb71-43e9-bc38-03013c1000f9 None None] MFA Rules not processed for user `bac4e6b76fe64217ac8fa44e71cffe6a`. Rule list: `[]` (Enabled: `True`). {{(pid=80144) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 11:05:49.095269 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.fernet_utils [None req-03844ac1-cb71-43e9-bc38-03013c1000f9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80144) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 11:05:49.135079 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 11:05:49.159362 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 1350/2671] 10.222.0.25 () {58 vars in 958 bytes} [Sun Aug 7 11:05:49 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 07 11:05:49.167107 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca1adb1f-1e30-4060-bc95-c706f97f838e None None] REQUEST_METHOD: `POST` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 11:05:49.167417 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca1adb1f-1e30-4060-bc95-c706f97f838e None None] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 11:05:49.167695 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca1adb1f-1e30-4060-bc95-c706f97f838e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 11:05:49.190178 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.auth.core [None req-ca1adb1f-1e30-4060-bc95-c706f97f838e None None] MFA Rules not processed for user `bac4e6b76fe64217ac8fa44e71cffe6a`. Rule list: `[]` (Enabled: `True`). {{(pid=80143) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 11:05:49.192279 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-ca1adb1f-1e30-4060-bc95-c706f97f838e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 11:05:49.201674 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1322/2672] 10.222.0.25 () {58 vars in 958 bytes} [Sun Aug 7 11:05:49 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 07 11:05:49.203495 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03844ac1-cb71-43e9-bc38-03013c1000f9 None None] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 11:05:49.204935 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.fernet_utils [None req-03844ac1-cb71-43e9-bc38-03013c1000f9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80144) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 11:05:49.205620 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca1adb1f-1e30-4060-bc95-c706f97f838e None None] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 11:05:49.215876 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca1adb1f-1e30-4060-bc95-c706f97f838e None None] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 11:05:49.220836 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9163c38d-c5f6-4d83-887f-857e8a6e2c2b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5ac47e3b09f43d19a23a0f457fe9e23', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 11:05:49.221445 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9163c38d-c5f6-4d83-887f-857e8a6e2c2b admin admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 11:05:49.221719 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9163c38d-c5f6-4d83-887f-857e8a6e2c2b admin admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 11:05:49.222025 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9163c38d-c5f6-4d83-887f-857e8a6e2c2b admin admin] PATH_INFO: `/v3/domains` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 11:05:49.222146 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03844ac1-cb71-43e9-bc38-03013c1000f9 None None] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 11:05:49.223058 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9163c38d-c5f6-4d83-887f-857e8a6e2c2b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80143) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 11:05:49.223363 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9163c38d-c5f6-4d83-887f-857e8a6e2c2b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 11:05:49.224955 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9163c38d-c5f6-4d83-887f-857e8a6e2c2b admin admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 11:05:49.229257 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a600a664-2756-4bce-81cf-60ba11a0ad6d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5ac47e3b09f43d19a23a0f457fe9e23', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 11:05:49.230164 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a600a664-2756-4bce-81cf-60ba11a0ad6d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 11:05:49.231368 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a600a664-2756-4bce-81cf-60ba11a0ad6d admin admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 11:05:49.232023 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a600a664-2756-4bce-81cf-60ba11a0ad6d admin admin] PATH_INFO: `/v3/projects/575a9f90c7674a0faea1b812e8ac4f71` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 11:05:49.233530 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a600a664-2756-4bce-81cf-60ba11a0ad6d admin admin] RBAC: Authorizing `identity:delete_project(project_id=575a9f90c7674a0faea1b812e8ac4f71)` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 11:05:49.236213 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a600a664-2756-4bce-81cf-60ba11a0ad6d admin admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 11:05:49.243154 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1323/2673] 10.222.0.25 () {58 vars in 1153 bytes} [Sun Aug 7 11:05:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 07 11:05:49.244412 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9163c38d-c5f6-4d83-887f-857e8a6e2c2b admin admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 11:05:49.255031 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9163c38d-c5f6-4d83-887f-857e8a6e2c2b admin admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 11:05:49.260262 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f101f938-fe79-4e4b-a2e9-86d9f8100170 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5ac47e3b09f43d19a23a0f457fe9e23', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 11:05:49.260948 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f101f938-fe79-4e4b-a2e9-86d9f8100170 admin admin] REQUEST_METHOD: `POST` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 11:05:49.261308 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f101f938-fe79-4e4b-a2e9-86d9f8100170 admin admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 11:05:49.261664 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f101f938-fe79-4e4b-a2e9-86d9f8100170 admin admin] PATH_INFO: `/v3/projects` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 11:05:49.262358 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f101f938-fe79-4e4b-a2e9-86d9f8100170 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 11:05:49.264297 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f101f938-fe79-4e4b-a2e9-86d9f8100170 admin admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 11:05:49.287719 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.notifications [None req-a600a664-2756-4bce-81cf-60ba11a0ad6d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '575a9f90c7674a0faea1b812e8ac4f71', 'request_id': 'req-a600a664-2756-4bce-81cf-60ba11a0ad6d'} {{(pid=80144) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 11:05:49.293107 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1324/2674] 10.222.0.25 () {60 vars in 1153 bytes} [Sun Aug 7 11:05:49 2022] POST /identity/v3/projects => generated 375 bytes in 49 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 07 11:05:49.293444 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.notifications [None req-a600a664-2756-4bce-81cf-60ba11a0ad6d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '575a9f90c7674a0faea1b812e8ac4f71', 'request_id': 'req-a600a664-2756-4bce-81cf-60ba11a0ad6d'} {{(pid=80144) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 11:05:49.297026 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f101f938-fe79-4e4b-a2e9-86d9f8100170 admin admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 11:05:49.301623 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.notifications [None req-a600a664-2756-4bce-81cf-60ba11a0ad6d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '575a9f90c7674a0faea1b812e8ac4f71', 'request_id': 'req-a600a664-2756-4bce-81cf-60ba11a0ad6d'} {{(pid=80144) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 11:05:49.313281 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.notifications [None req-a600a664-2756-4bce-81cf-60ba11a0ad6d admin admin] The token cache is being invalidate because project 575a9f90c7674a0faea1b812e8ac4f71 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80144) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 07 11:05:49.313886 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.notifications [None req-a600a664-2756-4bce-81cf-60ba11a0ad6d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80144) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 11:05:49.314368 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.notifications [None req-a600a664-2756-4bce-81cf-60ba11a0ad6d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80144) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 11:05:49.317922 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 1351/2675] 10.222.0.25 () {58 vars in 1233 bytes} [Sun Aug 7 11:05:49 2022] DELETE /identity/v3/projects/575a9f90c7674a0faea1b812e8ac4f71 => generated 0 bytes in 115 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 07 11:05:49.321787 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a600a664-2756-4bce-81cf-60ba11a0ad6d admin admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 11:05:49.323351 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.fernet_utils [None req-a600a664-2756-4bce-81cf-60ba11a0ad6d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80144) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 11:05:49.360956 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f101f938-fe79-4e4b-a2e9-86d9f8100170 admin admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 11:05:49.366366 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cca76c12-e56d-40c5-bdb6-42f84b615501 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5ac47e3b09f43d19a23a0f457fe9e23', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 11:05:49.367013 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cca76c12-e56d-40c5-bdb6-42f84b615501 admin admin] REQUEST_METHOD: `POST` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 11:05:49.367301 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cca76c12-e56d-40c5-bdb6-42f84b615501 admin admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 11:05:49.367578 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cca76c12-e56d-40c5-bdb6-42f84b615501 admin admin] PATH_INFO: `/v3/users` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 11:05:49.368210 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cca76c12-e56d-40c5-bdb6-42f84b615501 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 11:05:49.370131 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cca76c12-e56d-40c5-bdb6-42f84b615501 admin admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 11:05:49.391939 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a600a664-2756-4bce-81cf-60ba11a0ad6d admin admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 11:05:49.395307 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1325/2676] 10.222.0.25 () {60 vars in 1144 bytes} [Sun Aug 7 11:05:49 2022] POST /identity/v3/users => generated 288 bytes in 96 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 07 11:05:49.396331 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cca76c12-e56d-40c5-bdb6-42f84b615501 admin admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 11:05:49.397024 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81bc8743-02fb-4580-a738-dba64b7afb94 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5ac47e3b09f43d19a23a0f457fe9e23', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 11:05:49.397651 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81bc8743-02fb-4580-a738-dba64b7afb94 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 11:05:49.397929 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81bc8743-02fb-4580-a738-dba64b7afb94 admin admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 11:05:49.398041 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-cca76c12-e56d-40c5-bdb6-42f84b615501 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 11:05:49.398199 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81bc8743-02fb-4580-a738-dba64b7afb94 admin admin] PATH_INFO: `/v3/users/98f674549c524f949ca06044f3e2b23b` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 11:05:49.399369 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81bc8743-02fb-4580-a738-dba64b7afb94 admin admin] RBAC: Authorizing `identity:delete_user(user_id=98f674549c524f949ca06044f3e2b23b)` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 11:05:49.401254 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81bc8743-02fb-4580-a738-dba64b7afb94 admin admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 11:05:49.411908 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cca76c12-e56d-40c5-bdb6-42f84b615501 admin admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 11:05:49.415442 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87daac66-87ff-44e2-846c-3a941369c838 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5ac47e3b09f43d19a23a0f457fe9e23', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 11:05:49.416056 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87daac66-87ff-44e2-846c-3a941369c838 admin admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 11:05:49.416334 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87daac66-87ff-44e2-846c-3a941369c838 admin admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 11:05:49.416602 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87daac66-87ff-44e2-846c-3a941369c838 admin admin] PATH_INFO: `/v3/roles` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 11:05:49.417374 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87daac66-87ff-44e2-846c-3a941369c838 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 11:05:49.418982 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87daac66-87ff-44e2-846c-3a941369c838 admin admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 11:05:49.433758 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1326/2677] 10.222.0.25 () {58 vars in 1122 bytes} [Sun Aug 7 11:05:49 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 07 11:05:49.434907 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87daac66-87ff-44e2-846c-3a941369c838 admin admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 11:05:49.446287 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87daac66-87ff-44e2-846c-3a941369c838 admin admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 11:05:49.451276 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-225bb7e0-df49-4b97-abf8-5a0a46907396 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5ac47e3b09f43d19a23a0f457fe9e23', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 11:05:49.451814 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-225bb7e0-df49-4b97-abf8-5a0a46907396 admin admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 11:05:49.451941 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.notifications [None req-81bc8743-02fb-4580-a738-dba64b7afb94 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '98f674549c524f949ca06044f3e2b23b'} {{(pid=80144) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 11:05:49.452154 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-225bb7e0-df49-4b97-abf8-5a0a46907396 admin admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 11:05:49.452484 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-225bb7e0-df49-4b97-abf8-5a0a46907396 admin admin] PATH_INFO: `/v3/roles` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 11:05:49.453136 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-225bb7e0-df49-4b97-abf8-5a0a46907396 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 11:05:49.454717 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-225bb7e0-df49-4b97-abf8-5a0a46907396 admin admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 11:05:49.460728 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.notifications [None req-81bc8743-02fb-4580-a738-dba64b7afb94 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '98f674549c524f949ca06044f3e2b23b'} {{(pid=80144) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 11:05:49.465594 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1327/2678] 10.222.0.25 () {58 vars in 1122 bytes} [Sun Aug 7 11:05:49 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 07 11:05:49.468641 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.notifications [None req-81bc8743-02fb-4580-a738-dba64b7afb94 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '98f674549c524f949ca06044f3e2b23b'} {{(pid=80144) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 11:05:49.469517 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.notifications [None req-81bc8743-02fb-4580-a738-dba64b7afb94 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '98f674549c524f949ca06044f3e2b23b'} {{(pid=80144) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 11:05:49.469808 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-225bb7e0-df49-4b97-abf8-5a0a46907396 admin admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 11:05:49.470248 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.notifications [None req-81bc8743-02fb-4580-a738-dba64b7afb94 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '98f674549c524f949ca06044f3e2b23b'} {{(pid=80144) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 11:05:49.471528 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-225bb7e0-df49-4b97-abf8-5a0a46907396 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 11:05:49.475585 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=83,keystone:DELETE=15,keystone:INSERT=2 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 11:05:49.486348 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-225bb7e0-df49-4b97-abf8-5a0a46907396 admin admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 11:05:49.491749 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44b8f522-d5af-4674-8fa4-033a4e64eb1b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5ac47e3b09f43d19a23a0f457fe9e23', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 11:05:49.492447 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 1352/2679] 10.222.0.25 () {58 vars in 1224 bytes} [Sun Aug 7 11:05:49 2022] DELETE /identity/v3/users/98f674549c524f949ca06044f3e2b23b => generated 0 bytes in 171 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 07 11:05:49.492573 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44b8f522-d5af-4674-8fa4-033a4e64eb1b admin admin] REQUEST_METHOD: `PUT` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 11:05:49.492838 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44b8f522-d5af-4674-8fa4-033a4e64eb1b admin admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 11:05:49.493195 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44b8f522-d5af-4674-8fa4-033a4e64eb1b admin admin] PATH_INFO: `/v3/projects/17ff376c11c440a6bfae804294934fa4/users/49b7c5aa7b754daab0268eb612ca29a9/roles/b98dfb3b9fee4644af2887a3ac3fd2e6` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 11:05:49.508490 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44b8f522-d5af-4674-8fa4-033a4e64eb1b admin admin] RBAC: Authorizing `identity:create_grant(project_id=17ff376c11c440a6bfae804294934fa4, user_id=49b7c5aa7b754daab0268eb612ca29a9, role_id=b98dfb3b9fee4644af2887a3ac3fd2e6)` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 11:05:49.511174 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44b8f522-d5af-4674-8fa4-033a4e64eb1b admin admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 11:05:49.513458 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=88,keystone:DELETE=7,keystone:INSERT=5 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 11:05:49.526883 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1328/2680] 10.222.0.25 () {60 vars in 1483 bytes} [Sun Aug 7 11:05:49 2022] PUT /identity/v3/projects/17ff376c11c440a6bfae804294934fa4/users/49b7c5aa7b754daab0268eb612ca29a9/roles/b98dfb3b9fee4644af2887a3ac3fd2e6 => generated 0 bytes in 57 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 07 11:05:49.538254 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df68a7ca-3459-46e8-a391-fbe9b1eb9f81 None None] REQUEST_METHOD: `POST` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 11:05:49.538549 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df68a7ca-3459-46e8-a391-fbe9b1eb9f81 None None] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 11:05:49.538807 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df68a7ca-3459-46e8-a391-fbe9b1eb9f81 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 11:05:49.569542 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.auth.core [None req-df68a7ca-3459-46e8-a391-fbe9b1eb9f81 None None] MFA Rules not processed for user `49b7c5aa7b754daab0268eb612ca29a9`. Rule list: `[]` (Enabled: `True`). {{(pid=80144) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 11:05:49.571036 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.fernet_utils [None req-df68a7ca-3459-46e8-a391-fbe9b1eb9f81 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80144) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 11:05:49.638414 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 1353/2681] 10.222.0.25 () {58 vars in 958 bytes} [Sun Aug 7 11:05:49 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 07 11:05:49.650465 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44b8f522-d5af-4674-8fa4-033a4e64eb1b admin admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 11:05:49.652807 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-44b8f522-d5af-4674-8fa4-033a4e64eb1b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 11:05:49.711559 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44b8f522-d5af-4674-8fa4-033a4e64eb1b admin admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 11:05:49.715187 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a132405-3d6e-4f4a-89b8-ebeec5d3789c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88dbd6dfd1164994a3e16da553561910', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6de1f3cac02c4bcdb05dd2337176033d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 11:05:49.715796 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a132405-3d6e-4f4a-89b8-ebeec5d3789c service cinder] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 11:05:49.716077 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a132405-3d6e-4f4a-89b8-ebeec5d3789c service cinder] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 11:05:49.716352 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a132405-3d6e-4f4a-89b8-ebeec5d3789c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 11:05:49.726038 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a132405-3d6e-4f4a-89b8-ebeec5d3789c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 11:05:49.727532 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a132405-3d6e-4f4a-89b8-ebeec5d3789c service cinder] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 11:05:49.739349 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1329/2682] 10.222.0.25 () {60 vars in 1365 bytes} [Sun Aug 7 11:05:49 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 07 11:05:49.787145 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df68a7ca-3459-46e8-a391-fbe9b1eb9f81 None None] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 11:05:49.788727 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.fernet_utils [None req-df68a7ca-3459-46e8-a391-fbe9b1eb9f81 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80144) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 11:05:49.863683 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df68a7ca-3459-46e8-a391-fbe9b1eb9f81 None None] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 11:05:49.869499 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d16ee42c-bef9-4f62-b3a2-2cfae9cc27c4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5ac47e3b09f43d19a23a0f457fe9e23', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 11:05:49.870154 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d16ee42c-bef9-4f62-b3a2-2cfae9cc27c4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 11:05:49.870511 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d16ee42c-bef9-4f62-b3a2-2cfae9cc27c4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 11:05:49.870677 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d16ee42c-bef9-4f62-b3a2-2cfae9cc27c4 admin admin] PATH_INFO: `/v3/projects/86846851dd8841588845fa41831684ad` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 11:05:49.871974 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d16ee42c-bef9-4f62-b3a2-2cfae9cc27c4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=86846851dd8841588845fa41831684ad)` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 11:05:49.873889 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d16ee42c-bef9-4f62-b3a2-2cfae9cc27c4 admin admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 11:05:49.928514 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.notifications [None req-d16ee42c-bef9-4f62-b3a2-2cfae9cc27c4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '86846851dd8841588845fa41831684ad', 'request_id': 'req-d16ee42c-bef9-4f62-b3a2-2cfae9cc27c4'} {{(pid=80144) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 11:05:49.934151 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.notifications [None req-d16ee42c-bef9-4f62-b3a2-2cfae9cc27c4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '86846851dd8841588845fa41831684ad', 'request_id': 'req-d16ee42c-bef9-4f62-b3a2-2cfae9cc27c4'} {{(pid=80144) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 11:05:49.942938 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.notifications [None req-d16ee42c-bef9-4f62-b3a2-2cfae9cc27c4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '86846851dd8841588845fa41831684ad', 'request_id': 'req-d16ee42c-bef9-4f62-b3a2-2cfae9cc27c4'} {{(pid=80144) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 11:05:49.954952 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.notifications [None req-d16ee42c-bef9-4f62-b3a2-2cfae9cc27c4 admin admin] The token cache is being invalidate because project 86846851dd8841588845fa41831684ad was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80144) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 07 11:05:49.955247 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.notifications [None req-d16ee42c-bef9-4f62-b3a2-2cfae9cc27c4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80144) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 11:05:49.956026 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.notifications [None req-d16ee42c-bef9-4f62-b3a2-2cfae9cc27c4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80144) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 11:05:49.958822 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 1354/2683] 10.222.0.25 () {58 vars in 1233 bytes} [Sun Aug 7 11:05:49 2022] DELETE /identity/v3/projects/86846851dd8841588845fa41831684ad => generated 0 bytes in 172 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 07 11:05:49.963517 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a132405-3d6e-4f4a-89b8-ebeec5d3789c service cinder] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 11:05:49.976022 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a132405-3d6e-4f4a-89b8-ebeec5d3789c service cinder] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 11:05:50.055677 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5d65c16-2b32-46e5-9be1-62d0d7f9fb58 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5ac47e3b09f43d19a23a0f457fe9e23', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 11:05:50.056868 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5d65c16-2b32-46e5-9be1-62d0d7f9fb58 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 11:05:50.056868 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5d65c16-2b32-46e5-9be1-62d0d7f9fb58 admin admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 11:05:50.057279 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5d65c16-2b32-46e5-9be1-62d0d7f9fb58 admin admin] PATH_INFO: `/v3/users/b9779bac8dcc460d9990e98733ac72eb` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 11:05:50.058560 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5d65c16-2b32-46e5-9be1-62d0d7f9fb58 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b9779bac8dcc460d9990e98733ac72eb)` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 11:05:50.061341 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5d65c16-2b32-46e5-9be1-62d0d7f9fb58 admin admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 11:05:50.112069 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-b5d65c16-2b32-46e5-9be1-62d0d7f9fb58 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b9779bac8dcc460d9990e98733ac72eb'} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 11:05:50.119679 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-b5d65c16-2b32-46e5-9be1-62d0d7f9fb58 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b9779bac8dcc460d9990e98733ac72eb'} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 11:05:50.131935 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-b5d65c16-2b32-46e5-9be1-62d0d7f9fb58 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b9779bac8dcc460d9990e98733ac72eb'} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 11:05:50.132617 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-b5d65c16-2b32-46e5-9be1-62d0d7f9fb58 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b9779bac8dcc460d9990e98733ac72eb'} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 11:05:50.139540 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-b5d65c16-2b32-46e5-9be1-62d0d7f9fb58 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b9779bac8dcc460d9990e98733ac72eb'} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 11:05:50.145628 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1330/2684] 10.222.0.25 () {58 vars in 1224 bytes} [Sun Aug 7 11:05:49 2022] DELETE /identity/v3/users/b9779bac8dcc460d9990e98733ac72eb => generated 0 bytes in 183 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 07 11:05:50.431497 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d16ee42c-bef9-4f62-b3a2-2cfae9cc27c4 admin admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 11:05:50.432939 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.fernet_utils [None req-d16ee42c-bef9-4f62-b3a2-2cfae9cc27c4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80144) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 11:05:50.445271 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:DELETE=9,keystone:SELECT=90,keystone:INSERT=1 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 11:05:50.449402 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d16ee42c-bef9-4f62-b3a2-2cfae9cc27c4 admin admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 11:05:50.510519 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59bfef2d-c856-4658-bad9-e636f9b4e80d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5ac47e3b09f43d19a23a0f457fe9e23', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 11:05:50.511189 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59bfef2d-c856-4658-bad9-e636f9b4e80d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 11:05:50.511477 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59bfef2d-c856-4658-bad9-e636f9b4e80d admin admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 11:05:50.511755 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59bfef2d-c856-4658-bad9-e636f9b4e80d admin admin] PATH_INFO: `/v3/projects/d121077dda014d9aa4b0409aa1869b85` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 11:05:50.512875 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59bfef2d-c856-4658-bad9-e636f9b4e80d admin admin] RBAC: Authorizing `identity:delete_project(project_id=d121077dda014d9aa4b0409aa1869b85)` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 11:05:50.514755 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59bfef2d-c856-4658-bad9-e636f9b4e80d admin admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 11:05:50.561574 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.notifications [None req-59bfef2d-c856-4658-bad9-e636f9b4e80d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd121077dda014d9aa4b0409aa1869b85', 'request_id': 'req-59bfef2d-c856-4658-bad9-e636f9b4e80d'} {{(pid=80144) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 11:05:50.567847 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.notifications [None req-59bfef2d-c856-4658-bad9-e636f9b4e80d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd121077dda014d9aa4b0409aa1869b85', 'request_id': 'req-59bfef2d-c856-4658-bad9-e636f9b4e80d'} {{(pid=80144) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 11:05:50.575484 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.notifications [None req-59bfef2d-c856-4658-bad9-e636f9b4e80d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd121077dda014d9aa4b0409aa1869b85', 'request_id': 'req-59bfef2d-c856-4658-bad9-e636f9b4e80d'} {{(pid=80144) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 11:05:50.587593 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.notifications [None req-59bfef2d-c856-4658-bad9-e636f9b4e80d admin admin] The token cache is being invalidate because project d121077dda014d9aa4b0409aa1869b85 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80144) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 07 11:05:50.588139 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.notifications [None req-59bfef2d-c856-4658-bad9-e636f9b4e80d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80144) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 11:05:50.588795 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.notifications [None req-59bfef2d-c856-4658-bad9-e636f9b4e80d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80144) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 11:05:50.590290 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 1355/2685] 10.222.0.25 () {58 vars in 1233 bytes} [Sun Aug 7 11:05:50 2022] DELETE /identity/v3/projects/d121077dda014d9aa4b0409aa1869b85 => generated 0 bytes in 159 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 07 11:05:50.596317 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4600c294-7523-4908-9b19-2a691f223882 None None] REQUEST_METHOD: `POST` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 11:05:50.596798 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4600c294-7523-4908-9b19-2a691f223882 None None] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 11:05:50.597579 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4600c294-7523-4908-9b19-2a691f223882 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 11:05:50.617201 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.auth.core [None req-4600c294-7523-4908-9b19-2a691f223882 None None] MFA Rules not processed for user `bac4e6b76fe64217ac8fa44e71cffe6a`. Rule list: `[]` (Enabled: `True`). {{(pid=80143) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 11:05:50.618414 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-4600c294-7523-4908-9b19-2a691f223882 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 11:05:50.627731 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:INSERT=2,keystone:SELECT=88,keystone:DELETE=10 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 11:05:50.678377 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1331/2686] 10.222.0.25 () {58 vars in 958 bytes} [Sun Aug 7 11:05:50 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 07 11:05:50.687969 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb6789f4-6a09-445d-8921-a3ee5842177b None None] REQUEST_METHOD: `POST` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 11:05:50.688258 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb6789f4-6a09-445d-8921-a3ee5842177b None None] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 11:05:50.688539 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb6789f4-6a09-445d-8921-a3ee5842177b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 11:05:50.708248 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.auth.core [None req-bb6789f4-6a09-445d-8921-a3ee5842177b None None] MFA Rules not processed for user `bac4e6b76fe64217ac8fa44e71cffe6a`. Rule list: `[]` (Enabled: `True`). {{(pid=80144) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 11:05:50.709435 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.fernet_utils [None req-bb6789f4-6a09-445d-8921-a3ee5842177b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80144) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 11:05:50.717256 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 1356/2687] 10.222.0.25 () {58 vars in 958 bytes} [Sun Aug 7 11:05:50 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 07 11:05:50.721694 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4600c294-7523-4908-9b19-2a691f223882 None None] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 11:05:50.731948 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4600c294-7523-4908-9b19-2a691f223882 None None] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 11:05:50.736825 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0a5d2e6-864a-4193-9802-719824f0df4c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5ac47e3b09f43d19a23a0f457fe9e23', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 11:05:50.737422 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0a5d2e6-864a-4193-9802-719824f0df4c admin admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 11:05:50.737706 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0a5d2e6-864a-4193-9802-719824f0df4c admin admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 11:05:50.737984 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0a5d2e6-864a-4193-9802-719824f0df4c admin admin] PATH_INFO: `/v3/domains` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 11:05:50.738628 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0a5d2e6-864a-4193-9802-719824f0df4c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80143) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 11:05:50.738963 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0a5d2e6-864a-4193-9802-719824f0df4c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 11:05:50.740547 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0a5d2e6-864a-4193-9802-719824f0df4c admin admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 11:05:50.751702 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1332/2688] 10.222.0.25 () {58 vars in 1153 bytes} [Sun Aug 7 11:05:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 11:05:50.756043 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb6789f4-6a09-445d-8921-a3ee5842177b None None] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 11:05:50.765117 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb6789f4-6a09-445d-8921-a3ee5842177b None None] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 11:05:50.770402 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f3fd4e8-dbbe-4dd3-b84a-732e43857719 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5ac47e3b09f43d19a23a0f457fe9e23', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 11:05:50.771066 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f3fd4e8-dbbe-4dd3-b84a-732e43857719 admin admin] REQUEST_METHOD: `POST` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 11:05:50.771401 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f3fd4e8-dbbe-4dd3-b84a-732e43857719 admin admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 11:05:50.771742 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f3fd4e8-dbbe-4dd3-b84a-732e43857719 admin admin] PATH_INFO: `/v3/projects` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 11:05:50.772396 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f3fd4e8-dbbe-4dd3-b84a-732e43857719 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 11:05:50.774333 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f3fd4e8-dbbe-4dd3-b84a-732e43857719 admin admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 11:05:50.802134 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 1357/2689] 10.222.0.25 () {60 vars in 1153 bytes} [Sun Aug 7 11:05:50 2022] POST /identity/v3/projects => generated 381 bytes in 46 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 07 11:05:50.806434 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0a5d2e6-864a-4193-9802-719824f0df4c admin admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 11:05:50.817336 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0a5d2e6-864a-4193-9802-719824f0df4c admin admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 11:05:50.874344 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f17cf97e-db30-43ac-8167-4b65a35874e4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5ac47e3b09f43d19a23a0f457fe9e23', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 11:05:50.874982 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f17cf97e-db30-43ac-8167-4b65a35874e4 admin admin] REQUEST_METHOD: `POST` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 11:05:50.875269 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f17cf97e-db30-43ac-8167-4b65a35874e4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 11:05:50.875545 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f17cf97e-db30-43ac-8167-4b65a35874e4 admin admin] PATH_INFO: `/v3/users` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 11:05:50.876179 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f17cf97e-db30-43ac-8167-4b65a35874e4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 11:05:50.878102 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f17cf97e-db30-43ac-8167-4b65a35874e4 admin admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 11:05:50.899303 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1333/2690] 10.222.0.25 () {60 vars in 1144 bytes} [Sun Aug 7 11:05:50 2022] POST /identity/v3/users => generated 291 bytes in 93 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 07 11:05:50.903426 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f3fd4e8-dbbe-4dd3-b84a-732e43857719 admin admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 11:05:50.914280 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f3fd4e8-dbbe-4dd3-b84a-732e43857719 admin admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 11:05:50.918890 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58b3025d-2917-414f-84f1-0d0c7b812a81 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5ac47e3b09f43d19a23a0f457fe9e23', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 11:05:50.919522 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58b3025d-2917-414f-84f1-0d0c7b812a81 admin admin] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 11:05:50.919803 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58b3025d-2917-414f-84f1-0d0c7b812a81 admin admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 11:05:50.920079 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58b3025d-2917-414f-84f1-0d0c7b812a81 admin admin] PATH_INFO: `/v3/roles` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 11:05:50.920722 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58b3025d-2917-414f-84f1-0d0c7b812a81 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 11:05:50.922294 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58b3025d-2917-414f-84f1-0d0c7b812a81 admin admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 11:05:50.932161 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 1358/2691] 10.222.0.25 () {58 vars in 1122 bytes} [Sun Aug 7 11:05:50 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 07 11:05:50.935822 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f17cf97e-db30-43ac-8167-4b65a35874e4 admin admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 11:05:50.946413 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f17cf97e-db30-43ac-8167-4b65a35874e4 admin admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 11:05:50.951969 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-548c9e8f-cf10-459a-b8eb-49904ddcd394 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5ac47e3b09f43d19a23a0f457fe9e23', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 11:05:50.952571 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-548c9e8f-cf10-459a-b8eb-49904ddcd394 admin admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 11:05:50.952853 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-548c9e8f-cf10-459a-b8eb-49904ddcd394 admin admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 11:05:50.953128 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-548c9e8f-cf10-459a-b8eb-49904ddcd394 admin admin] PATH_INFO: `/v3/roles` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 11:05:50.953846 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-548c9e8f-cf10-459a-b8eb-49904ddcd394 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 11:05:50.955656 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-548c9e8f-cf10-459a-b8eb-49904ddcd394 admin admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 11:05:50.967336 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1334/2692] 10.222.0.25 () {58 vars in 1122 bytes} [Sun Aug 7 11:05:50 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 07 11:05:50.971555 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58b3025d-2917-414f-84f1-0d0c7b812a81 admin admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 11:05:50.982496 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58b3025d-2917-414f-84f1-0d0c7b812a81 admin admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 11:05:50.987391 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5091702b-3987-49d4-90fc-133a0458a0ca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5ac47e3b09f43d19a23a0f457fe9e23', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 11:05:50.988656 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5091702b-3987-49d4-90fc-133a0458a0ca admin admin] REQUEST_METHOD: `PUT` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 11:05:50.988884 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5091702b-3987-49d4-90fc-133a0458a0ca admin admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 11:05:50.989144 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5091702b-3987-49d4-90fc-133a0458a0ca admin admin] PATH_INFO: `/v3/projects/a9a18432475e468787652b19610e921b/users/83e242684e474d85b21a6d34e61a8941/roles/b98dfb3b9fee4644af2887a3ac3fd2e6` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 11:05:51.003458 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5091702b-3987-49d4-90fc-133a0458a0ca admin admin] RBAC: Authorizing `identity:create_grant(project_id=a9a18432475e468787652b19610e921b, user_id=83e242684e474d85b21a6d34e61a8941, role_id=b98dfb3b9fee4644af2887a3ac3fd2e6)` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 11:05:51.006054 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5091702b-3987-49d4-90fc-133a0458a0ca admin admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 11:05:51.018323 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 1359/2693] 10.222.0.25 () {60 vars in 1483 bytes} [Sun Aug 7 11:05:50 2022] PUT /identity/v3/projects/a9a18432475e468787652b19610e921b/users/83e242684e474d85b21a6d34e61a8941/roles/b98dfb3b9fee4644af2887a3ac3fd2e6 => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 07 11:05:51.029702 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad5b4d4d-781c-4da0-aa6b-0f8fbf723b01 None None] REQUEST_METHOD: `POST` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 11:05:51.030008 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad5b4d4d-781c-4da0-aa6b-0f8fbf723b01 None None] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 11:05:51.030281 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad5b4d4d-781c-4da0-aa6b-0f8fbf723b01 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 11:05:51.058834 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.auth.core [None req-ad5b4d4d-781c-4da0-aa6b-0f8fbf723b01 None None] MFA Rules not processed for user `83e242684e474d85b21a6d34e61a8941`. Rule list: `[]` (Enabled: `True`). {{(pid=80143) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 11:05:51.060118 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-ad5b4d4d-781c-4da0-aa6b-0f8fbf723b01 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 11:05:51.127518 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1335/2694] 10.222.0.25 () {58 vars in 958 bytes} [Sun Aug 7 11:05:51 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 07 11:05:51.132152 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5091702b-3987-49d4-90fc-133a0458a0ca admin admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 11:05:51.170400 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 11:05:51.203202 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5091702b-3987-49d4-90fc-133a0458a0ca admin admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 11:05:51.208443 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dddc4267-40f6-42ec-ba3e-6d7c57369606 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5ac47e3b09f43d19a23a0f457fe9e23', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 11:05:51.209066 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dddc4267-40f6-42ec-ba3e-6d7c57369606 admin admin] REQUEST_METHOD: `POST` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 11:05:51.209351 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dddc4267-40f6-42ec-ba3e-6d7c57369606 admin admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 11:05:51.209628 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dddc4267-40f6-42ec-ba3e-6d7c57369606 admin admin] PATH_INFO: `/v3/projects` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 11:05:51.210250 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dddc4267-40f6-42ec-ba3e-6d7c57369606 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 11:05:51.212212 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dddc4267-40f6-42ec-ba3e-6d7c57369606 admin admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 11:05:51.245284 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 1360/2695] 10.222.0.25 () {60 vars in 1153 bytes} [Sun Aug 7 11:05:51 2022] POST /identity/v3/projects => generated 379 bytes in 111 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 07 11:05:51.246610 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad5b4d4d-781c-4da0-aa6b-0f8fbf723b01 None None] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 11:05:51.259641 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad5b4d4d-781c-4da0-aa6b-0f8fbf723b01 None None] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 11:05:51.267030 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 11:05:51.320831 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7531844f-e0c6-40cb-a2ab-8e9eb524aea9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5ac47e3b09f43d19a23a0f457fe9e23', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 11:05:51.321460 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7531844f-e0c6-40cb-a2ab-8e9eb524aea9 admin admin] REQUEST_METHOD: `POST` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 11:05:51.321746 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7531844f-e0c6-40cb-a2ab-8e9eb524aea9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 11:05:51.322022 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7531844f-e0c6-40cb-a2ab-8e9eb524aea9 admin admin] PATH_INFO: `/v3/users` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 11:05:51.322647 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7531844f-e0c6-40cb-a2ab-8e9eb524aea9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 11:05:51.324700 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7531844f-e0c6-40cb-a2ab-8e9eb524aea9 admin admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 11:05:51.349598 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1336/2696] 10.222.0.25 () {60 vars in 1144 bytes} [Sun Aug 7 11:05:51 2022] POST /identity/v3/users => generated 296 bytes in 103 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 07 11:05:51.353561 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dddc4267-40f6-42ec-ba3e-6d7c57369606 admin admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 11:05:51.365570 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dddc4267-40f6-42ec-ba3e-6d7c57369606 admin admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 11:05:51.373518 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03a301b1-f683-428f-8ff5-82b5b78f1a1a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5ac47e3b09f43d19a23a0f457fe9e23', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 11:05:51.374138 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03a301b1-f683-428f-8ff5-82b5b78f1a1a admin admin] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 11:05:51.374422 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03a301b1-f683-428f-8ff5-82b5b78f1a1a admin admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 11:05:51.374701 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03a301b1-f683-428f-8ff5-82b5b78f1a1a admin admin] PATH_INFO: `/v3/roles` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 11:05:51.375381 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03a301b1-f683-428f-8ff5-82b5b78f1a1a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 11:05:51.377002 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03a301b1-f683-428f-8ff5-82b5b78f1a1a admin admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 11:05:51.388656 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 1361/2697] 10.222.0.25 () {58 vars in 1122 bytes} [Sun Aug 7 11:05:51 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 07 11:05:51.392765 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7531844f-e0c6-40cb-a2ab-8e9eb524aea9 admin admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 11:05:51.403140 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7531844f-e0c6-40cb-a2ab-8e9eb524aea9 admin admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 11:05:51.407866 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11856419-fa42-4385-877a-32c998dfde63 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5ac47e3b09f43d19a23a0f457fe9e23', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 11:05:51.408524 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11856419-fa42-4385-877a-32c998dfde63 admin admin] REQUEST_METHOD: `PUT` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 11:05:51.408802 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11856419-fa42-4385-877a-32c998dfde63 admin admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 11:05:51.409076 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11856419-fa42-4385-877a-32c998dfde63 admin admin] PATH_INFO: `/v3/projects/bff93fe614c14e0aa5b551567b08f656/users/9626959db23544f4b206487bd3a683b5/roles/81b1f1bcee2a473088fbbd327c1c5b23` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 11:05:51.429553 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11856419-fa42-4385-877a-32c998dfde63 admin admin] RBAC: Authorizing `identity:create_grant(project_id=bff93fe614c14e0aa5b551567b08f656, user_id=9626959db23544f4b206487bd3a683b5, role_id=81b1f1bcee2a473088fbbd327c1c5b23)` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 11:05:51.432311 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11856419-fa42-4385-877a-32c998dfde63 admin admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 11:05:51.450966 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1337/2698] 10.222.0.25 () {60 vars in 1483 bytes} [Sun Aug 7 11:05:51 2022] PUT /identity/v3/projects/bff93fe614c14e0aa5b551567b08f656/users/9626959db23544f4b206487bd3a683b5/roles/81b1f1bcee2a473088fbbd327c1c5b23 => generated 0 bytes in 59 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 07 11:05:51.456036 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03a301b1-f683-428f-8ff5-82b5b78f1a1a admin admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 11:05:51.466161 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03a301b1-f683-428f-8ff5-82b5b78f1a1a admin admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 11:05:51.525106 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e2c224b-1580-4275-b6b0-7d6ebf29b3bb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5ac47e3b09f43d19a23a0f457fe9e23', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 11:05:51.525766 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e2c224b-1580-4275-b6b0-7d6ebf29b3bb admin admin] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 11:05:51.526117 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e2c224b-1580-4275-b6b0-7d6ebf29b3bb admin admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 11:05:51.526394 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e2c224b-1580-4275-b6b0-7d6ebf29b3bb admin admin] PATH_INFO: `/v3/roles` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 11:05:51.527075 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e2c224b-1580-4275-b6b0-7d6ebf29b3bb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 11:05:51.528663 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e2c224b-1580-4275-b6b0-7d6ebf29b3bb admin admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 11:05:51.542905 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 1362/2699] 10.222.0.25 () {58 vars in 1122 bytes} [Sun Aug 7 11:05:51 2022] GET /identity/v3/roles => generated 1420 bytes in 84 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 07 11:05:51.543879 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11856419-fa42-4385-877a-32c998dfde63 admin admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 11:05:51.555709 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11856419-fa42-4385-877a-32c998dfde63 admin admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 11:05:51.560390 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98ff25a9-6b7a-493c-8d40-43b9b472b6a3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5ac47e3b09f43d19a23a0f457fe9e23', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 11:05:51.561120 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98ff25a9-6b7a-493c-8d40-43b9b472b6a3 admin admin] REQUEST_METHOD: `PUT` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 11:05:51.561533 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98ff25a9-6b7a-493c-8d40-43b9b472b6a3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 11:05:51.561829 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98ff25a9-6b7a-493c-8d40-43b9b472b6a3 admin admin] PATH_INFO: `/v3/projects/bff93fe614c14e0aa5b551567b08f656/users/9626959db23544f4b206487bd3a683b5/roles/81b1f1bcee2a473088fbbd327c1c5b23` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 11:05:51.563796 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98ff25a9-6b7a-493c-8d40-43b9b472b6a3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=bff93fe614c14e0aa5b551567b08f656, user_id=9626959db23544f4b206487bd3a683b5, role_id=81b1f1bcee2a473088fbbd327c1c5b23)` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 11:05:51.567025 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98ff25a9-6b7a-493c-8d40-43b9b472b6a3 admin admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 11:05:51.578348 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1338/2700] 10.222.0.25 () {60 vars in 1483 bytes} [Sun Aug 7 11:05:51 2022] PUT /identity/v3/projects/bff93fe614c14e0aa5b551567b08f656/users/9626959db23544f4b206487bd3a683b5/roles/81b1f1bcee2a473088fbbd327c1c5b23 => generated 0 bytes in 35 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 07 11:05:51.589577 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9dbb701-3a42-4664-a890-ba2fca15b8ea None None] REQUEST_METHOD: `POST` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 11:05:51.589938 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9dbb701-3a42-4664-a890-ba2fca15b8ea None None] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 11:05:51.590284 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9dbb701-3a42-4664-a890-ba2fca15b8ea None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 11:05:51.619763 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.auth.core [None req-d9dbb701-3a42-4664-a890-ba2fca15b8ea None None] MFA Rules not processed for user `9626959db23544f4b206487bd3a683b5`. Rule list: `[]` (Enabled: `True`). {{(pid=80144) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 11:05:51.621002 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.fernet_utils [None req-d9dbb701-3a42-4664-a890-ba2fca15b8ea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80144) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 11:05:51.696765 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 1363/2701] 10.222.0.25 () {58 vars in 958 bytes} [Sun Aug 7 11:05:51 2022] POST /identity/v3/auth/tokens => generated 3323 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 07 11:05:51.703793 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98ff25a9-6b7a-493c-8d40-43b9b472b6a3 admin admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 11:05:51.705542 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-98ff25a9-6b7a-493c-8d40-43b9b472b6a3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 11:05:51.768182 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98ff25a9-6b7a-493c-8d40-43b9b472b6a3 admin admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 11:05:51.772265 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d43b6a23-63c2-4033-8bf9-a5ac6130101a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88dbd6dfd1164994a3e16da553561910', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6de1f3cac02c4bcdb05dd2337176033d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 11:05:51.772879 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d43b6a23-63c2-4033-8bf9-a5ac6130101a service cinder] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 11:05:51.773163 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d43b6a23-63c2-4033-8bf9-a5ac6130101a service cinder] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 11:05:51.773441 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d43b6a23-63c2-4033-8bf9-a5ac6130101a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 11:05:51.783353 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d43b6a23-63c2-4033-8bf9-a5ac6130101a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 11:05:51.785762 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d43b6a23-63c2-4033-8bf9-a5ac6130101a service cinder] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 11:05:51.798231 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1339/2702] 10.222.0.25 () {60 vars in 1365 bytes} [Sun Aug 7 11:05:51 2022] GET /identity/v3/auth/tokens => generated 3323 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 07 11:05:52.235307 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9dbb701-3a42-4664-a890-ba2fca15b8ea None None] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 11:05:52.264969 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 11:05:52.299249 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9dbb701-3a42-4664-a890-ba2fca15b8ea None None] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 11:05:52.303904 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7506be00-2534-458c-aee8-284075d770b5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5ac47e3b09f43d19a23a0f457fe9e23', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 11:05:52.304536 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7506be00-2534-458c-aee8-284075d770b5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 11:05:52.304819 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7506be00-2534-458c-aee8-284075d770b5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 11:05:52.305096 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7506be00-2534-458c-aee8-284075d770b5 admin admin] PATH_INFO: `/v3/users/83e242684e474d85b21a6d34e61a8941` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 11:05:52.306218 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7506be00-2534-458c-aee8-284075d770b5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=83e242684e474d85b21a6d34e61a8941)` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 11:05:52.308148 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7506be00-2534-458c-aee8-284075d770b5 admin admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 11:05:52.354232 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.notifications [None req-7506be00-2534-458c-aee8-284075d770b5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '83e242684e474d85b21a6d34e61a8941'} {{(pid=80144) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 11:05:52.361877 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.notifications [None req-7506be00-2534-458c-aee8-284075d770b5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '83e242684e474d85b21a6d34e61a8941'} {{(pid=80144) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 11:05:52.369417 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.notifications [None req-7506be00-2534-458c-aee8-284075d770b5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '83e242684e474d85b21a6d34e61a8941'} {{(pid=80144) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 11:05:52.370052 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.notifications [None req-7506be00-2534-458c-aee8-284075d770b5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '83e242684e474d85b21a6d34e61a8941'} {{(pid=80144) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 11:05:52.370675 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.notifications [None req-7506be00-2534-458c-aee8-284075d770b5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '83e242684e474d85b21a6d34e61a8941'} {{(pid=80144) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 11:05:52.387941 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 1364/2703] 10.222.0.25 () {58 vars in 1224 bytes} [Sun Aug 7 11:05:52 2022] DELETE /identity/v3/users/83e242684e474d85b21a6d34e61a8941 => generated 0 bytes in 153 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 07 11:05:52.392411 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-693e5a3a-0c04-45cc-bb16-356fad4e433e None None] REQUEST_METHOD: `POST` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 11:05:52.392697 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-693e5a3a-0c04-45cc-bb16-356fad4e433e None None] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 11:05:52.392971 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-693e5a3a-0c04-45cc-bb16-356fad4e433e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 11:05:52.413556 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.auth.core [None req-693e5a3a-0c04-45cc-bb16-356fad4e433e None None] MFA Rules not processed for user `bac4e6b76fe64217ac8fa44e71cffe6a`. Rule list: `[]` (Enabled: `True`). {{(pid=80143) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 11:05:52.414741 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-693e5a3a-0c04-45cc-bb16-356fad4e433e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 11:05:52.473783 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1340/2704] 10.222.0.25 () {58 vars in 958 bytes} [Sun Aug 7 11:05:52 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 07 11:05:52.483970 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7506be00-2534-458c-aee8-284075d770b5 admin admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 11:05:52.485479 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.fernet_utils [None req-7506be00-2534-458c-aee8-284075d770b5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80144) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 11:05:52.541912 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7506be00-2534-458c-aee8-284075d770b5 admin admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 11:05:52.545564 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3182a6b6-b053-4861-9707-feab51e50bc5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be1283c9bf47424c9aa3ad27cab03027', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6de1f3cac02c4bcdb05dd2337176033d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 11:05:52.546162 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3182a6b6-b053-4861-9707-feab51e50bc5 service neutron] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 11:05:52.546440 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3182a6b6-b053-4861-9707-feab51e50bc5 service neutron] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 11:05:52.546708 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3182a6b6-b053-4861-9707-feab51e50bc5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 11:05:52.555580 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3182a6b6-b053-4861-9707-feab51e50bc5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 11:05:52.556949 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3182a6b6-b053-4861-9707-feab51e50bc5 service neutron] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 11:05:52.566606 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 1365/2705] 10.222.0.25 () {60 vars in 1365 bytes} [Sun Aug 7 11:05:52 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 07 11:05:52.830676 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-693e5a3a-0c04-45cc-bb16-356fad4e433e None None] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 11:05:52.834111 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3182a6b6-b053-4861-9707-feab51e50bc5 service neutron] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 11:05:52.835672 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-693e5a3a-0c04-45cc-bb16-356fad4e433e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 11:05:52.835791 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.fernet_utils [None req-3182a6b6-b053-4861-9707-feab51e50bc5 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80144) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 11:05:52.848985 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-693e5a3a-0c04-45cc-bb16-356fad4e433e None None] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 11:05:52.851906 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3182a6b6-b053-4861-9707-feab51e50bc5 service neutron] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 11:05:52.856628 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-296512ee-c7e7-4f93-9c5d-eaf4ce6f78f3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5ac47e3b09f43d19a23a0f457fe9e23', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 11:05:52.856816 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bda46fb6-50e6-414e-a6a9-d2c68c68a9ea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5ac47e3b09f43d19a23a0f457fe9e23', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 11:05:52.857271 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-296512ee-c7e7-4f93-9c5d-eaf4ce6f78f3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 11:05:52.857495 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bda46fb6-50e6-414e-a6a9-d2c68c68a9ea admin admin] REQUEST_METHOD: `DELETE` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 11:05:52.857614 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-296512ee-c7e7-4f93-9c5d-eaf4ce6f78f3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 11:05:52.857774 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bda46fb6-50e6-414e-a6a9-d2c68c68a9ea admin admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 11:05:52.857888 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-296512ee-c7e7-4f93-9c5d-eaf4ce6f78f3 admin admin] PATH_INFO: `/v3/users/49b7c5aa7b754daab0268eb612ca29a9` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 11:05:52.858047 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bda46fb6-50e6-414e-a6a9-d2c68c68a9ea admin admin] PATH_INFO: `/v3/projects/a9a18432475e468787652b19610e921b` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 11:05:52.859270 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-296512ee-c7e7-4f93-9c5d-eaf4ce6f78f3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=49b7c5aa7b754daab0268eb612ca29a9)` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 11:05:52.859520 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bda46fb6-50e6-414e-a6a9-d2c68c68a9ea admin admin] RBAC: Authorizing `identity:delete_project(project_id=a9a18432475e468787652b19610e921b)` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 11:05:52.861356 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-296512ee-c7e7-4f93-9c5d-eaf4ce6f78f3 admin admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 11:05:52.861569 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bda46fb6-50e6-414e-a6a9-d2c68c68a9ea admin admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 11:05:52.865071 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 11:05:52.910668 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=83,keystone:DELETE=16,keystone:INSERT=1 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 11:05:52.913230 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.notifications [None req-296512ee-c7e7-4f93-9c5d-eaf4ce6f78f3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '49b7c5aa7b754daab0268eb612ca29a9'} {{(pid=80144) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 11:05:52.921785 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-bda46fb6-50e6-414e-a6a9-d2c68c68a9ea admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a9a18432475e468787652b19610e921b', 'request_id': 'req-bda46fb6-50e6-414e-a6a9-d2c68c68a9ea'} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 11:05:52.922822 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.notifications [None req-296512ee-c7e7-4f93-9c5d-eaf4ce6f78f3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '49b7c5aa7b754daab0268eb612ca29a9'} {{(pid=80144) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 11:05:52.928368 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-bda46fb6-50e6-414e-a6a9-d2c68c68a9ea admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a9a18432475e468787652b19610e921b', 'request_id': 'req-bda46fb6-50e6-414e-a6a9-d2c68c68a9ea'} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 11:05:52.933735 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.notifications [None req-296512ee-c7e7-4f93-9c5d-eaf4ce6f78f3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '49b7c5aa7b754daab0268eb612ca29a9'} {{(pid=80144) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 11:05:52.934508 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.notifications [None req-296512ee-c7e7-4f93-9c5d-eaf4ce6f78f3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '49b7c5aa7b754daab0268eb612ca29a9'} {{(pid=80144) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 11:05:52.935138 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.notifications [None req-296512ee-c7e7-4f93-9c5d-eaf4ce6f78f3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '49b7c5aa7b754daab0268eb612ca29a9'} {{(pid=80144) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 11:05:52.936152 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-bda46fb6-50e6-414e-a6a9-d2c68c68a9ea admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a9a18432475e468787652b19610e921b', 'request_id': 'req-bda46fb6-50e6-414e-a6a9-d2c68c68a9ea'} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 11:05:52.949038 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-bda46fb6-50e6-414e-a6a9-d2c68c68a9ea admin admin] The token cache is being invalidate because project a9a18432475e468787652b19610e921b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80143) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 07 11:05:52.949332 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-bda46fb6-50e6-414e-a6a9-d2c68c68a9ea admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 11:05:52.949953 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-bda46fb6-50e6-414e-a6a9-d2c68c68a9ea admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 11:05:52.951473 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1341/2706] 10.222.0.25 () {58 vars in 1233 bytes} [Sun Aug 7 11:05:52 2022] DELETE /identity/v3/projects/a9a18432475e468787652b19610e921b => generated 0 bytes in 121 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 07 11:05:52.954774 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 1366/2707] 10.222.0.25 () {58 vars in 1224 bytes} [Sun Aug 7 11:05:52 2022] DELETE /identity/v3/users/49b7c5aa7b754daab0268eb612ca29a9 => generated 0 bytes in 121 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 07 11:05:52.955387 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bda46fb6-50e6-414e-a6a9-d2c68c68a9ea admin admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 11:05:52.956931 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-bda46fb6-50e6-414e-a6a9-d2c68c68a9ea admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 11:05:52.959324 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7ee9451-8335-4e8b-b667-ee74d59a71c7 None None] REQUEST_METHOD: `POST` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 11:05:52.959619 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7ee9451-8335-4e8b-b667-ee74d59a71c7 None None] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 11:05:52.959901 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7ee9451-8335-4e8b-b667-ee74d59a71c7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 11:05:52.979124 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.auth.core [None req-e7ee9451-8335-4e8b-b667-ee74d59a71c7 None None] MFA Rules not processed for user `bac4e6b76fe64217ac8fa44e71cffe6a`. Rule list: `[]` (Enabled: `True`). {{(pid=80144) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 11:05:52.980314 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.fernet_utils [None req-e7ee9451-8335-4e8b-b667-ee74d59a71c7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80144) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 11:05:53.023158 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bda46fb6-50e6-414e-a6a9-d2c68c68a9ea admin admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 11:05:53.042542 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 1367/2708] 10.222.0.25 () {58 vars in 958 bytes} [Sun Aug 7 11:05:52 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 07 11:05:53.052236 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7ee9451-8335-4e8b-b667-ee74d59a71c7 None None] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 11:05:53.053646 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.fernet_utils [None req-e7ee9451-8335-4e8b-b667-ee74d59a71c7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80144) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 11:05:53.080433 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23a7df43-1fe8-4f1e-a471-b07e7f6748c0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5ac47e3b09f43d19a23a0f457fe9e23', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 11:05:53.080996 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23a7df43-1fe8-4f1e-a471-b07e7f6748c0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 11:05:53.081895 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23a7df43-1fe8-4f1e-a471-b07e7f6748c0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 11:05:53.082399 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23a7df43-1fe8-4f1e-a471-b07e7f6748c0 admin admin] PATH_INFO: `/v3/users/9626959db23544f4b206487bd3a683b5` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 11:05:53.083566 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23a7df43-1fe8-4f1e-a471-b07e7f6748c0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=9626959db23544f4b206487bd3a683b5)` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 11:05:53.085472 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23a7df43-1fe8-4f1e-a471-b07e7f6748c0 admin admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 11:05:53.111926 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7ee9451-8335-4e8b-b667-ee74d59a71c7 None None] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 11:05:53.116038 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e760b31-5bbf-4a2b-b669-6e6ab03391d3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be1283c9bf47424c9aa3ad27cab03027', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6de1f3cac02c4bcdb05dd2337176033d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 11:05:53.116689 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e760b31-5bbf-4a2b-b669-6e6ab03391d3 service neutron] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 11:05:53.117088 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e760b31-5bbf-4a2b-b669-6e6ab03391d3 service neutron] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 11:05:53.117432 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e760b31-5bbf-4a2b-b669-6e6ab03391d3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 11:05:53.126955 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 11:05:53.127136 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e760b31-5bbf-4a2b-b669-6e6ab03391d3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 11:05:53.128532 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e760b31-5bbf-4a2b-b669-6e6ab03391d3 service neutron] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 11:05:53.133776 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-23a7df43-1fe8-4f1e-a471-b07e7f6748c0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9626959db23544f4b206487bd3a683b5'} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 11:05:53.140392 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 1368/2709] 10.222.0.25 () {60 vars in 1365 bytes} [Sun Aug 7 11:05:53 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 07 11:05:53.145179 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-23a7df43-1fe8-4f1e-a471-b07e7f6748c0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9626959db23544f4b206487bd3a683b5'} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 11:05:53.157615 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-23a7df43-1fe8-4f1e-a471-b07e7f6748c0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9626959db23544f4b206487bd3a683b5'} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 11:05:53.158387 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-23a7df43-1fe8-4f1e-a471-b07e7f6748c0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9626959db23544f4b206487bd3a683b5'} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 11:05:53.166194 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-23a7df43-1fe8-4f1e-a471-b07e7f6748c0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9626959db23544f4b206487bd3a683b5'} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 11:05:53.172546 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1342/2710] 10.222.0.25 () {58 vars in 1224 bytes} [Sun Aug 7 11:05:52 2022] DELETE /identity/v3/users/9626959db23544f4b206487bd3a683b5 => generated 0 bytes in 218 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 07 11:05:53.428679 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e760b31-5bbf-4a2b-b669-6e6ab03391d3 service neutron] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 11:05:53.430554 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.fernet_utils [None req-0e760b31-5bbf-4a2b-b669-6e6ab03391d3 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80144) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 11:05:53.445946 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e760b31-5bbf-4a2b-b669-6e6ab03391d3 service neutron] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 11:05:53.476769 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23a7df43-1fe8-4f1e-a471-b07e7f6748c0 admin admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 11:05:53.478287 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-23a7df43-1fe8-4f1e-a471-b07e7f6748c0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 11:05:53.516994 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1830d5bf-a163-4ea3-8bab-ae7f843a8731 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5ac47e3b09f43d19a23a0f457fe9e23', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 11:05:53.517631 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1830d5bf-a163-4ea3-8bab-ae7f843a8731 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 11:05:53.517909 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1830d5bf-a163-4ea3-8bab-ae7f843a8731 admin admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 11:05:53.518177 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1830d5bf-a163-4ea3-8bab-ae7f843a8731 admin admin] PATH_INFO: `/v3/projects/17ff376c11c440a6bfae804294934fa4` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 11:05:53.520042 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1830d5bf-a163-4ea3-8bab-ae7f843a8731 admin admin] RBAC: Authorizing `identity:delete_project(project_id=17ff376c11c440a6bfae804294934fa4)` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 11:05:53.521923 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1830d5bf-a163-4ea3-8bab-ae7f843a8731 admin admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 11:05:53.528829 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:DELETE=4,keystone:SELECT=95,keystone:INSERT=1 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 11:05:53.563255 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23a7df43-1fe8-4f1e-a471-b07e7f6748c0 admin admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 11:05:53.573562 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.notifications [None req-1830d5bf-a163-4ea3-8bab-ae7f843a8731 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '17ff376c11c440a6bfae804294934fa4', 'request_id': 'req-1830d5bf-a163-4ea3-8bab-ae7f843a8731'} {{(pid=80144) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 11:05:53.579079 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.notifications [None req-1830d5bf-a163-4ea3-8bab-ae7f843a8731 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '17ff376c11c440a6bfae804294934fa4', 'request_id': 'req-1830d5bf-a163-4ea3-8bab-ae7f843a8731'} {{(pid=80144) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 11:05:53.588012 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.notifications [None req-1830d5bf-a163-4ea3-8bab-ae7f843a8731 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '17ff376c11c440a6bfae804294934fa4', 'request_id': 'req-1830d5bf-a163-4ea3-8bab-ae7f843a8731'} {{(pid=80144) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 11:05:53.600253 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.notifications [None req-1830d5bf-a163-4ea3-8bab-ae7f843a8731 admin admin] The token cache is being invalidate because project 17ff376c11c440a6bfae804294934fa4 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80144) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 07 11:05:53.600736 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.notifications [None req-1830d5bf-a163-4ea3-8bab-ae7f843a8731 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80144) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 11:05:53.601397 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.notifications [None req-1830d5bf-a163-4ea3-8bab-ae7f843a8731 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80144) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 11:05:53.602926 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 1369/2711] 10.222.0.25 () {58 vars in 1233 bytes} [Sun Aug 7 11:05:53 2022] DELETE /identity/v3/projects/17ff376c11c440a6bfae804294934fa4 => generated 0 bytes in 175 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 07 11:05:53.607610 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5422b2a7-cf91-426e-99ca-9b65138278c3 None None] REQUEST_METHOD: `POST` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 11:05:53.607891 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5422b2a7-cf91-426e-99ca-9b65138278c3 None None] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 11:05:53.608165 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5422b2a7-cf91-426e-99ca-9b65138278c3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 11:05:53.625113 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8def458b-afb4-4f82-81a7-26c9b920c82b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5ac47e3b09f43d19a23a0f457fe9e23', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 11:05:53.625748 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8def458b-afb4-4f82-81a7-26c9b920c82b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 11:05:53.626029 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8def458b-afb4-4f82-81a7-26c9b920c82b admin admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 11:05:53.626301 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8def458b-afb4-4f82-81a7-26c9b920c82b admin admin] PATH_INFO: `/v3/projects/bff93fe614c14e0aa5b551567b08f656` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 11:05:53.627444 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8def458b-afb4-4f82-81a7-26c9b920c82b admin admin] RBAC: Authorizing `identity:delete_project(project_id=bff93fe614c14e0aa5b551567b08f656)` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 11:05:53.628941 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.auth.core [None req-5422b2a7-cf91-426e-99ca-9b65138278c3 None None] MFA Rules not processed for user `bac4e6b76fe64217ac8fa44e71cffe6a`. Rule list: `[]` (Enabled: `True`). {{(pid=80144) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 11:05:53.630169 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.fernet_utils [None req-5422b2a7-cf91-426e-99ca-9b65138278c3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80144) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 11:05:53.630593 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8def458b-afb4-4f82-81a7-26c9b920c82b admin admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 11:05:53.639505 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 1370/2712] 10.222.0.25 () {58 vars in 958 bytes} [Sun Aug 7 11:05:53 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 07 11:05:53.648648 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6c58d55-35dc-4101-af67-266a6595709d None None] REQUEST_METHOD: `POST` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 11:05:53.648930 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6c58d55-35dc-4101-af67-266a6595709d None None] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 11:05:53.649204 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6c58d55-35dc-4101-af67-266a6595709d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 11:05:53.668560 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.auth.core [None req-a6c58d55-35dc-4101-af67-266a6595709d None None] MFA Rules not processed for user `bac4e6b76fe64217ac8fa44e71cffe6a`. Rule list: `[]` (Enabled: `True`). {{(pid=80144) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 11:05:53.669745 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.fernet_utils [None req-a6c58d55-35dc-4101-af67-266a6595709d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80144) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 11:05:53.678950 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-8def458b-afb4-4f82-81a7-26c9b920c82b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'bff93fe614c14e0aa5b551567b08f656', 'request_id': 'req-8def458b-afb4-4f82-81a7-26c9b920c82b'} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 11:05:53.684620 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-8def458b-afb4-4f82-81a7-26c9b920c82b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'bff93fe614c14e0aa5b551567b08f656', 'request_id': 'req-8def458b-afb4-4f82-81a7-26c9b920c82b'} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 11:05:53.691496 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-8def458b-afb4-4f82-81a7-26c9b920c82b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'bff93fe614c14e0aa5b551567b08f656', 'request_id': 'req-8def458b-afb4-4f82-81a7-26c9b920c82b'} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 11:05:53.694032 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:DELETE=11,keystone:SELECT=87,keystone:INSERT=2 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 11:05:53.705916 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-8def458b-afb4-4f82-81a7-26c9b920c82b admin admin] The token cache is being invalidate because project bff93fe614c14e0aa5b551567b08f656 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80143) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 07 11:05:53.706285 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-8def458b-afb4-4f82-81a7-26c9b920c82b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 11:05:53.706916 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-8def458b-afb4-4f82-81a7-26c9b920c82b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 11:05:53.708447 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1343/2713] 10.222.0.25 () {58 vars in 1233 bytes} [Sun Aug 7 11:05:53 2022] DELETE /identity/v3/projects/bff93fe614c14e0aa5b551567b08f656 => generated 0 bytes in 233 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 07 11:05:53.713110 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-956ba0f5-fd89-493b-8944-0b92b63ab585 None None] REQUEST_METHOD: `POST` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 11:05:53.713398 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-956ba0f5-fd89-493b-8944-0b92b63ab585 None None] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 11:05:53.713676 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-956ba0f5-fd89-493b-8944-0b92b63ab585 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 11:05:53.730517 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 1371/2714] 10.222.0.25 () {58 vars in 958 bytes} [Sun Aug 7 11:05:53 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 07 11:05:53.732902 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.auth.core [None req-956ba0f5-fd89-493b-8944-0b92b63ab585 None None] MFA Rules not processed for user `bac4e6b76fe64217ac8fa44e71cffe6a`. Rule list: `[]` (Enabled: `True`). {{(pid=80143) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 11:05:53.734385 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-956ba0f5-fd89-493b-8944-0b92b63ab585 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 11:05:53.735038 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6c58d55-35dc-4101-af67-266a6595709d None None] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 11:05:53.743066 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1344/2715] 10.222.0.25 () {58 vars in 958 bytes} [Sun Aug 7 11:05:53 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 07 11:05:53.749256 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6c58d55-35dc-4101-af67-266a6595709d None None] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 11:05:53.749363 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e631788e-0837-4287-9e36-12d4a75eded9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5ac47e3b09f43d19a23a0f457fe9e23', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 11:05:53.749902 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e631788e-0837-4287-9e36-12d4a75eded9 admin admin] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 11:05:53.750187 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e631788e-0837-4287-9e36-12d4a75eded9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 11:05:53.750467 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf397dc1-c241-4f98-9725-1c0f64eeaaf7 None None] REQUEST_METHOD: `POST` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 11:05:53.750584 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e631788e-0837-4287-9e36-12d4a75eded9 admin admin] PATH_INFO: `/v3/domains` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 11:05:53.750760 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf397dc1-c241-4f98-9725-1c0f64eeaaf7 None None] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 11:05:53.751099 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf397dc1-c241-4f98-9725-1c0f64eeaaf7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 11:05:53.751232 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e631788e-0837-4287-9e36-12d4a75eded9 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80144) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 11:05:53.751450 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e631788e-0837-4287-9e36-12d4a75eded9 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 11:05:53.753027 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e631788e-0837-4287-9e36-12d4a75eded9 admin admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 11:05:53.766576 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 1372/2716] 10.222.0.25 () {58 vars in 1153 bytes} [Sun Aug 7 11:05:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 07 11:05:53.770948 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e631788e-0837-4287-9e36-12d4a75eded9 admin admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 11:05:53.774485 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.auth.core [None req-bf397dc1-c241-4f98-9725-1c0f64eeaaf7 None None] MFA Rules not processed for user `bac4e6b76fe64217ac8fa44e71cffe6a`. Rule list: `[]` (Enabled: `True`). {{(pid=80143) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 11:05:53.775767 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-bf397dc1-c241-4f98-9725-1c0f64eeaaf7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 11:05:53.780790 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e631788e-0837-4287-9e36-12d4a75eded9 admin admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 11:05:53.783870 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1345/2717] 10.222.0.25 () {58 vars in 958 bytes} [Sun Aug 7 11:05:53 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 07 11:05:53.786172 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1ad735c-676a-43c4-8935-3747768d9be4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5ac47e3b09f43d19a23a0f457fe9e23', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 11:05:53.786881 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1ad735c-676a-43c4-8935-3747768d9be4 admin admin] REQUEST_METHOD: `POST` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 11:05:53.787788 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1ad735c-676a-43c4-8935-3747768d9be4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 11:05:53.788072 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1ad735c-676a-43c4-8935-3747768d9be4 admin admin] PATH_INFO: `/v3/projects` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 11:05:53.788302 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf397dc1-c241-4f98-9725-1c0f64eeaaf7 None None] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 11:05:53.788720 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1ad735c-676a-43c4-8935-3747768d9be4 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 11:05:53.790620 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1ad735c-676a-43c4-8935-3747768d9be4 admin admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 11:05:53.797579 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf397dc1-c241-4f98-9725-1c0f64eeaaf7 None None] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 11:05:53.802822 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e62e9915-810e-46d2-aa0e-71c67510883a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5ac47e3b09f43d19a23a0f457fe9e23', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 11:05:53.803451 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e62e9915-810e-46d2-aa0e-71c67510883a admin admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 11:05:53.803776 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e62e9915-810e-46d2-aa0e-71c67510883a admin admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 11:05:53.804061 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e62e9915-810e-46d2-aa0e-71c67510883a admin admin] PATH_INFO: `/v3/domains` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 11:05:53.804714 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e62e9915-810e-46d2-aa0e-71c67510883a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80143) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 11:05:53.805028 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e62e9915-810e-46d2-aa0e-71c67510883a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 11:05:53.806614 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e62e9915-810e-46d2-aa0e-71c67510883a admin admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 11:05:53.819328 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1346/2718] 10.222.0.25 () {58 vars in 1153 bytes} [Sun Aug 7 11:05:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 07 11:05:53.823579 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e62e9915-810e-46d2-aa0e-71c67510883a admin admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 11:05:53.823984 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 1373/2719] 10.222.0.25 () {60 vars in 1153 bytes} [Sun Aug 7 11:05:53 2022] POST /identity/v3/projects => generated 377 bytes in 53 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 07 11:05:53.827823 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1ad735c-676a-43c4-8935-3747768d9be4 admin admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 11:05:53.837403 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e62e9915-810e-46d2-aa0e-71c67510883a admin admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 11:05:53.879105 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 11:05:53.890440 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1ad735c-676a-43c4-8935-3747768d9be4 admin admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 11:05:53.896235 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e32417f-2706-42e9-ab74-b283e084c8a1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5ac47e3b09f43d19a23a0f457fe9e23', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 11:05:53.896853 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e32417f-2706-42e9-ab74-b283e084c8a1 admin admin] REQUEST_METHOD: `POST` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 11:05:53.896988 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfa7555f-7d84-4de5-9830-29635711ef16 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5ac47e3b09f43d19a23a0f457fe9e23', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 11:05:53.897130 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e32417f-2706-42e9-ab74-b283e084c8a1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 11:05:53.897390 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e32417f-2706-42e9-ab74-b283e084c8a1 admin admin] PATH_INFO: `/v3/projects` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 11:05:53.897572 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfa7555f-7d84-4de5-9830-29635711ef16 admin admin] REQUEST_METHOD: `POST` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 11:05:53.897850 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfa7555f-7d84-4de5-9830-29635711ef16 admin admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 11:05:53.898018 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e32417f-2706-42e9-ab74-b283e084c8a1 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 11:05:53.898262 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfa7555f-7d84-4de5-9830-29635711ef16 admin admin] PATH_INFO: `/v3/users` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 11:05:53.898878 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfa7555f-7d84-4de5-9830-29635711ef16 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 11:05:53.900001 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e32417f-2706-42e9-ab74-b283e084c8a1 admin admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 11:05:53.900792 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfa7555f-7d84-4de5-9830-29635711ef16 admin admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 11:05:53.923721 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 1374/2720] 10.222.0.25 () {60 vars in 1144 bytes} [Sun Aug 7 11:05:53 2022] POST /identity/v3/users => generated 289 bytes in 96 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 07 11:05:53.928346 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfa7555f-7d84-4de5-9830-29635711ef16 admin admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 11:05:53.936781 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1347/2721] 10.222.0.25 () {60 vars in 1153 bytes} [Sun Aug 7 11:05:53 2022] POST /identity/v3/projects => generated 371 bytes in 111 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 07 11:05:53.937797 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e32417f-2706-42e9-ab74-b283e084c8a1 admin admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 11:05:53.938325 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfa7555f-7d84-4de5-9830-29635711ef16 admin admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 11:05:53.997800 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca7f397d-73ee-45c9-a8b9-43615c9a3c42 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5ac47e3b09f43d19a23a0f457fe9e23', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 11:05:53.998413 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca7f397d-73ee-45c9-a8b9-43615c9a3c42 admin admin] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 11:05:53.998693 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca7f397d-73ee-45c9-a8b9-43615c9a3c42 admin admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 11:05:53.998994 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca7f397d-73ee-45c9-a8b9-43615c9a3c42 admin admin] PATH_INFO: `/v3/roles` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 11:05:53.999636 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca7f397d-73ee-45c9-a8b9-43615c9a3c42 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 11:05:54.001216 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca7f397d-73ee-45c9-a8b9-43615c9a3c42 admin admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 11:05:54.006497 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e32417f-2706-42e9-ab74-b283e084c8a1 admin admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 11:05:54.011568 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 1375/2722] 10.222.0.25 () {58 vars in 1122 bytes} [Sun Aug 7 11:05:53 2022] GET /identity/v3/roles => generated 1420 bytes in 84 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 07 11:05:54.011824 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec0276df-67e3-4aa3-b2b5-4bdabeb48fb6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5ac47e3b09f43d19a23a0f457fe9e23', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 11:05:54.012434 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec0276df-67e3-4aa3-b2b5-4bdabeb48fb6 admin admin] REQUEST_METHOD: `POST` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 11:05:54.012721 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec0276df-67e3-4aa3-b2b5-4bdabeb48fb6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 11:05:54.012993 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec0276df-67e3-4aa3-b2b5-4bdabeb48fb6 admin admin] PATH_INFO: `/v3/users` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 11:05:54.013625 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec0276df-67e3-4aa3-b2b5-4bdabeb48fb6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 11:05:54.015601 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca7f397d-73ee-45c9-a8b9-43615c9a3c42 admin admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 11:05:54.015965 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec0276df-67e3-4aa3-b2b5-4bdabeb48fb6 admin admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 11:05:54.025907 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca7f397d-73ee-45c9-a8b9-43615c9a3c42 admin admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 11:05:54.031576 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60423eee-29f6-419e-b60d-09e57279dc19 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5ac47e3b09f43d19a23a0f457fe9e23', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 11:05:54.032283 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60423eee-29f6-419e-b60d-09e57279dc19 admin admin] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 11:05:54.032577 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60423eee-29f6-419e-b60d-09e57279dc19 admin admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 11:05:54.032865 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60423eee-29f6-419e-b60d-09e57279dc19 admin admin] PATH_INFO: `/v3/roles` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 11:05:54.033506 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60423eee-29f6-419e-b60d-09e57279dc19 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 11:05:54.035434 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60423eee-29f6-419e-b60d-09e57279dc19 admin admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 11:05:54.037672 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1348/2723] 10.222.0.25 () {60 vars in 1144 bytes} [Sun Aug 7 11:05:53 2022] POST /identity/v3/users => generated 286 bytes in 100 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 07 11:05:54.041688 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec0276df-67e3-4aa3-b2b5-4bdabeb48fb6 admin admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 11:05:54.046857 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 1376/2724] 10.222.0.25 () {58 vars in 1122 bytes} [Sun Aug 7 11:05:54 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 07 11:05:54.050912 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60423eee-29f6-419e-b60d-09e57279dc19 admin admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 11:05:54.052622 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec0276df-67e3-4aa3-b2b5-4bdabeb48fb6 admin admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 11:05:54.057217 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7acebeed-da8b-4ddf-828a-4f6cd429a4ca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5ac47e3b09f43d19a23a0f457fe9e23', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 11:05:54.057834 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7acebeed-da8b-4ddf-828a-4f6cd429a4ca admin admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 11:05:54.058110 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7acebeed-da8b-4ddf-828a-4f6cd429a4ca admin admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 11:05:54.058381 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7acebeed-da8b-4ddf-828a-4f6cd429a4ca admin admin] PATH_INFO: `/v3/roles` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 11:05:54.059045 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7acebeed-da8b-4ddf-828a-4f6cd429a4ca admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 11:05:54.060616 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7acebeed-da8b-4ddf-828a-4f6cd429a4ca admin admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 11:05:54.060734 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60423eee-29f6-419e-b60d-09e57279dc19 admin admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 11:05:54.065394 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74da89f4-7ef0-4de1-8836-2e9e260ddf54 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5ac47e3b09f43d19a23a0f457fe9e23', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 11:05:54.066557 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74da89f4-7ef0-4de1-8836-2e9e260ddf54 admin admin] REQUEST_METHOD: `PUT` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 11:05:54.066848 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74da89f4-7ef0-4de1-8836-2e9e260ddf54 admin admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 11:05:54.067141 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74da89f4-7ef0-4de1-8836-2e9e260ddf54 admin admin] PATH_INFO: `/v3/projects/ea8a464a016e4642b273c1b92d418914/users/bdc645bf0a00442ca1bd24c2f4796e8f/roles/b98dfb3b9fee4644af2887a3ac3fd2e6` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 11:05:54.071140 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1349/2725] 10.222.0.25 () {58 vars in 1122 bytes} [Sun Aug 7 11:05:54 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 07 11:05:54.075039 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7acebeed-da8b-4ddf-828a-4f6cd429a4ca admin admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 11:05:54.084310 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74da89f4-7ef0-4de1-8836-2e9e260ddf54 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ea8a464a016e4642b273c1b92d418914, user_id=bdc645bf0a00442ca1bd24c2f4796e8f, role_id=b98dfb3b9fee4644af2887a3ac3fd2e6)` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 11:05:54.085478 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7acebeed-da8b-4ddf-828a-4f6cd429a4ca admin admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 11:05:54.087004 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74da89f4-7ef0-4de1-8836-2e9e260ddf54 admin admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 11:05:54.091255 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3843621c-aa5e-45cc-bc53-a6f4ec80bdaf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5ac47e3b09f43d19a23a0f457fe9e23', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 11:05:54.091960 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3843621c-aa5e-45cc-bc53-a6f4ec80bdaf admin admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 11:05:54.092300 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3843621c-aa5e-45cc-bc53-a6f4ec80bdaf admin admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 11:05:54.092624 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3843621c-aa5e-45cc-bc53-a6f4ec80bdaf admin admin] PATH_INFO: `/v3/roles` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 11:05:54.093416 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3843621c-aa5e-45cc-bc53-a6f4ec80bdaf admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 11:05:54.095316 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3843621c-aa5e-45cc-bc53-a6f4ec80bdaf admin admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 11:05:54.101795 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 1377/2726] 10.222.0.25 () {60 vars in 1483 bytes} [Sun Aug 7 11:05:54 2022] PUT /identity/v3/projects/ea8a464a016e4642b273c1b92d418914/users/bdc645bf0a00442ca1bd24c2f4796e8f/roles/b98dfb3b9fee4644af2887a3ac3fd2e6 => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 07 11:05:54.107789 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1350/2727] 10.222.0.25 () {58 vars in 1122 bytes} [Sun Aug 7 11:05:54 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 07 11:05:54.113514 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74da89f4-7ef0-4de1-8836-2e9e260ddf54 admin admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 11:05:54.114957 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c21105d-e0cc-44a0-bf8a-aac796fece05 None None] REQUEST_METHOD: `POST` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 11:05:54.115258 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c21105d-e0cc-44a0-bf8a-aac796fece05 None None] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 11:05:54.115535 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c21105d-e0cc-44a0-bf8a-aac796fece05 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 11:05:54.144678 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.auth.core [None req-1c21105d-e0cc-44a0-bf8a-aac796fece05 None None] MFA Rules not processed for user `bdc645bf0a00442ca1bd24c2f4796e8f`. Rule list: `[]` (Enabled: `True`). {{(pid=80143) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 11:05:54.145847 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-1c21105d-e0cc-44a0-bf8a-aac796fece05 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 11:05:54.154275 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:DELETE=1,keystone:SELECT=95,keystone:INSERT=4 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 11:05:54.178558 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74da89f4-7ef0-4de1-8836-2e9e260ddf54 admin admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 11:05:54.183244 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00d9f6e6-0d1a-4001-bc5e-62c88912fe34 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5ac47e3b09f43d19a23a0f457fe9e23', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 11:05:54.183907 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00d9f6e6-0d1a-4001-bc5e-62c88912fe34 admin admin] REQUEST_METHOD: `PUT` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 11:05:54.184183 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00d9f6e6-0d1a-4001-bc5e-62c88912fe34 admin admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 11:05:54.184474 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00d9f6e6-0d1a-4001-bc5e-62c88912fe34 admin admin] PATH_INFO: `/v3/projects/4867894c8e5840c198d233005521fd99/users/c8d7c5d4a1634a71bad9102577bef117/roles/b98dfb3b9fee4644af2887a3ac3fd2e6` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 11:05:54.199025 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00d9f6e6-0d1a-4001-bc5e-62c88912fe34 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4867894c8e5840c198d233005521fd99, user_id=c8d7c5d4a1634a71bad9102577bef117, role_id=b98dfb3b9fee4644af2887a3ac3fd2e6)` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 11:05:54.201635 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00d9f6e6-0d1a-4001-bc5e-62c88912fe34 admin admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 11:05:54.216065 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1351/2728] 10.222.0.25 () {58 vars in 958 bytes} [Sun Aug 7 11:05:54 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 07 11:05:54.216449 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 1378/2729] 10.222.0.25 () {60 vars in 1483 bytes} [Sun Aug 7 11:05:54 2022] PUT /identity/v3/projects/4867894c8e5840c198d233005521fd99/users/c8d7c5d4a1634a71bad9102577bef117/roles/b98dfb3b9fee4644af2887a3ac3fd2e6 => generated 0 bytes in 103 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 07 11:05:54.221176 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c21105d-e0cc-44a0-bf8a-aac796fece05 None None] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 11:05:54.221701 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4208f207-4cb0-41db-a1a4-19b09d9f18d1 None None] REQUEST_METHOD: `POST` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 11:05:54.222196 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4208f207-4cb0-41db-a1a4-19b09d9f18d1 None None] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 11:05:54.222473 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4208f207-4cb0-41db-a1a4-19b09d9f18d1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 11:05:54.231978 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c21105d-e0cc-44a0-bf8a-aac796fece05 None None] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 11:05:54.241462 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.auth.core [None req-4208f207-4cb0-41db-a1a4-19b09d9f18d1 None None] MFA Rules not processed for user `bac4e6b76fe64217ac8fa44e71cffe6a`. Rule list: `[]` (Enabled: `True`). {{(pid=80144) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 11:05:54.242649 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.fernet_utils [None req-4208f207-4cb0-41db-a1a4-19b09d9f18d1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80144) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 11:05:54.264998 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 11:05:54.291730 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2227a267-19e9-4d1e-bf73-9ed88059e904 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5ac47e3b09f43d19a23a0f457fe9e23', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 11:05:54.292341 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2227a267-19e9-4d1e-bf73-9ed88059e904 admin admin] REQUEST_METHOD: `POST` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 11:05:54.292620 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2227a267-19e9-4d1e-bf73-9ed88059e904 admin admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 11:05:54.292892 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2227a267-19e9-4d1e-bf73-9ed88059e904 admin admin] PATH_INFO: `/v3/projects` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 11:05:54.293515 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2227a267-19e9-4d1e-bf73-9ed88059e904 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 11:05:54.295441 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2227a267-19e9-4d1e-bf73-9ed88059e904 admin admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 11:05:54.306681 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 1379/2730] 10.222.0.25 () {58 vars in 958 bytes} [Sun Aug 7 11:05:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 07 11:05:54.317397 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4208f207-4cb0-41db-a1a4-19b09d9f18d1 None None] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 11:05:54.319009 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.fernet_utils [None req-4208f207-4cb0-41db-a1a4-19b09d9f18d1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80144) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 11:05:54.324344 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1352/2731] 10.222.0.25 () {60 vars in 1153 bytes} [Sun Aug 7 11:05:54 2022] POST /identity/v3/projects => generated 375 bytes in 104 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 07 11:05:54.328185 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2227a267-19e9-4d1e-bf73-9ed88059e904 admin admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 11:05:54.382187 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4208f207-4cb0-41db-a1a4-19b09d9f18d1 None None] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 11:05:54.389927 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2227a267-19e9-4d1e-bf73-9ed88059e904 admin admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 11:05:54.395303 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48ea1d0b-8a87-4c9a-9071-caa1c96aa894 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5ac47e3b09f43d19a23a0f457fe9e23', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 11:05:54.395840 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48ea1d0b-8a87-4c9a-9071-caa1c96aa894 admin admin] REQUEST_METHOD: `POST` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 11:05:54.396117 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48ea1d0b-8a87-4c9a-9071-caa1c96aa894 admin admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 11:05:54.396390 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48ea1d0b-8a87-4c9a-9071-caa1c96aa894 admin admin] PATH_INFO: `/v3/users` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 11:05:54.397002 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48ea1d0b-8a87-4c9a-9071-caa1c96aa894 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 11:05:54.398956 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48ea1d0b-8a87-4c9a-9071-caa1c96aa894 admin admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 11:05:54.426391 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1353/2732] 10.222.0.25 () {60 vars in 1144 bytes} [Sun Aug 7 11:05:54 2022] POST /identity/v3/users => generated 288 bytes in 95 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 07 11:05:54.427566 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48ea1d0b-8a87-4c9a-9071-caa1c96aa894 admin admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 11:05:54.435100 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f547135d-3f6a-4f95-a108-2b3c1b9c2abf service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be1283c9bf47424c9aa3ad27cab03027', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6de1f3cac02c4bcdb05dd2337176033d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 11:05:54.435707 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f547135d-3f6a-4f95-a108-2b3c1b9c2abf service neutron] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 11:05:54.435986 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f547135d-3f6a-4f95-a108-2b3c1b9c2abf service neutron] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 11:05:54.436256 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f547135d-3f6a-4f95-a108-2b3c1b9c2abf service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 11:05:54.437059 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48ea1d0b-8a87-4c9a-9071-caa1c96aa894 admin admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 11:05:54.442630 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3321916-69e6-4788-8cbd-9e39718c4257 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5ac47e3b09f43d19a23a0f457fe9e23', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 11:05:54.443256 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3321916-69e6-4788-8cbd-9e39718c4257 admin admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 11:05:54.443537 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3321916-69e6-4788-8cbd-9e39718c4257 admin admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 11:05:54.443810 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3321916-69e6-4788-8cbd-9e39718c4257 admin admin] PATH_INFO: `/v3/roles` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 11:05:54.444446 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3321916-69e6-4788-8cbd-9e39718c4257 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 11:05:54.446029 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3321916-69e6-4788-8cbd-9e39718c4257 admin admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 11:05:54.446223 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f547135d-3f6a-4f95-a108-2b3c1b9c2abf service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 11:05:54.447648 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f547135d-3f6a-4f95-a108-2b3c1b9c2abf service neutron] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 11:05:54.457016 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1354/2733] 10.222.0.25 () {58 vars in 1122 bytes} [Sun Aug 7 11:05:54 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 07 11:05:54.460029 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 1380/2734] 10.222.0.25 () {60 vars in 1365 bytes} [Sun Aug 7 11:05:54 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 140 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 07 11:05:54.463903 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f547135d-3f6a-4f95-a108-2b3c1b9c2abf service neutron] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 11:05:54.473845 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f547135d-3f6a-4f95-a108-2b3c1b9c2abf service neutron] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 11:05:54.478484 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeeb4bce-95c8-4e63-b174-37e43bcebeb7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5ac47e3b09f43d19a23a0f457fe9e23', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 11:05:54.479124 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeeb4bce-95c8-4e63-b174-37e43bcebeb7 admin admin] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 11:05:54.479405 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeeb4bce-95c8-4e63-b174-37e43bcebeb7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 11:05:54.479675 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeeb4bce-95c8-4e63-b174-37e43bcebeb7 admin admin] PATH_INFO: `/v3/roles` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 11:05:54.480305 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eeeb4bce-95c8-4e63-b174-37e43bcebeb7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 11:05:54.481878 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eeeb4bce-95c8-4e63-b174-37e43bcebeb7 admin admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 11:05:54.498857 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 1381/2735] 10.222.0.25 () {58 vars in 1122 bytes} [Sun Aug 7 11:05:54 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 07 11:05:54.500047 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3321916-69e6-4788-8cbd-9e39718c4257 admin admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 11:05:54.511550 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3321916-69e6-4788-8cbd-9e39718c4257 admin admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 11:05:54.516761 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bd02def-5fd1-4cea-b07b-034f1a8a1567 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5ac47e3b09f43d19a23a0f457fe9e23', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 11:05:54.517622 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bd02def-5fd1-4cea-b07b-034f1a8a1567 admin admin] REQUEST_METHOD: `PUT` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 11:05:54.517996 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bd02def-5fd1-4cea-b07b-034f1a8a1567 admin admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 11:05:54.518464 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bd02def-5fd1-4cea-b07b-034f1a8a1567 admin admin] PATH_INFO: `/v3/projects/4750603e9555454695ecc1f56e9185ca/users/1b29247339dc40739f05cd5af0134f6b/roles/b98dfb3b9fee4644af2887a3ac3fd2e6` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 11:05:54.531166 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 11:05:54.536584 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bd02def-5fd1-4cea-b07b-034f1a8a1567 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4750603e9555454695ecc1f56e9185ca, user_id=1b29247339dc40739f05cd5af0134f6b, role_id=b98dfb3b9fee4644af2887a3ac3fd2e6)` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 11:05:54.539552 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bd02def-5fd1-4cea-b07b-034f1a8a1567 admin admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 11:05:54.556784 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1355/2736] 10.222.0.25 () {60 vars in 1483 bytes} [Sun Aug 7 11:05:54 2022] PUT /identity/v3/projects/4750603e9555454695ecc1f56e9185ca/users/1b29247339dc40739f05cd5af0134f6b/roles/b98dfb3b9fee4644af2887a3ac3fd2e6 => generated 0 bytes in 57 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 07 11:05:54.567150 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffe4f210-714d-445d-a227-f24e3cf2b667 None None] REQUEST_METHOD: `POST` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 11:05:54.567651 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffe4f210-714d-445d-a227-f24e3cf2b667 None None] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 11:05:54.567850 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffe4f210-714d-445d-a227-f24e3cf2b667 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 11:05:54.598285 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.auth.core [None req-ffe4f210-714d-445d-a227-f24e3cf2b667 None None] MFA Rules not processed for user `1b29247339dc40739f05cd5af0134f6b`. Rule list: `[]` (Enabled: `True`). {{(pid=80144) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 11:05:54.599604 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.fernet_utils [None req-ffe4f210-714d-445d-a227-f24e3cf2b667 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80144) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 11:05:54.669519 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 1382/2737] 10.222.0.25 () {58 vars in 958 bytes} [Sun Aug 7 11:05:54 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 07 11:05:54.674284 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bd02def-5fd1-4cea-b07b-034f1a8a1567 admin admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 11:05:54.743745 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bd02def-5fd1-4cea-b07b-034f1a8a1567 admin admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 11:05:54.748408 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf32fdea-53c7-4a64-bb45-01dddbe7928c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5ac47e3b09f43d19a23a0f457fe9e23', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 11:05:54.749018 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf32fdea-53c7-4a64-bb45-01dddbe7928c admin admin] REQUEST_METHOD: `POST` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 11:05:54.749299 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf32fdea-53c7-4a64-bb45-01dddbe7928c admin admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 11:05:54.749575 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf32fdea-53c7-4a64-bb45-01dddbe7928c admin admin] PATH_INFO: `/v3/projects` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 11:05:54.750215 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf32fdea-53c7-4a64-bb45-01dddbe7928c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 11:05:54.752180 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf32fdea-53c7-4a64-bb45-01dddbe7928c admin admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 11:05:54.787512 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1356/2738] 10.222.0.25 () {60 vars in 1153 bytes} [Sun Aug 7 11:05:54 2022] POST /identity/v3/projects => generated 377 bytes in 114 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 07 11:05:54.792147 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffe4f210-714d-445d-a227-f24e3cf2b667 None None] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 11:05:54.804905 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=100 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 11:05:54.813625 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffe4f210-714d-445d-a227-f24e3cf2b667 None None] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 11:05:54.880385 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f4ff81a-854d-4d34-ba9e-c21a77066813 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5ac47e3b09f43d19a23a0f457fe9e23', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 11:05:54.881097 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f4ff81a-854d-4d34-ba9e-c21a77066813 admin admin] REQUEST_METHOD: `POST` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 11:05:54.881376 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f4ff81a-854d-4d34-ba9e-c21a77066813 admin admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 11:05:54.881669 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f4ff81a-854d-4d34-ba9e-c21a77066813 admin admin] PATH_INFO: `/v3/users` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 11:05:54.882423 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f4ff81a-854d-4d34-ba9e-c21a77066813 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 11:05:54.884439 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f4ff81a-854d-4d34-ba9e-c21a77066813 admin admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 11:05:54.910962 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 1383/2739] 10.222.0.25 () {60 vars in 1144 bytes} [Sun Aug 7 11:05:54 2022] POST /identity/v3/users => generated 295 bytes in 117 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 07 11:05:54.912297 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf32fdea-53c7-4a64-bb45-01dddbe7928c admin admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 11:05:54.922994 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf32fdea-53c7-4a64-bb45-01dddbe7928c admin admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 11:05:54.927890 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b65c1b2-bea7-42cf-b258-2fde3f6e684f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5ac47e3b09f43d19a23a0f457fe9e23', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 11:05:54.928497 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b65c1b2-bea7-42cf-b258-2fde3f6e684f admin admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 11:05:54.928779 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b65c1b2-bea7-42cf-b258-2fde3f6e684f admin admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 11:05:54.929053 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b65c1b2-bea7-42cf-b258-2fde3f6e684f admin admin] PATH_INFO: `/v3/roles` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 11:05:54.929701 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b65c1b2-bea7-42cf-b258-2fde3f6e684f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 11:05:54.931328 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b65c1b2-bea7-42cf-b258-2fde3f6e684f admin admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 11:05:54.943120 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1357/2740] 10.222.0.25 () {58 vars in 1122 bytes} [Sun Aug 7 11:05:54 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 07 11:05:54.947272 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f4ff81a-854d-4d34-ba9e-c21a77066813 admin admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 11:05:54.958508 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f4ff81a-854d-4d34-ba9e-c21a77066813 admin admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 11:05:54.963556 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e577db00-d714-4683-93d7-4b9958c15eae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5ac47e3b09f43d19a23a0f457fe9e23', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 11:05:54.964219 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e577db00-d714-4683-93d7-4b9958c15eae admin admin] REQUEST_METHOD: `PUT` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 11:05:54.964500 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e577db00-d714-4683-93d7-4b9958c15eae admin admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 11:05:54.964777 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e577db00-d714-4683-93d7-4b9958c15eae admin admin] PATH_INFO: `/v3/projects/14a01bbaecbc4cca972eaf82fdd0460d/users/b007599b6be249eb84eca414d0c51b59/roles/81b1f1bcee2a473088fbbd327c1c5b23` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 11:05:54.980233 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e577db00-d714-4683-93d7-4b9958c15eae admin admin] RBAC: Authorizing `identity:create_grant(project_id=14a01bbaecbc4cca972eaf82fdd0460d, user_id=b007599b6be249eb84eca414d0c51b59, role_id=81b1f1bcee2a473088fbbd327c1c5b23)` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 11:05:54.983044 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e577db00-d714-4683-93d7-4b9958c15eae admin admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 11:05:54.996105 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 1384/2741] 10.222.0.25 () {60 vars in 1483 bytes} [Sun Aug 7 11:05:54 2022] PUT /identity/v3/projects/14a01bbaecbc4cca972eaf82fdd0460d/users/b007599b6be249eb84eca414d0c51b59/roles/81b1f1bcee2a473088fbbd327c1c5b23 => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 07 11:05:55.000619 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b65c1b2-bea7-42cf-b258-2fde3f6e684f admin admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 11:05:55.012862 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b65c1b2-bea7-42cf-b258-2fde3f6e684f admin admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 11:05:55.074585 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0203090-ceb8-441e-8885-fe97e223917f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5ac47e3b09f43d19a23a0f457fe9e23', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 11:05:55.075236 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0203090-ceb8-441e-8885-fe97e223917f admin admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 11:05:55.075519 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0203090-ceb8-441e-8885-fe97e223917f admin admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 11:05:55.075795 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0203090-ceb8-441e-8885-fe97e223917f admin admin] PATH_INFO: `/v3/roles` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 11:05:55.076444 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0203090-ceb8-441e-8885-fe97e223917f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 11:05:55.078042 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0203090-ceb8-441e-8885-fe97e223917f admin admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 11:05:55.089312 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1358/2742] 10.222.0.25 () {58 vars in 1122 bytes} [Sun Aug 7 11:05:54 2022] GET /identity/v3/roles => generated 1420 bytes in 90 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 07 11:05:55.093700 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e577db00-d714-4683-93d7-4b9958c15eae admin admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 11:05:55.104791 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e577db00-d714-4683-93d7-4b9958c15eae admin admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 11:05:55.112223 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2db5deb-74b8-4f43-b4ac-f51713e15842 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5ac47e3b09f43d19a23a0f457fe9e23', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 11:05:55.113401 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2db5deb-74b8-4f43-b4ac-f51713e15842 admin admin] REQUEST_METHOD: `PUT` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 11:05:55.113684 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2db5deb-74b8-4f43-b4ac-f51713e15842 admin admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 11:05:55.113983 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2db5deb-74b8-4f43-b4ac-f51713e15842 admin admin] PATH_INFO: `/v3/projects/14a01bbaecbc4cca972eaf82fdd0460d/users/b007599b6be249eb84eca414d0c51b59/roles/81b1f1bcee2a473088fbbd327c1c5b23` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 11:05:55.116075 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2db5deb-74b8-4f43-b4ac-f51713e15842 admin admin] RBAC: Authorizing `identity:create_grant(project_id=14a01bbaecbc4cca972eaf82fdd0460d, user_id=b007599b6be249eb84eca414d0c51b59, role_id=81b1f1bcee2a473088fbbd327c1c5b23)` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 11:05:55.118868 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2db5deb-74b8-4f43-b4ac-f51713e15842 admin admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 11:05:55.130510 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7530af5c-98d6-41d0-83f5-2ad7e40d6929 None None] REQUEST_METHOD: `POST` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 11:05:55.130650 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 1385/2743] 10.222.0.25 () {60 vars in 1483 bytes} [Sun Aug 7 11:05:55 2022] PUT /identity/v3/projects/14a01bbaecbc4cca972eaf82fdd0460d/users/b007599b6be249eb84eca414d0c51b59/roles/81b1f1bcee2a473088fbbd327c1c5b23 => generated 0 bytes in 37 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 07 11:05:55.130796 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7530af5c-98d6-41d0-83f5-2ad7e40d6929 None None] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 11:05:55.131102 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7530af5c-98d6-41d0-83f5-2ad7e40d6929 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 11:05:55.141782 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad399cc5-6832-45fa-a42d-6acea80cc65c None None] REQUEST_METHOD: `POST` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 11:05:55.142109 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad399cc5-6832-45fa-a42d-6acea80cc65c None None] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 11:05:55.142339 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad399cc5-6832-45fa-a42d-6acea80cc65c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 11:05:55.164141 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.auth.core [None req-7530af5c-98d6-41d0-83f5-2ad7e40d6929 None None] MFA Rules not processed for user `c8d7c5d4a1634a71bad9102577bef117`. Rule list: `[]` (Enabled: `True`). {{(pid=80143) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 11:05:55.165276 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-7530af5c-98d6-41d0-83f5-2ad7e40d6929 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 11:05:55.175970 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.auth.core [None req-ad399cc5-6832-45fa-a42d-6acea80cc65c None None] MFA Rules not processed for user `b007599b6be249eb84eca414d0c51b59`. Rule list: `[]` (Enabled: `True`). {{(pid=80144) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 11:05:55.177290 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.fernet_utils [None req-ad399cc5-6832-45fa-a42d-6acea80cc65c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80144) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 11:05:55.267020 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 1386/2744] 10.222.0.25 () {58 vars in 958 bytes} [Sun Aug 7 11:05:55 2022] POST /identity/v3/auth/tokens => generated 3321 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 07 11:05:55.280493 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad399cc5-6832-45fa-a42d-6acea80cc65c None None] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 11:05:55.282172 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.fernet_utils [None req-ad399cc5-6832-45fa-a42d-6acea80cc65c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80144) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 11:05:55.336447 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 11:05:55.349980 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad399cc5-6832-45fa-a42d-6acea80cc65c None None] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 11:05:55.353692 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cd57a04-2088-44d7-ad3a-433897eabbb7 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88dbd6dfd1164994a3e16da553561910', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6de1f3cac02c4bcdb05dd2337176033d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 11:05:55.354233 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cd57a04-2088-44d7-ad3a-433897eabbb7 service cinder] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 11:05:55.354504 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cd57a04-2088-44d7-ad3a-433897eabbb7 service cinder] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 11:05:55.354772 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cd57a04-2088-44d7-ad3a-433897eabbb7 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 11:05:55.360649 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 11:05:55.381561 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1359/2745] 10.222.0.25 () {58 vars in 958 bytes} [Sun Aug 7 11:05:55 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 252 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 07 11:05:55.392462 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7530af5c-98d6-41d0-83f5-2ad7e40d6929 None None] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 11:05:55.393843 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-7530af5c-98d6-41d0-83f5-2ad7e40d6929 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 11:05:55.406408 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7530af5c-98d6-41d0-83f5-2ad7e40d6929 None None] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 11:05:55.414522 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97b21c8d-8a52-4b22-bec8-eaef2cf5c413 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88dbd6dfd1164994a3e16da553561910', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6de1f3cac02c4bcdb05dd2337176033d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 11:05:55.414762 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cd57a04-2088-44d7-ad3a-433897eabbb7 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 11:05:55.415093 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97b21c8d-8a52-4b22-bec8-eaef2cf5c413 service cinder] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 11:05:55.415375 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97b21c8d-8a52-4b22-bec8-eaef2cf5c413 service cinder] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 11:05:55.415640 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97b21c8d-8a52-4b22-bec8-eaef2cf5c413 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 11:05:55.416296 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cd57a04-2088-44d7-ad3a-433897eabbb7 service cinder] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 11:05:55.424603 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97b21c8d-8a52-4b22-bec8-eaef2cf5c413 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 11:05:55.426658 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 1387/2746] 10.222.0.25 () {60 vars in 1365 bytes} [Sun Aug 7 11:05:55 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 147 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 07 11:05:55.426860 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97b21c8d-8a52-4b22-bec8-eaef2cf5c413 service cinder] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 11:05:55.435948 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1360/2747] 10.222.0.25 () {60 vars in 1365 bytes} [Sun Aug 7 11:05:55 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 07 11:05:56.944020 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cd57a04-2088-44d7-ad3a-433897eabbb7 service cinder] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 11:05:56.952560 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cd57a04-2088-44d7-ad3a-433897eabbb7 service cinder] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 11:05:56.956200 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c549baae-7118-4ba4-834b-64be3f757d00 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88dbd6dfd1164994a3e16da553561910', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6de1f3cac02c4bcdb05dd2337176033d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 11:05:56.956794 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c549baae-7118-4ba4-834b-64be3f757d00 service cinder] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 11:05:56.957070 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c549baae-7118-4ba4-834b-64be3f757d00 service cinder] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 11:05:56.957341 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c549baae-7118-4ba4-834b-64be3f757d00 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 11:05:57.015114 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c549baae-7118-4ba4-834b-64be3f757d00 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 11:05:57.016554 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c549baae-7118-4ba4-834b-64be3f757d00 service cinder] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 11:05:57.025668 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 1388/2748] 10.222.0.25 () {60 vars in 1365 bytes} [Sun Aug 7 11:05:56 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 82 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 07 11:05:57.553878 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97b21c8d-8a52-4b22-bec8-eaef2cf5c413 service cinder] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 11:05:57.567437 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97b21c8d-8a52-4b22-bec8-eaef2cf5c413 service cinder] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 11:05:57.571042 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-502b80ef-5f85-4d0e-ba74-9084f8f501bd service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88dbd6dfd1164994a3e16da553561910', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6de1f3cac02c4bcdb05dd2337176033d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 11:05:57.571636 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-502b80ef-5f85-4d0e-ba74-9084f8f501bd service cinder] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 11:05:57.571910 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-502b80ef-5f85-4d0e-ba74-9084f8f501bd service cinder] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 11:05:57.572175 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-502b80ef-5f85-4d0e-ba74-9084f8f501bd service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 11:05:57.581007 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-502b80ef-5f85-4d0e-ba74-9084f8f501bd service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 11:05:57.582393 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-502b80ef-5f85-4d0e-ba74-9084f8f501bd service cinder] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 11:05:57.591431 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1361/2749] 10.222.0.25 () {60 vars in 1365 bytes} [Sun Aug 7 11:05:57 2022] GET /identity/v3/auth/tokens => generated 3321 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 07 11:05:57.873756 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcd03349-b69b-414c-b7eb-1302320ce03d None None] REQUEST_METHOD: `POST` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 11:05:57.874142 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcd03349-b69b-414c-b7eb-1302320ce03d None None] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 11:05:57.874450 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcd03349-b69b-414c-b7eb-1302320ce03d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 11:05:57.893777 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.auth.core [None req-fcd03349-b69b-414c-b7eb-1302320ce03d None None] MFA Rules not processed for user `c8d7c5d4a1634a71bad9102577bef117`. Rule list: `[]` (Enabled: `True`). {{(pid=80144) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 11:05:57.895220 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.fernet_utils [None req-fcd03349-b69b-414c-b7eb-1302320ce03d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80144) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 11:05:57.902235 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 1389/2750] 10.222.0.25 () {58 vars in 958 bytes} [Sun Aug 7 11:05:57 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 07 11:05:58.139726 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-502b80ef-5f85-4d0e-ba74-9084f8f501bd service cinder] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 11:05:58.193213 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-502b80ef-5f85-4d0e-ba74-9084f8f501bd service cinder] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 11:05:58.196749 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-002f09bc-596f-4e9a-8bd1-56272343d58c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be1283c9bf47424c9aa3ad27cab03027', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6de1f3cac02c4bcdb05dd2337176033d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 11:05:58.197337 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-002f09bc-596f-4e9a-8bd1-56272343d58c service neutron] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 11:05:58.197612 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-002f09bc-596f-4e9a-8bd1-56272343d58c service neutron] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 11:05:58.197880 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-002f09bc-596f-4e9a-8bd1-56272343d58c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 11:05:58.199592 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-002f09bc-596f-4e9a-8bd1-56272343d58c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 11:05:58.209406 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-002f09bc-596f-4e9a-8bd1-56272343d58c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 11:05:58.210799 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-002f09bc-596f-4e9a-8bd1-56272343d58c service neutron] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 11:05:58.219192 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1362/2751] 10.222.0.25 () {60 vars in 1365 bytes} [Sun Aug 7 11:05:58 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 80 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 07 11:06:01.504316 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcd03349-b69b-414c-b7eb-1302320ce03d None None] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 11:06:01.568075 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcd03349-b69b-414c-b7eb-1302320ce03d None None] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 11:06:01.573502 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4032d607-42f4-4dc5-8b76-05aab61746fb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5ac47e3b09f43d19a23a0f457fe9e23', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 11:06:01.574140 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4032d607-42f4-4dc5-8b76-05aab61746fb admin admin] REQUEST_METHOD: `DELETE` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 11:06:01.574418 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4032d607-42f4-4dc5-8b76-05aab61746fb admin admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 11:06:01.574691 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4032d607-42f4-4dc5-8b76-05aab61746fb admin admin] PATH_INFO: `/v3/users/bdc645bf0a00442ca1bd24c2f4796e8f` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 11:06:01.575846 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4032d607-42f4-4dc5-8b76-05aab61746fb admin admin] RBAC: Authorizing `identity:delete_user(user_id=bdc645bf0a00442ca1bd24c2f4796e8f)` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 11:06:01.577762 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4032d607-42f4-4dc5-8b76-05aab61746fb admin admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 11:06:01.613224 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=95,keystone:DELETE=5 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 11:06:01.626796 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.notifications [None req-4032d607-42f4-4dc5-8b76-05aab61746fb admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'bdc645bf0a00442ca1bd24c2f4796e8f'} {{(pid=80144) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 11:06:01.634087 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.notifications [None req-4032d607-42f4-4dc5-8b76-05aab61746fb admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'bdc645bf0a00442ca1bd24c2f4796e8f'} {{(pid=80144) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 11:06:01.641402 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.notifications [None req-4032d607-42f4-4dc5-8b76-05aab61746fb admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'bdc645bf0a00442ca1bd24c2f4796e8f'} {{(pid=80144) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 11:06:01.642017 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.notifications [None req-4032d607-42f4-4dc5-8b76-05aab61746fb admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'bdc645bf0a00442ca1bd24c2f4796e8f'} {{(pid=80144) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 11:06:01.642621 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.notifications [None req-4032d607-42f4-4dc5-8b76-05aab61746fb admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'bdc645bf0a00442ca1bd24c2f4796e8f'} {{(pid=80144) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 11:06:01.660883 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 1390/2752] 10.222.0.25 () {58 vars in 1224 bytes} [Sun Aug 7 11:06:01 2022] DELETE /identity/v3/users/bdc645bf0a00442ca1bd24c2f4796e8f => generated 0 bytes in 158 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 07 11:06:01.665330 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd7d141d-a745-4848-8d93-79eacfedae09 None None] REQUEST_METHOD: `POST` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 11:06:01.665785 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd7d141d-a745-4848-8d93-79eacfedae09 None None] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 11:06:01.665881 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd7d141d-a745-4848-8d93-79eacfedae09 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 11:06:01.684093 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.auth.core [None req-bd7d141d-a745-4848-8d93-79eacfedae09 None None] MFA Rules not processed for user `bac4e6b76fe64217ac8fa44e71cffe6a`. Rule list: `[]` (Enabled: `True`). {{(pid=80143) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 11:06:01.685271 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-bd7d141d-a745-4848-8d93-79eacfedae09 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 11:06:01.743083 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1363/2753] 10.222.0.25 () {58 vars in 958 bytes} [Sun Aug 7 11:06:01 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 07 11:06:01.753510 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4032d607-42f4-4dc5-8b76-05aab61746fb admin admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 11:06:01.754905 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.fernet_utils [None req-4032d607-42f4-4dc5-8b76-05aab61746fb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80144) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 11:06:01.811015 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4032d607-42f4-4dc5-8b76-05aab61746fb admin admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 11:06:01.814894 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f4b1c64-34a5-4080-b416-947190f9b411 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be1283c9bf47424c9aa3ad27cab03027', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6de1f3cac02c4bcdb05dd2337176033d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 11:06:01.816202 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f4b1c64-34a5-4080-b416-947190f9b411 service neutron] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 11:06:01.816489 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f4b1c64-34a5-4080-b416-947190f9b411 service neutron] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 11:06:01.816767 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f4b1c64-34a5-4080-b416-947190f9b411 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 11:06:01.826068 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f4b1c64-34a5-4080-b416-947190f9b411 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 11:06:01.827533 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f4b1c64-34a5-4080-b416-947190f9b411 service neutron] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 11:06:01.837400 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 1391/2754] 10.222.0.25 () {60 vars in 1365 bytes} [Sun Aug 7 11:06:01 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 07 11:06:02.108383 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd7d141d-a745-4848-8d93-79eacfedae09 None None] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 11:06:02.110085 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-bd7d141d-a745-4848-8d93-79eacfedae09 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 11:06:02.123219 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd7d141d-a745-4848-8d93-79eacfedae09 None None] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 11:06:02.129267 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59bdcb3d-165e-4529-ad64-4431fad08b7c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5ac47e3b09f43d19a23a0f457fe9e23', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 11:06:02.129902 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59bdcb3d-165e-4529-ad64-4431fad08b7c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 11:06:02.130181 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59bdcb3d-165e-4529-ad64-4431fad08b7c admin admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 11:06:02.130453 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59bdcb3d-165e-4529-ad64-4431fad08b7c admin admin] PATH_INFO: `/v3/projects/ea8a464a016e4642b273c1b92d418914` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 11:06:02.131637 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59bdcb3d-165e-4529-ad64-4431fad08b7c admin admin] RBAC: Authorizing `identity:delete_project(project_id=ea8a464a016e4642b273c1b92d418914)` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 11:06:02.133527 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59bdcb3d-165e-4529-ad64-4431fad08b7c admin admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 11:06:02.180046 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-59bdcb3d-165e-4529-ad64-4431fad08b7c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ea8a464a016e4642b273c1b92d418914', 'request_id': 'req-59bdcb3d-165e-4529-ad64-4431fad08b7c'} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 11:06:02.185383 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-59bdcb3d-165e-4529-ad64-4431fad08b7c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ea8a464a016e4642b273c1b92d418914', 'request_id': 'req-59bdcb3d-165e-4529-ad64-4431fad08b7c'} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 11:06:02.187275 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=94,keystone:DELETE=6 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 11:06:02.193049 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-59bdcb3d-165e-4529-ad64-4431fad08b7c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ea8a464a016e4642b273c1b92d418914', 'request_id': 'req-59bdcb3d-165e-4529-ad64-4431fad08b7c'} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 11:06:02.205768 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-59bdcb3d-165e-4529-ad64-4431fad08b7c admin admin] The token cache is being invalidate because project ea8a464a016e4642b273c1b92d418914 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80143) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 07 11:06:02.206067 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-59bdcb3d-165e-4529-ad64-4431fad08b7c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 11:06:02.206680 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-59bdcb3d-165e-4529-ad64-4431fad08b7c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 11:06:02.208265 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1364/2755] 10.222.0.25 () {58 vars in 1233 bytes} [Sun Aug 7 11:06:02 2022] DELETE /identity/v3/projects/ea8a464a016e4642b273c1b92d418914 => generated 0 bytes in 101 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 07 11:06:02.212255 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f4b1c64-34a5-4080-b416-947190f9b411 service neutron] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 11:06:02.222600 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f4b1c64-34a5-4080-b416-947190f9b411 service neutron] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 11:06:02.280454 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-496de77d-48ce-4b36-adbc-a139903100ec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5ac47e3b09f43d19a23a0f457fe9e23', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 11:06:02.281015 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-496de77d-48ce-4b36-adbc-a139903100ec admin admin] REQUEST_METHOD: `DELETE` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 11:06:02.281299 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-496de77d-48ce-4b36-adbc-a139903100ec admin admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 11:06:02.281573 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-496de77d-48ce-4b36-adbc-a139903100ec admin admin] PATH_INFO: `/v3/users/1b29247339dc40739f05cd5af0134f6b` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 11:06:02.282713 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-496de77d-48ce-4b36-adbc-a139903100ec admin admin] RBAC: Authorizing `identity:delete_user(user_id=1b29247339dc40739f05cd5af0134f6b)` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 11:06:02.285356 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-496de77d-48ce-4b36-adbc-a139903100ec admin admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 11:06:02.332764 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.notifications [None req-496de77d-48ce-4b36-adbc-a139903100ec admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1b29247339dc40739f05cd5af0134f6b'} {{(pid=80144) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 11:06:02.337311 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 11:06:02.340933 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.notifications [None req-496de77d-48ce-4b36-adbc-a139903100ec admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1b29247339dc40739f05cd5af0134f6b'} {{(pid=80144) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 11:06:02.350271 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.notifications [None req-496de77d-48ce-4b36-adbc-a139903100ec admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1b29247339dc40739f05cd5af0134f6b'} {{(pid=80144) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 11:06:02.351006 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.notifications [None req-496de77d-48ce-4b36-adbc-a139903100ec admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1b29247339dc40739f05cd5af0134f6b'} {{(pid=80144) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 11:06:02.351676 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.notifications [None req-496de77d-48ce-4b36-adbc-a139903100ec admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1b29247339dc40739f05cd5af0134f6b'} {{(pid=80144) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 11:06:02.371072 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 1392/2756] 10.222.0.25 () {58 vars in 1224 bytes} [Sun Aug 7 11:06:02 2022] DELETE /identity/v3/users/1b29247339dc40739f05cd5af0134f6b => generated 0 bytes in 159 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 07 11:06:02.650636 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59bdcb3d-165e-4529-ad64-4431fad08b7c admin admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 11:06:02.652156 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-59bdcb3d-165e-4529-ad64-4431fad08b7c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 11:06:02.667333 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59bdcb3d-165e-4529-ad64-4431fad08b7c admin admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 11:06:02.726040 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c93f4f2-e53c-4382-a9cb-cfe64ed26fb0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5ac47e3b09f43d19a23a0f457fe9e23', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 11:06:02.726671 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c93f4f2-e53c-4382-a9cb-cfe64ed26fb0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 11:06:02.726976 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c93f4f2-e53c-4382-a9cb-cfe64ed26fb0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 11:06:02.727275 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c93f4f2-e53c-4382-a9cb-cfe64ed26fb0 admin admin] PATH_INFO: `/v3/projects/4750603e9555454695ecc1f56e9185ca` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 11:06:02.728411 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c93f4f2-e53c-4382-a9cb-cfe64ed26fb0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=4750603e9555454695ecc1f56e9185ca)` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 11:06:02.730347 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c93f4f2-e53c-4382-a9cb-cfe64ed26fb0 admin admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 11:06:02.775308 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-5c93f4f2-e53c-4382-a9cb-cfe64ed26fb0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4750603e9555454695ecc1f56e9185ca', 'request_id': 'req-5c93f4f2-e53c-4382-a9cb-cfe64ed26fb0'} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 11:06:02.780728 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-5c93f4f2-e53c-4382-a9cb-cfe64ed26fb0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4750603e9555454695ecc1f56e9185ca', 'request_id': 'req-5c93f4f2-e53c-4382-a9cb-cfe64ed26fb0'} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 11:06:02.789151 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-5c93f4f2-e53c-4382-a9cb-cfe64ed26fb0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4750603e9555454695ecc1f56e9185ca', 'request_id': 'req-5c93f4f2-e53c-4382-a9cb-cfe64ed26fb0'} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 11:06:02.800761 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-5c93f4f2-e53c-4382-a9cb-cfe64ed26fb0 admin admin] The token cache is being invalidate because project 4750603e9555454695ecc1f56e9185ca was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80143) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 07 11:06:02.801050 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-5c93f4f2-e53c-4382-a9cb-cfe64ed26fb0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 11:06:02.801672 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-5c93f4f2-e53c-4382-a9cb-cfe64ed26fb0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 11:06:02.803156 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1365/2757] 10.222.0.25 () {58 vars in 1233 bytes} [Sun Aug 7 11:06:02 2022] DELETE /identity/v3/projects/4750603e9555454695ecc1f56e9185ca => generated 0 bytes in 153 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 07 11:06:02.807108 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-496de77d-48ce-4b36-adbc-a139903100ec admin admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 11:06:02.808652 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.fernet_utils [None req-496de77d-48ce-4b36-adbc-a139903100ec admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80144) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 11:06:02.821502 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-496de77d-48ce-4b36-adbc-a139903100ec admin admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 11:06:02.877424 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb4e20e1-e782-4d87-a6ce-efa1c8e08d75 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5ac47e3b09f43d19a23a0f457fe9e23', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 11:06:02.878420 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb4e20e1-e782-4d87-a6ce-efa1c8e08d75 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 11:06:02.878725 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb4e20e1-e782-4d87-a6ce-efa1c8e08d75 admin admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 11:06:02.879035 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb4e20e1-e782-4d87-a6ce-efa1c8e08d75 admin admin] PATH_INFO: `/v3/users/b007599b6be249eb84eca414d0c51b59` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 11:06:02.880160 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb4e20e1-e782-4d87-a6ce-efa1c8e08d75 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b007599b6be249eb84eca414d0c51b59)` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 11:06:02.882060 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb4e20e1-e782-4d87-a6ce-efa1c8e08d75 admin admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 11:06:02.928799 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.notifications [None req-eb4e20e1-e782-4d87-a6ce-efa1c8e08d75 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b007599b6be249eb84eca414d0c51b59'} {{(pid=80144) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 11:06:02.935507 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.notifications [None req-eb4e20e1-e782-4d87-a6ce-efa1c8e08d75 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b007599b6be249eb84eca414d0c51b59'} {{(pid=80144) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 11:06:02.943191 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.notifications [None req-eb4e20e1-e782-4d87-a6ce-efa1c8e08d75 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b007599b6be249eb84eca414d0c51b59'} {{(pid=80144) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 11:06:02.943875 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.notifications [None req-eb4e20e1-e782-4d87-a6ce-efa1c8e08d75 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b007599b6be249eb84eca414d0c51b59'} {{(pid=80144) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 11:06:02.944494 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.notifications [None req-eb4e20e1-e782-4d87-a6ce-efa1c8e08d75 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b007599b6be249eb84eca414d0c51b59'} {{(pid=80144) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 11:06:02.962258 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 1393/2758] 10.222.0.25 () {58 vars in 1224 bytes} [Sun Aug 7 11:06:02 2022] DELETE /identity/v3/users/b007599b6be249eb84eca414d0c51b59 => generated 0 bytes in 156 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 07 11:06:03.235063 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c93f4f2-e53c-4382-a9cb-cfe64ed26fb0 admin admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 11:06:03.236492 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-5c93f4f2-e53c-4382-a9cb-cfe64ed26fb0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 11:06:03.249423 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c93f4f2-e53c-4382-a9cb-cfe64ed26fb0 admin admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 11:06:03.313831 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-723a2093-b450-49bb-83ad-3b22ddac43a8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5ac47e3b09f43d19a23a0f457fe9e23', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 11:06:03.314491 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-723a2093-b450-49bb-83ad-3b22ddac43a8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 11:06:03.314769 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-723a2093-b450-49bb-83ad-3b22ddac43a8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 11:06:03.315071 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-723a2093-b450-49bb-83ad-3b22ddac43a8 admin admin] PATH_INFO: `/v3/projects/14a01bbaecbc4cca972eaf82fdd0460d` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 11:06:03.317057 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-723a2093-b450-49bb-83ad-3b22ddac43a8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=14a01bbaecbc4cca972eaf82fdd0460d)` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 11:06:03.318991 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-723a2093-b450-49bb-83ad-3b22ddac43a8 admin admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 11:06:03.341251 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=90,keystone:INSERT=2,keystone:DELETE=8 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 11:06:03.364985 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-723a2093-b450-49bb-83ad-3b22ddac43a8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '14a01bbaecbc4cca972eaf82fdd0460d', 'request_id': 'req-723a2093-b450-49bb-83ad-3b22ddac43a8'} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 11:06:03.370286 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-723a2093-b450-49bb-83ad-3b22ddac43a8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '14a01bbaecbc4cca972eaf82fdd0460d', 'request_id': 'req-723a2093-b450-49bb-83ad-3b22ddac43a8'} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 11:06:03.376235 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-723a2093-b450-49bb-83ad-3b22ddac43a8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '14a01bbaecbc4cca972eaf82fdd0460d', 'request_id': 'req-723a2093-b450-49bb-83ad-3b22ddac43a8'} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 11:06:03.387622 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-723a2093-b450-49bb-83ad-3b22ddac43a8 admin admin] The token cache is being invalidate because project 14a01bbaecbc4cca972eaf82fdd0460d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80143) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 07 11:06:03.387910 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-723a2093-b450-49bb-83ad-3b22ddac43a8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 11:06:03.388531 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-723a2093-b450-49bb-83ad-3b22ddac43a8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 11:06:03.389986 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1366/2759] 10.222.0.25 () {58 vars in 1233 bytes} [Sun Aug 7 11:06:03 2022] DELETE /identity/v3/projects/14a01bbaecbc4cca972eaf82fdd0460d => generated 0 bytes in 156 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 07 11:06:12.342003 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cab6cee-2405-4f47-90d6-545a12133c66 None None] REQUEST_METHOD: `POST` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 11:06:12.342524 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cab6cee-2405-4f47-90d6-545a12133c66 None None] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 11:06:12.342524 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cab6cee-2405-4f47-90d6-545a12133c66 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 11:06:12.362006 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.auth.core [None req-0cab6cee-2405-4f47-90d6-545a12133c66 None None] MFA Rules not processed for user `c8d7c5d4a1634a71bad9102577bef117`. Rule list: `[]` (Enabled: `True`). {{(pid=80144) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 11:06:12.363216 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.fernet_utils [None req-0cab6cee-2405-4f47-90d6-545a12133c66 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80144) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 11:06:12.409123 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=85,keystone:INSERT=2,keystone:DELETE=13 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 11:06:12.422765 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 1394/2760] 10.222.0.25 () {58 vars in 958 bytes} [Sun Aug 7 11:06:12 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 07 11:06:13.384759 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:DELETE=7,keystone:SELECT=15,keystone:INSERT=1 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 11:06:22.420032 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=3 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 11:06:34.496234 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0abc7855-7523-49d9-88b8-315d8b33dde7 None None] REQUEST_METHOD: `POST` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 11:06:34.496693 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0abc7855-7523-49d9-88b8-315d8b33dde7 None None] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 11:06:34.496787 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0abc7855-7523-49d9-88b8-315d8b33dde7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 11:06:34.546677 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.auth.core [None req-0abc7855-7523-49d9-88b8-315d8b33dde7 None None] MFA Rules not processed for user `de1d6c24b1a14d10a20d82a4c65793d8`. Rule list: `[]` (Enabled: `True`). {{(pid=80143) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 11:06:34.547939 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-0abc7855-7523-49d9-88b8-315d8b33dde7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 11:06:34.611431 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1367/2761] 10.222.0.25 () {58 vars in 1009 bytes} [Sun Aug 7 11:06:34 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 07 11:06:34.656538 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cab6cee-2405-4f47-90d6-545a12133c66 None None] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 11:06:34.657940 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.fernet_utils [None req-0cab6cee-2405-4f47-90d6-545a12133c66 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80144) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 11:06:34.759725 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cab6cee-2405-4f47-90d6-545a12133c66 None None] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 11:06:34.764781 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e8497d6-4114-4118-be59-741b53eec006 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69490ea2a9a340bf91aca38bf5655b45', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6de1f3cac02c4bcdb05dd2337176033d', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 11:06:34.765387 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e8497d6-4114-4118-be59-741b53eec006 service swift] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 11:06:34.765672 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e8497d6-4114-4118-be59-741b53eec006 service swift] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 11:06:34.765948 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e8497d6-4114-4118-be59-741b53eec006 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 11:06:34.774588 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e8497d6-4114-4118-be59-741b53eec006 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 11:06:34.776194 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e8497d6-4114-4118-be59-741b53eec006 service swift] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 11:06:34.784221 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 1395/2762] 10.222.0.25 () {60 vars in 1384 bytes} [Sun Aug 7 11:06:34 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 128 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 07 11:06:44.608269 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=40 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 11:06:44.793926 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=41 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 11:06:50.414441 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5486b212-45ce-4763-bae5-7305abb624b8 None None] REQUEST_METHOD: `POST` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 11:06:50.414903 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5486b212-45ce-4763-bae5-7305abb624b8 None None] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 11:06:50.415040 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5486b212-45ce-4763-bae5-7305abb624b8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 11:06:50.434695 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.auth.core [None req-5486b212-45ce-4763-bae5-7305abb624b8 None None] MFA Rules not processed for user `de1d6c24b1a14d10a20d82a4c65793d8`. Rule list: `[]` (Enabled: `True`). {{(pid=80143) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 11:06:50.435931 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-5486b212-45ce-4763-bae5-7305abb624b8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 11:06:50.444619 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1368/2763] 10.222.0.25 () {58 vars in 967 bytes} [Sun Aug 7 11:06:50 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 07 11:06:50.456546 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e8497d6-4114-4118-be59-741b53eec006 service swift] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 11:06:50.467087 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e8497d6-4114-4118-be59-741b53eec006 service swift] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 11:06:50.472069 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1feaf0c-7130-4efa-b966-454314d6021d service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '69490ea2a9a340bf91aca38bf5655b45', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6de1f3cac02c4bcdb05dd2337176033d', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 11:06:50.472645 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1feaf0c-7130-4efa-b966-454314d6021d service swift] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 11:06:50.472929 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1feaf0c-7130-4efa-b966-454314d6021d service swift] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 11:06:50.473729 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1feaf0c-7130-4efa-b966-454314d6021d service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 11:06:50.481745 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1feaf0c-7130-4efa-b966-454314d6021d service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 11:06:50.483343 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1feaf0c-7130-4efa-b966-454314d6021d service swift] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 11:06:50.492296 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 1396/2764] 10.222.0.25 () {60 vars in 1384 bytes} [Sun Aug 7 11:06:50 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 07 11:06:51.367482 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5486b212-45ce-4763-bae5-7305abb624b8 None None] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 11:06:51.368881 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-5486b212-45ce-4763-bae5-7305abb624b8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 11:06:51.424690 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5486b212-45ce-4763-bae5-7305abb624b8 None None] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 11:06:51.428375 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55853380-f8c8-4608-99ed-ab299c86239d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88dbd6dfd1164994a3e16da553561910', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6de1f3cac02c4bcdb05dd2337176033d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 11:06:51.428975 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55853380-f8c8-4608-99ed-ab299c86239d service cinder] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 11:06:51.429255 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55853380-f8c8-4608-99ed-ab299c86239d service cinder] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 11:06:51.429551 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55853380-f8c8-4608-99ed-ab299c86239d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 11:06:51.431310 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-55853380-f8c8-4608-99ed-ab299c86239d service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 11:06:51.510482 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55853380-f8c8-4608-99ed-ab299c86239d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 11:06:51.511932 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55853380-f8c8-4608-99ed-ab299c86239d service cinder] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 11:06:51.530372 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1369/2765] 10.222.0.25 () {60 vars in 1365 bytes} [Sun Aug 7 11:06:51 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 163 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 07 11:06:52.346518 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1feaf0c-7130-4efa-b966-454314d6021d service swift] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 11:06:52.348041 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.fernet_utils [None req-f1feaf0c-7130-4efa-b966-454314d6021d service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80144) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 11:06:52.401187 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55853380-f8c8-4608-99ed-ab299c86239d service cinder] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 11:06:52.402680 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-55853380-f8c8-4608-99ed-ab299c86239d service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 11:06:52.433012 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1feaf0c-7130-4efa-b966-454314d6021d service swift] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 11:06:52.438226 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7534010-680e-45c7-a17b-ef837552f402 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6ee2e4c08af04c219ed395d7ca7085a6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6de1f3cac02c4bcdb05dd2337176033d', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 11:06:52.438785 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7534010-680e-45c7-a17b-ef837552f402 service placement] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 11:06:52.439097 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7534010-680e-45c7-a17b-ef837552f402 service placement] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 11:06:52.439372 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7534010-680e-45c7-a17b-ef837552f402 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 11:06:52.441167 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.fernet_utils [None req-d7534010-680e-45c7-a17b-ef837552f402 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80144) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 11:06:52.453007 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7534010-680e-45c7-a17b-ef837552f402 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 11:06:52.454570 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7534010-680e-45c7-a17b-ef837552f402 service placement] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 11:06:52.465110 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 1397/2766] 10.222.0.25 () {60 vars in 1365 bytes} [Sun Aug 7 11:06:52 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 119 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 07 11:06:52.476449 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55853380-f8c8-4608-99ed-ab299c86239d service cinder] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 11:06:52.481096 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e9ed0b0-4e50-42a2-a752-2970766af0cc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5ac47e3b09f43d19a23a0f457fe9e23', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 11:06:52.481728 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e9ed0b0-4e50-42a2-a752-2970766af0cc admin admin] REQUEST_METHOD: `DELETE` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 11:06:52.482015 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e9ed0b0-4e50-42a2-a752-2970766af0cc admin admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 11:06:52.482293 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e9ed0b0-4e50-42a2-a752-2970766af0cc admin admin] PATH_INFO: `/v3/users/c8d7c5d4a1634a71bad9102577bef117` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 11:06:52.483485 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e9ed0b0-4e50-42a2-a752-2970766af0cc admin admin] RBAC: Authorizing `identity:delete_user(user_id=c8d7c5d4a1634a71bad9102577bef117)` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 11:06:52.485655 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e9ed0b0-4e50-42a2-a752-2970766af0cc admin admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 11:06:52.508155 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=97,keystone:DELETE=3 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 11:06:52.533624 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-3e9ed0b0-4e50-42a2-a752-2970766af0cc admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c8d7c5d4a1634a71bad9102577bef117'} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 11:06:52.541253 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-3e9ed0b0-4e50-42a2-a752-2970766af0cc admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c8d7c5d4a1634a71bad9102577bef117'} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 11:06:52.553064 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-3e9ed0b0-4e50-42a2-a752-2970766af0cc admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c8d7c5d4a1634a71bad9102577bef117'} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 11:06:52.553746 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-3e9ed0b0-4e50-42a2-a752-2970766af0cc admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c8d7c5d4a1634a71bad9102577bef117'} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 11:06:52.561098 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-3e9ed0b0-4e50-42a2-a752-2970766af0cc admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c8d7c5d4a1634a71bad9102577bef117'} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 11:06:52.566999 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1370/2767] 10.222.0.25 () {58 vars in 1224 bytes} [Sun Aug 7 11:06:52 2022] DELETE /identity/v3/users/c8d7c5d4a1634a71bad9102577bef117 => generated 0 bytes in 166 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 07 11:06:52.740943 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7534010-680e-45c7-a17b-ef837552f402 service placement] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 11:06:52.751753 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7534010-680e-45c7-a17b-ef837552f402 service placement] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 11:06:52.812968 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94e01486-2eef-4776-80af-4bfbc506030e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5ac47e3b09f43d19a23a0f457fe9e23', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 11:06:52.813608 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94e01486-2eef-4776-80af-4bfbc506030e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 11:06:52.813896 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94e01486-2eef-4776-80af-4bfbc506030e admin admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 11:06:52.814176 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94e01486-2eef-4776-80af-4bfbc506030e admin admin] PATH_INFO: `/v3/projects/4867894c8e5840c198d233005521fd99` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 11:06:52.815371 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94e01486-2eef-4776-80af-4bfbc506030e admin admin] RBAC: Authorizing `identity:delete_project(project_id=4867894c8e5840c198d233005521fd99)` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 11:06:52.817820 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94e01486-2eef-4776-80af-4bfbc506030e admin admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 11:06:52.870598 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.notifications [None req-94e01486-2eef-4776-80af-4bfbc506030e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4867894c8e5840c198d233005521fd99', 'request_id': 'req-94e01486-2eef-4776-80af-4bfbc506030e'} {{(pid=80144) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 11:06:52.876216 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.notifications [None req-94e01486-2eef-4776-80af-4bfbc506030e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4867894c8e5840c198d233005521fd99', 'request_id': 'req-94e01486-2eef-4776-80af-4bfbc506030e'} {{(pid=80144) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 11:06:52.884363 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.notifications [None req-94e01486-2eef-4776-80af-4bfbc506030e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4867894c8e5840c198d233005521fd99', 'request_id': 'req-94e01486-2eef-4776-80af-4bfbc506030e'} {{(pid=80144) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 11:06:52.897822 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.notifications [None req-94e01486-2eef-4776-80af-4bfbc506030e admin admin] The token cache is being invalidate because project 4867894c8e5840c198d233005521fd99 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80144) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 07 11:06:52.897822 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.notifications [None req-94e01486-2eef-4776-80af-4bfbc506030e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80144) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 11:06:52.897822 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.notifications [None req-94e01486-2eef-4776-80af-4bfbc506030e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80144) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 11:06:52.899707 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 1398/2768] 10.222.0.25 () {58 vars in 1233 bytes} [Sun Aug 7 11:06:52 2022] DELETE /identity/v3/projects/4867894c8e5840c198d233005521fd99 => generated 0 bytes in 159 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 07 11:06:52.903865 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94e01486-2eef-4776-80af-4bfbc506030e admin admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 11:06:52.904010 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-465e01ee-8319-466c-868f-71dd6f2601a1 None None] REQUEST_METHOD: `POST` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 11:06:52.904285 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-465e01ee-8319-466c-868f-71dd6f2601a1 None None] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 11:06:52.904562 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-465e01ee-8319-466c-868f-71dd6f2601a1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 11:06:52.905305 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.fernet_utils [None req-94e01486-2eef-4776-80af-4bfbc506030e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80144) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 11:06:52.917104 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 11:06:52.924036 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.auth.core [None req-465e01ee-8319-466c-868f-71dd6f2601a1 None None] MFA Rules not processed for user `bac4e6b76fe64217ac8fa44e71cffe6a`. Rule list: `[]` (Enabled: `True`). {{(pid=80143) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 11:06:52.925357 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-465e01ee-8319-466c-868f-71dd6f2601a1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 11:06:52.973764 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94e01486-2eef-4776-80af-4bfbc506030e admin admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 11:06:52.977514 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-127dd7e4-4517-44c5-9c7a-449a8457daa5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be1283c9bf47424c9aa3ad27cab03027', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6de1f3cac02c4bcdb05dd2337176033d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 11:06:52.978152 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-127dd7e4-4517-44c5-9c7a-449a8457daa5 service neutron] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 11:06:52.978438 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-127dd7e4-4517-44c5-9c7a-449a8457daa5 service neutron] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 11:06:52.978714 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-127dd7e4-4517-44c5-9c7a-449a8457daa5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 11:06:52.980544 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.fernet_utils [None req-127dd7e4-4517-44c5-9c7a-449a8457daa5 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80144) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 11:06:52.987464 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1371/2769] 10.222.0.25 () {58 vars in 958 bytes} [Sun Aug 7 11:06:52 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 07 11:06:52.993215 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-127dd7e4-4517-44c5-9c7a-449a8457daa5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 11:06:52.994744 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-127dd7e4-4517-44c5-9c7a-449a8457daa5 service neutron] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 11:06:52.996104 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adb03391-e818-431e-9fa2-ebd636b66be3 None None] REQUEST_METHOD: `POST` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 11:06:52.996454 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adb03391-e818-431e-9fa2-ebd636b66be3 None None] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 11:06:52.996792 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adb03391-e818-431e-9fa2-ebd636b66be3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 11:06:53.005296 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 1399/2770] 10.222.0.25 () {60 vars in 1365 bytes} [Sun Aug 7 11:06:52 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 102 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 07 11:06:53.017553 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.auth.core [None req-adb03391-e818-431e-9fa2-ebd636b66be3 None None] MFA Rules not processed for user `bac4e6b76fe64217ac8fa44e71cffe6a`. Rule list: `[]` (Enabled: `True`). {{(pid=80143) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 11:06:53.018731 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-adb03391-e818-431e-9fa2-ebd636b66be3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 11:06:53.027136 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1372/2771] 10.222.0.25 () {58 vars in 958 bytes} [Sun Aug 7 11:06:52 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 07 11:06:53.031429 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-127dd7e4-4517-44c5-9c7a-449a8457daa5 service neutron] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 11:06:53.041256 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-127dd7e4-4517-44c5-9c7a-449a8457daa5 service neutron] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 11:06:53.045905 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-639e7bdc-21f4-4fe0-b5bd-51741d543416 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5ac47e3b09f43d19a23a0f457fe9e23', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 11:06:53.046540 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-639e7bdc-21f4-4fe0-b5bd-51741d543416 admin admin] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 11:06:53.046824 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-639e7bdc-21f4-4fe0-b5bd-51741d543416 admin admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 11:06:53.047145 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-639e7bdc-21f4-4fe0-b5bd-51741d543416 admin admin] PATH_INFO: `/v3/domains` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 11:06:53.047797 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-639e7bdc-21f4-4fe0-b5bd-51741d543416 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80144) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 11:06:53.048111 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-639e7bdc-21f4-4fe0-b5bd-51741d543416 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 11:06:53.050197 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-639e7bdc-21f4-4fe0-b5bd-51741d543416 admin admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 11:06:53.061294 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 1400/2772] 10.222.0.25 () {58 vars in 1153 bytes} [Sun Aug 7 11:06:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 11:06:53.065581 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adb03391-e818-431e-9fa2-ebd636b66be3 None None] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 11:06:53.075685 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adb03391-e818-431e-9fa2-ebd636b66be3 None None] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 11:06:53.080464 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8e9855a-a822-4030-94e2-0b8a9dc8ccaa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5ac47e3b09f43d19a23a0f457fe9e23', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 11:06:53.081060 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8e9855a-a822-4030-94e2-0b8a9dc8ccaa admin admin] REQUEST_METHOD: `POST` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 11:06:53.081330 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8e9855a-a822-4030-94e2-0b8a9dc8ccaa admin admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 11:06:53.081596 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8e9855a-a822-4030-94e2-0b8a9dc8ccaa admin admin] PATH_INFO: `/v3/projects` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 11:06:53.082208 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8e9855a-a822-4030-94e2-0b8a9dc8ccaa admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 11:06:53.084882 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8e9855a-a822-4030-94e2-0b8a9dc8ccaa admin admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 11:06:53.112345 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1373/2773] 10.222.0.25 () {60 vars in 1153 bytes} [Sun Aug 7 11:06:53 2022] POST /identity/v3/projects => generated 375 bytes in 47 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 07 11:06:53.116868 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-639e7bdc-21f4-4fe0-b5bd-51741d543416 admin admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 11:06:53.127023 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-639e7bdc-21f4-4fe0-b5bd-51741d543416 admin admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 11:06:53.186702 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cc8cb09-7aa5-4a7d-98a5-09ed5e195e61 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5ac47e3b09f43d19a23a0f457fe9e23', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 11:06:53.187343 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cc8cb09-7aa5-4a7d-98a5-09ed5e195e61 admin admin] REQUEST_METHOD: `POST` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 11:06:53.187626 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cc8cb09-7aa5-4a7d-98a5-09ed5e195e61 admin admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 11:06:53.187922 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cc8cb09-7aa5-4a7d-98a5-09ed5e195e61 admin admin] PATH_INFO: `/v3/users` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 11:06:53.188578 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cc8cb09-7aa5-4a7d-98a5-09ed5e195e61 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 11:06:53.190452 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cc8cb09-7aa5-4a7d-98a5-09ed5e195e61 admin admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 11:06:53.216455 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 1401/2774] 10.222.0.25 () {60 vars in 1144 bytes} [Sun Aug 7 11:06:53 2022] POST /identity/v3/users => generated 288 bytes in 97 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 07 11:06:53.218040 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8e9855a-a822-4030-94e2-0b8a9dc8ccaa admin admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 11:06:53.229657 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8e9855a-a822-4030-94e2-0b8a9dc8ccaa admin admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 11:06:53.234816 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bd39f34-dc39-422b-995a-59974cc65f04 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5ac47e3b09f43d19a23a0f457fe9e23', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 11:06:53.235469 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bd39f34-dc39-422b-995a-59974cc65f04 admin admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 11:06:53.235746 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bd39f34-dc39-422b-995a-59974cc65f04 admin admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 11:06:53.236022 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bd39f34-dc39-422b-995a-59974cc65f04 admin admin] PATH_INFO: `/v3/roles` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 11:06:53.236663 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bd39f34-dc39-422b-995a-59974cc65f04 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 11:06:53.238310 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bd39f34-dc39-422b-995a-59974cc65f04 admin admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 11:06:53.250071 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1374/2775] 10.222.0.25 () {58 vars in 1122 bytes} [Sun Aug 7 11:06:53 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 07 11:06:53.254262 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cc8cb09-7aa5-4a7d-98a5-09ed5e195e61 admin admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 11:06:53.265397 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cc8cb09-7aa5-4a7d-98a5-09ed5e195e61 admin admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 11:06:53.270085 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc51d505-934e-4bba-9dd9-91bf92121954 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5ac47e3b09f43d19a23a0f457fe9e23', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 11:06:53.270916 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc51d505-934e-4bba-9dd9-91bf92121954 admin admin] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 11:06:53.271231 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc51d505-934e-4bba-9dd9-91bf92121954 admin admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 11:06:53.271503 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc51d505-934e-4bba-9dd9-91bf92121954 admin admin] PATH_INFO: `/v3/roles` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 11:06:53.272149 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc51d505-934e-4bba-9dd9-91bf92121954 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 11:06:53.273744 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc51d505-934e-4bba-9dd9-91bf92121954 admin admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 11:06:53.286113 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 1402/2776] 10.222.0.25 () {58 vars in 1122 bytes} [Sun Aug 7 11:06:53 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 07 11:06:53.290566 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bd39f34-dc39-422b-995a-59974cc65f04 admin admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 11:06:53.300823 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bd39f34-dc39-422b-995a-59974cc65f04 admin admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 11:06:53.306265 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7db3917-b9bf-4ab1-a1ea-b3f2af95adb6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5ac47e3b09f43d19a23a0f457fe9e23', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 11:06:53.306948 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7db3917-b9bf-4ab1-a1ea-b3f2af95adb6 admin admin] REQUEST_METHOD: `PUT` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 11:06:53.307239 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7db3917-b9bf-4ab1-a1ea-b3f2af95adb6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 11:06:53.307509 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7db3917-b9bf-4ab1-a1ea-b3f2af95adb6 admin admin] PATH_INFO: `/v3/projects/b2ccc5524e134ceea843b70ac49979b1/users/9b12d779775546f392f64db839173141/roles/b98dfb3b9fee4644af2887a3ac3fd2e6` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 11:06:53.323723 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7db3917-b9bf-4ab1-a1ea-b3f2af95adb6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b2ccc5524e134ceea843b70ac49979b1, user_id=9b12d779775546f392f64db839173141, role_id=b98dfb3b9fee4644af2887a3ac3fd2e6)` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 11:06:53.326354 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7db3917-b9bf-4ab1-a1ea-b3f2af95adb6 admin admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 11:06:53.340801 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1375/2777] 10.222.0.25 () {60 vars in 1483 bytes} [Sun Aug 7 11:06:53 2022] PUT /identity/v3/projects/b2ccc5524e134ceea843b70ac49979b1/users/9b12d779775546f392f64db839173141/roles/b98dfb3b9fee4644af2887a3ac3fd2e6 => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 07 11:06:53.351629 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fa3b375-5866-4439-91b1-b23414deaf96 None None] REQUEST_METHOD: `POST` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 11:06:53.352205 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fa3b375-5866-4439-91b1-b23414deaf96 None None] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 11:06:53.352205 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fa3b375-5866-4439-91b1-b23414deaf96 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 11:06:53.382632 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.auth.core [None req-3fa3b375-5866-4439-91b1-b23414deaf96 None None] MFA Rules not processed for user `9b12d779775546f392f64db839173141`. Rule list: `[]` (Enabled: `True`). {{(pid=80144) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 11:06:53.383896 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.fernet_utils [None req-3fa3b375-5866-4439-91b1-b23414deaf96 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80144) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 11:06:53.458267 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 1403/2778] 10.222.0.25 () {58 vars in 958 bytes} [Sun Aug 7 11:06:53 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 07 11:06:53.469407 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7db3917-b9bf-4ab1-a1ea-b3f2af95adb6 admin admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 11:06:53.471134 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-c7db3917-b9bf-4ab1-a1ea-b3f2af95adb6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 11:06:53.513038 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:DELETE=7,keystone:SELECT=90,keystone:INSERT=3 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 11:06:53.533764 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7db3917-b9bf-4ab1-a1ea-b3f2af95adb6 admin admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 11:06:53.538007 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4b46ebb-a4e3-4d94-84d0-6750fa0413ac service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88dbd6dfd1164994a3e16da553561910', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6de1f3cac02c4bcdb05dd2337176033d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 11:06:53.538610 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4b46ebb-a4e3-4d94-84d0-6750fa0413ac service cinder] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 11:06:53.538893 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4b46ebb-a4e3-4d94-84d0-6750fa0413ac service cinder] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 11:06:53.539273 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4b46ebb-a4e3-4d94-84d0-6750fa0413ac service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 11:06:53.547861 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4b46ebb-a4e3-4d94-84d0-6750fa0413ac service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 11:06:53.549274 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4b46ebb-a4e3-4d94-84d0-6750fa0413ac service cinder] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 11:06:53.558579 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1376/2779] 10.222.0.25 () {60 vars in 1365 bytes} [Sun Aug 7 11:06:53 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 07 11:06:56.253970 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fa3b375-5866-4439-91b1-b23414deaf96 None None] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 11:06:56.255541 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.fernet_utils [None req-3fa3b375-5866-4439-91b1-b23414deaf96 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80144) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 11:06:56.263868 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 11:06:56.341807 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fa3b375-5866-4439-91b1-b23414deaf96 None None] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 11:06:56.346312 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5e32339-b009-4483-bab4-dca3cffa6077 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5ac47e3b09f43d19a23a0f457fe9e23', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 11:06:56.346967 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5e32339-b009-4483-bab4-dca3cffa6077 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 11:06:56.347250 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5e32339-b009-4483-bab4-dca3cffa6077 admin admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 11:06:56.347523 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5e32339-b009-4483-bab4-dca3cffa6077 admin admin] PATH_INFO: `/v3/users/06231f178e0d494084915b9368fa3b9b` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 11:06:56.349256 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5e32339-b009-4483-bab4-dca3cffa6077 admin admin] RBAC: Authorizing `identity:delete_user(user_id=06231f178e0d494084915b9368fa3b9b)` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 11:06:56.351348 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5e32339-b009-4483-bab4-dca3cffa6077 admin admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 11:06:56.404751 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.notifications [None req-c5e32339-b009-4483-bab4-dca3cffa6077 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '06231f178e0d494084915b9368fa3b9b'} {{(pid=80144) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 11:06:56.412401 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.notifications [None req-c5e32339-b009-4483-bab4-dca3cffa6077 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '06231f178e0d494084915b9368fa3b9b'} {{(pid=80144) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 11:06:56.425462 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.notifications [None req-c5e32339-b009-4483-bab4-dca3cffa6077 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '06231f178e0d494084915b9368fa3b9b'} {{(pid=80144) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 11:06:56.426126 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.notifications [None req-c5e32339-b009-4483-bab4-dca3cffa6077 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '06231f178e0d494084915b9368fa3b9b'} {{(pid=80144) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 11:06:56.426788 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.notifications [None req-c5e32339-b009-4483-bab4-dca3cffa6077 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '06231f178e0d494084915b9368fa3b9b'} {{(pid=80144) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 11:06:56.445633 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 1404/2780] 10.222.0.25 () {58 vars in 1224 bytes} [Sun Aug 7 11:06:56 2022] DELETE /identity/v3/users/06231f178e0d494084915b9368fa3b9b => generated 0 bytes in 192 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 07 11:06:56.622313 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4b46ebb-a4e3-4d94-84d0-6750fa0413ac service cinder] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 11:06:56.635174 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4b46ebb-a4e3-4d94-84d0-6750fa0413ac service cinder] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 11:06:56.695319 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd257b34-a62f-4239-a62c-e14abdd52bfb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5ac47e3b09f43d19a23a0f457fe9e23', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 11:06:56.695960 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd257b34-a62f-4239-a62c-e14abdd52bfb admin admin] REQUEST_METHOD: `DELETE` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 11:06:56.696237 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd257b34-a62f-4239-a62c-e14abdd52bfb admin admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 11:06:56.696508 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd257b34-a62f-4239-a62c-e14abdd52bfb admin admin] PATH_INFO: `/v3/projects/01328619d6b24d359c0352ef4119ea90` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 11:06:56.697654 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd257b34-a62f-4239-a62c-e14abdd52bfb admin admin] RBAC: Authorizing `identity:delete_project(project_id=01328619d6b24d359c0352ef4119ea90)` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 11:06:56.699765 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd257b34-a62f-4239-a62c-e14abdd52bfb admin admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 11:06:56.765438 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-dd257b34-a62f-4239-a62c-e14abdd52bfb admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '01328619d6b24d359c0352ef4119ea90', 'request_id': 'req-dd257b34-a62f-4239-a62c-e14abdd52bfb'} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 11:06:56.770887 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-dd257b34-a62f-4239-a62c-e14abdd52bfb admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '01328619d6b24d359c0352ef4119ea90', 'request_id': 'req-dd257b34-a62f-4239-a62c-e14abdd52bfb'} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 11:06:56.778304 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-dd257b34-a62f-4239-a62c-e14abdd52bfb admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '01328619d6b24d359c0352ef4119ea90', 'request_id': 'req-dd257b34-a62f-4239-a62c-e14abdd52bfb'} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 11:06:56.792079 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-dd257b34-a62f-4239-a62c-e14abdd52bfb admin admin] The token cache is being invalidate because project 01328619d6b24d359c0352ef4119ea90 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80143) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 07 11:06:56.792375 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-dd257b34-a62f-4239-a62c-e14abdd52bfb admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 11:06:56.792999 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-dd257b34-a62f-4239-a62c-e14abdd52bfb admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 11:06:56.794541 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1377/2781] 10.222.0.25 () {58 vars in 1233 bytes} [Sun Aug 7 11:06:56 2022] DELETE /identity/v3/projects/01328619d6b24d359c0352ef4119ea90 => generated 0 bytes in 173 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 07 11:06:56.799475 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa842dee-4901-4bb0-bb36-a33b86808446 None None] REQUEST_METHOD: `POST` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 11:06:56.799765 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa842dee-4901-4bb0-bb36-a33b86808446 None None] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 11:06:56.800044 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa842dee-4901-4bb0-bb36-a33b86808446 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 11:06:56.821438 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.auth.core [None req-aa842dee-4901-4bb0-bb36-a33b86808446 None None] MFA Rules not processed for user `bac4e6b76fe64217ac8fa44e71cffe6a`. Rule list: `[]` (Enabled: `True`). {{(pid=80144) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 11:06:56.822636 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.fernet_utils [None req-aa842dee-4901-4bb0-bb36-a33b86808446 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80144) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 11:06:56.881066 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 1405/2782] 10.222.0.25 () {58 vars in 958 bytes} [Sun Aug 7 11:06:56 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 07 11:06:56.888850 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1480f224-c62e-4d93-b9e2-d71bc9bbad48 None None] REQUEST_METHOD: `POST` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 11:06:56.889140 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1480f224-c62e-4d93-b9e2-d71bc9bbad48 None None] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 11:06:56.889416 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1480f224-c62e-4d93-b9e2-d71bc9bbad48 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 11:06:56.908907 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.auth.core [None req-1480f224-c62e-4d93-b9e2-d71bc9bbad48 None None] MFA Rules not processed for user `bac4e6b76fe64217ac8fa44e71cffe6a`. Rule list: `[]` (Enabled: `True`). {{(pid=80143) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 11:06:56.910096 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-1480f224-c62e-4d93-b9e2-d71bc9bbad48 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 11:06:56.918574 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1378/2783] 10.222.0.25 () {58 vars in 958 bytes} [Sun Aug 7 11:06:56 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 07 11:06:56.922716 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa842dee-4901-4bb0-bb36-a33b86808446 None None] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 11:06:56.932488 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa842dee-4901-4bb0-bb36-a33b86808446 None None] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 11:06:56.937120 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f321611-c6b3-4f93-bd86-572c4296b623 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5ac47e3b09f43d19a23a0f457fe9e23', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 11:06:56.937715 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f321611-c6b3-4f93-bd86-572c4296b623 admin admin] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 11:06:56.937997 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f321611-c6b3-4f93-bd86-572c4296b623 admin admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 11:06:56.938270 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f321611-c6b3-4f93-bd86-572c4296b623 admin admin] PATH_INFO: `/v3/domains` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 11:06:56.938907 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f321611-c6b3-4f93-bd86-572c4296b623 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80144) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 11:06:56.939243 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f321611-c6b3-4f93-bd86-572c4296b623 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 11:06:56.940808 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f321611-c6b3-4f93-bd86-572c4296b623 admin admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 11:06:56.952401 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 1406/2784] 10.222.0.25 () {58 vars in 1153 bytes} [Sun Aug 7 11:06:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 11:06:56.956561 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1480f224-c62e-4d93-b9e2-d71bc9bbad48 None None] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 11:06:56.966330 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1480f224-c62e-4d93-b9e2-d71bc9bbad48 None None] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 11:06:56.970903 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bee1756-acfb-4742-b743-edff825f11eb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5ac47e3b09f43d19a23a0f457fe9e23', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 11:06:56.971516 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bee1756-acfb-4742-b743-edff825f11eb admin admin] REQUEST_METHOD: `POST` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 11:06:56.971795 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bee1756-acfb-4742-b743-edff825f11eb admin admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 11:06:56.972073 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bee1756-acfb-4742-b743-edff825f11eb admin admin] PATH_INFO: `/v3/projects` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 11:06:56.972687 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bee1756-acfb-4742-b743-edff825f11eb admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 11:06:56.975281 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bee1756-acfb-4742-b743-edff825f11eb admin admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 11:06:57.001677 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1379/2785] 10.222.0.25 () {60 vars in 1153 bytes} [Sun Aug 7 11:06:56 2022] POST /identity/v3/projects => generated 367 bytes in 46 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 07 11:06:57.006434 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f321611-c6b3-4f93-bd86-572c4296b623 admin admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 11:06:57.016271 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f321611-c6b3-4f93-bd86-572c4296b623 admin admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 11:06:57.025082 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 11:06:57.074681 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ac46e50-2846-4566-a84c-62059613c7e1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5ac47e3b09f43d19a23a0f457fe9e23', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 11:06:57.075317 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ac46e50-2846-4566-a84c-62059613c7e1 admin admin] REQUEST_METHOD: `POST` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 11:06:57.075601 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ac46e50-2846-4566-a84c-62059613c7e1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 11:06:57.075877 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ac46e50-2846-4566-a84c-62059613c7e1 admin admin] PATH_INFO: `/v3/users` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 11:06:57.076496 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ac46e50-2846-4566-a84c-62059613c7e1 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 11:06:57.078398 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ac46e50-2846-4566-a84c-62059613c7e1 admin admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 11:06:57.100708 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 1407/2786] 10.222.0.25 () {60 vars in 1144 bytes} [Sun Aug 7 11:06:57 2022] POST /identity/v3/users => generated 284 bytes in 95 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 07 11:06:57.104747 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bee1756-acfb-4742-b743-edff825f11eb admin admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 11:06:57.115371 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bee1756-acfb-4742-b743-edff825f11eb admin admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 11:06:57.119888 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a4c5e80-9c30-4782-9e39-c33dfcb5d0ad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5ac47e3b09f43d19a23a0f457fe9e23', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 11:06:57.120488 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a4c5e80-9c30-4782-9e39-c33dfcb5d0ad admin admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 11:06:57.120768 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a4c5e80-9c30-4782-9e39-c33dfcb5d0ad admin admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 11:06:57.121042 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a4c5e80-9c30-4782-9e39-c33dfcb5d0ad admin admin] PATH_INFO: `/v3/roles` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 11:06:57.121676 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a4c5e80-9c30-4782-9e39-c33dfcb5d0ad admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 11:06:57.123292 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a4c5e80-9c30-4782-9e39-c33dfcb5d0ad admin admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 11:06:57.133199 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1380/2787] 10.222.0.25 () {58 vars in 1122 bytes} [Sun Aug 7 11:06:57 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 07 11:06:57.136922 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ac46e50-2846-4566-a84c-62059613c7e1 admin admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 11:06:57.147615 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ac46e50-2846-4566-a84c-62059613c7e1 admin admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 11:06:57.152186 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c455d2c-9374-45b8-99a4-2690ce5d9afc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5ac47e3b09f43d19a23a0f457fe9e23', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 11:06:57.152782 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c455d2c-9374-45b8-99a4-2690ce5d9afc admin admin] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 11:06:57.153063 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c455d2c-9374-45b8-99a4-2690ce5d9afc admin admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 11:06:57.153338 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c455d2c-9374-45b8-99a4-2690ce5d9afc admin admin] PATH_INFO: `/v3/roles` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 11:06:57.153971 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c455d2c-9374-45b8-99a4-2690ce5d9afc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 11:06:57.155573 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c455d2c-9374-45b8-99a4-2690ce5d9afc admin admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 11:06:57.167429 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 1408/2788] 10.222.0.25 () {58 vars in 1122 bytes} [Sun Aug 7 11:06:57 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 07 11:06:57.171481 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a4c5e80-9c30-4782-9e39-c33dfcb5d0ad admin admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 11:06:57.181287 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a4c5e80-9c30-4782-9e39-c33dfcb5d0ad admin admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 11:06:57.185815 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6ef72d4-a2ce-4b4c-a95f-b94f2db8b4dc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5ac47e3b09f43d19a23a0f457fe9e23', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 11:06:57.186457 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6ef72d4-a2ce-4b4c-a95f-b94f2db8b4dc admin admin] REQUEST_METHOD: `PUT` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 11:06:57.186737 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6ef72d4-a2ce-4b4c-a95f-b94f2db8b4dc admin admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 11:06:57.187055 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6ef72d4-a2ce-4b4c-a95f-b94f2db8b4dc admin admin] PATH_INFO: `/v3/projects/c7a01822626b4010aa9a21bed41ea4c3/users/a87816bf5e1c46f4b58a6c65fcc3e193/roles/b98dfb3b9fee4644af2887a3ac3fd2e6` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 11:06:57.200968 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6ef72d4-a2ce-4b4c-a95f-b94f2db8b4dc admin admin] RBAC: Authorizing `identity:create_grant(project_id=c7a01822626b4010aa9a21bed41ea4c3, user_id=a87816bf5e1c46f4b58a6c65fcc3e193, role_id=b98dfb3b9fee4644af2887a3ac3fd2e6)` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 11:06:57.203611 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6ef72d4-a2ce-4b4c-a95f-b94f2db8b4dc admin admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 11:06:57.217784 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1381/2789] 10.222.0.25 () {60 vars in 1483 bytes} [Sun Aug 7 11:06:57 2022] PUT /identity/v3/projects/c7a01822626b4010aa9a21bed41ea4c3/users/a87816bf5e1c46f4b58a6c65fcc3e193/roles/b98dfb3b9fee4644af2887a3ac3fd2e6 => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 07 11:06:57.230771 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba16ff82-7585-41b4-bc31-da0782049d78 None None] REQUEST_METHOD: `POST` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 11:06:57.231115 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba16ff82-7585-41b4-bc31-da0782049d78 None None] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 11:06:57.231380 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba16ff82-7585-41b4-bc31-da0782049d78 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 11:06:57.261579 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.auth.core [None req-ba16ff82-7585-41b4-bc31-da0782049d78 None None] MFA Rules not processed for user `a87816bf5e1c46f4b58a6c65fcc3e193`. Rule list: `[]` (Enabled: `True`). {{(pid=80144) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 11:06:57.262737 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.fernet_utils [None req-ba16ff82-7585-41b4-bc31-da0782049d78 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80144) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 11:06:57.325463 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 1409/2790] 10.222.0.25 () {58 vars in 958 bytes} [Sun Aug 7 11:06:57 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 07 11:06:57.330085 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6ef72d4-a2ce-4b4c-a95f-b94f2db8b4dc admin admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 11:06:57.336397 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 11:06:57.394934 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6ef72d4-a2ce-4b4c-a95f-b94f2db8b4dc admin admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 11:06:57.400474 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c53665e0-8da2-4703-b0bd-43cd2be4a208 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5ac47e3b09f43d19a23a0f457fe9e23', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 11:06:57.401079 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c53665e0-8da2-4703-b0bd-43cd2be4a208 admin admin] REQUEST_METHOD: `POST` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 11:06:57.401362 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c53665e0-8da2-4703-b0bd-43cd2be4a208 admin admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 11:06:57.401636 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c53665e0-8da2-4703-b0bd-43cd2be4a208 admin admin] PATH_INFO: `/v3/projects` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 11:06:57.402264 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c53665e0-8da2-4703-b0bd-43cd2be4a208 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 11:06:57.404193 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c53665e0-8da2-4703-b0bd-43cd2be4a208 admin admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 11:06:57.434356 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1382/2791] 10.222.0.25 () {60 vars in 1153 bytes} [Sun Aug 7 11:06:57 2022] POST /identity/v3/projects => generated 365 bytes in 101 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 07 11:06:57.435806 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba16ff82-7585-41b4-bc31-da0782049d78 None None] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 11:06:57.446943 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba16ff82-7585-41b4-bc31-da0782049d78 None None] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 11:06:57.505224 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cec441b5-3a70-4c6e-8b3b-4024788edd87 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5ac47e3b09f43d19a23a0f457fe9e23', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 11:06:57.505850 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cec441b5-3a70-4c6e-8b3b-4024788edd87 admin admin] REQUEST_METHOD: `POST` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 11:06:57.506134 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cec441b5-3a70-4c6e-8b3b-4024788edd87 admin admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 11:06:57.506410 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cec441b5-3a70-4c6e-8b3b-4024788edd87 admin admin] PATH_INFO: `/v3/users` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 11:06:57.507070 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cec441b5-3a70-4c6e-8b3b-4024788edd87 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 11:06:57.509007 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cec441b5-3a70-4c6e-8b3b-4024788edd87 admin admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 11:06:57.532670 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 1410/2792] 10.222.0.25 () {60 vars in 1144 bytes} [Sun Aug 7 11:06:57 2022] POST /identity/v3/users => generated 289 bytes in 97 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 07 11:06:57.536694 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c53665e0-8da2-4703-b0bd-43cd2be4a208 admin admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 11:06:57.547165 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c53665e0-8da2-4703-b0bd-43cd2be4a208 admin admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 11:06:57.552170 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d574ae07-9395-48cc-b666-5e4bbd48067d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5ac47e3b09f43d19a23a0f457fe9e23', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 11:06:57.552767 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d574ae07-9395-48cc-b666-5e4bbd48067d admin admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 11:06:57.553046 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d574ae07-9395-48cc-b666-5e4bbd48067d admin admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 11:06:57.553319 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d574ae07-9395-48cc-b666-5e4bbd48067d admin admin] PATH_INFO: `/v3/roles` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 11:06:57.553954 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d574ae07-9395-48cc-b666-5e4bbd48067d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 11:06:57.555552 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d574ae07-9395-48cc-b666-5e4bbd48067d admin admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 11:06:57.566014 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1383/2793] 10.222.0.25 () {58 vars in 1122 bytes} [Sun Aug 7 11:06:57 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 07 11:06:57.570859 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cec441b5-3a70-4c6e-8b3b-4024788edd87 admin admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 11:06:57.581942 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cec441b5-3a70-4c6e-8b3b-4024788edd87 admin admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 11:06:57.588078 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b2cd7ca-8c83-4767-94c0-9264423fb1db admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5ac47e3b09f43d19a23a0f457fe9e23', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 11:06:57.588746 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b2cd7ca-8c83-4767-94c0-9264423fb1db admin admin] REQUEST_METHOD: `PUT` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 11:06:57.589029 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b2cd7ca-8c83-4767-94c0-9264423fb1db admin admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 11:06:57.589308 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b2cd7ca-8c83-4767-94c0-9264423fb1db admin admin] PATH_INFO: `/v3/projects/747b7cbabb1d4cb7bec32d4f05432c63/users/1a14bd06bacc452e988c20befdf534c4/roles/81b1f1bcee2a473088fbbd327c1c5b23` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 11:06:57.592675 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 11:06:57.609805 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b2cd7ca-8c83-4767-94c0-9264423fb1db admin admin] RBAC: Authorizing `identity:create_grant(project_id=747b7cbabb1d4cb7bec32d4f05432c63, user_id=1a14bd06bacc452e988c20befdf534c4, role_id=81b1f1bcee2a473088fbbd327c1c5b23)` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 11:06:57.612526 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b2cd7ca-8c83-4767-94c0-9264423fb1db admin admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 11:06:57.625286 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 1411/2794] 10.222.0.25 () {60 vars in 1483 bytes} [Sun Aug 7 11:06:57 2022] PUT /identity/v3/projects/747b7cbabb1d4cb7bec32d4f05432c63/users/1a14bd06bacc452e988c20befdf534c4/roles/81b1f1bcee2a473088fbbd327c1c5b23 => generated 0 bytes in 55 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 07 11:06:57.629384 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d574ae07-9395-48cc-b666-5e4bbd48067d admin admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 11:06:57.639809 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d574ae07-9395-48cc-b666-5e4bbd48067d admin admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 11:06:57.694741 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4249e392-98d4-4e8a-b56d-c13a161e4673 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5ac47e3b09f43d19a23a0f457fe9e23', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 11:06:57.695605 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4249e392-98d4-4e8a-b56d-c13a161e4673 admin admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 11:06:57.695986 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4249e392-98d4-4e8a-b56d-c13a161e4673 admin admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 11:06:57.696350 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4249e392-98d4-4e8a-b56d-c13a161e4673 admin admin] PATH_INFO: `/v3/roles` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 11:06:57.697074 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4249e392-98d4-4e8a-b56d-c13a161e4673 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 11:06:57.698778 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4249e392-98d4-4e8a-b56d-c13a161e4673 admin admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 11:06:57.708400 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1384/2795] 10.222.0.25 () {58 vars in 1122 bytes} [Sun Aug 7 11:06:57 2022] GET /identity/v3/roles => generated 1420 bytes in 79 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 07 11:06:57.712832 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b2cd7ca-8c83-4767-94c0-9264423fb1db admin admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 11:06:57.722832 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b2cd7ca-8c83-4767-94c0-9264423fb1db admin admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 11:06:57.727612 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1299440d-5175-4bc1-9fde-a462b07daf14 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5ac47e3b09f43d19a23a0f457fe9e23', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 11:06:57.728249 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1299440d-5175-4bc1-9fde-a462b07daf14 admin admin] REQUEST_METHOD: `PUT` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 11:06:57.728526 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1299440d-5175-4bc1-9fde-a462b07daf14 admin admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 11:06:57.728797 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1299440d-5175-4bc1-9fde-a462b07daf14 admin admin] PATH_INFO: `/v3/projects/747b7cbabb1d4cb7bec32d4f05432c63/users/1a14bd06bacc452e988c20befdf534c4/roles/81b1f1bcee2a473088fbbd327c1c5b23` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 11:06:57.730667 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1299440d-5175-4bc1-9fde-a462b07daf14 admin admin] RBAC: Authorizing `identity:create_grant(project_id=747b7cbabb1d4cb7bec32d4f05432c63, user_id=1a14bd06bacc452e988c20befdf534c4, role_id=81b1f1bcee2a473088fbbd327c1c5b23)` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 11:06:57.733326 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1299440d-5175-4bc1-9fde-a462b07daf14 admin admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 11:06:57.744087 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 1412/2796] 10.222.0.25 () {60 vars in 1483 bytes} [Sun Aug 7 11:06:57 2022] PUT /identity/v3/projects/747b7cbabb1d4cb7bec32d4f05432c63/users/1a14bd06bacc452e988c20befdf534c4/roles/81b1f1bcee2a473088fbbd327c1c5b23 => generated 0 bytes in 32 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 07 11:06:57.754029 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f79cf5d2-af84-493b-b124-d0852e2b0954 None None] REQUEST_METHOD: `POST` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 11:06:57.754312 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f79cf5d2-af84-493b-b124-d0852e2b0954 None None] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 11:06:57.755320 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f79cf5d2-af84-493b-b124-d0852e2b0954 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 11:06:57.784600 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.auth.core [None req-f79cf5d2-af84-493b-b124-d0852e2b0954 None None] MFA Rules not processed for user `1a14bd06bacc452e988c20befdf534c4`. Rule list: `[]` (Enabled: `True`). {{(pid=80143) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 11:06:57.785874 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-f79cf5d2-af84-493b-b124-d0852e2b0954 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 11:06:57.859929 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 11:06:57.868050 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1385/2797] 10.222.0.25 () {58 vars in 958 bytes} [Sun Aug 7 11:06:57 2022] POST /identity/v3/auth/tokens => generated 3309 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 07 11:06:57.876178 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1299440d-5175-4bc1-9fde-a462b07daf14 admin admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 11:06:57.877963 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.fernet_utils [None req-1299440d-5175-4bc1-9fde-a462b07daf14 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80144) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 11:06:57.948103 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1299440d-5175-4bc1-9fde-a462b07daf14 admin admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 11:06:57.952955 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94f26c66-bddf-4fc3-bd69-7d7520ba3d8a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88dbd6dfd1164994a3e16da553561910', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6de1f3cac02c4bcdb05dd2337176033d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 11:06:57.953565 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94f26c66-bddf-4fc3-bd69-7d7520ba3d8a service cinder] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 11:06:57.953844 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94f26c66-bddf-4fc3-bd69-7d7520ba3d8a service cinder] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 11:06:57.954117 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94f26c66-bddf-4fc3-bd69-7d7520ba3d8a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 11:06:57.964417 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94f26c66-bddf-4fc3-bd69-7d7520ba3d8a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 11:06:57.965837 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94f26c66-bddf-4fc3-bd69-7d7520ba3d8a service cinder] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 11:06:57.976627 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 1413/2798] 10.222.0.25 () {60 vars in 1365 bytes} [Sun Aug 7 11:06:57 2022] GET /identity/v3/auth/tokens => generated 3309 bytes in 101 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 07 11:06:59.382277 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f79cf5d2-af84-493b-b124-d0852e2b0954 None None] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 11:06:59.443638 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f79cf5d2-af84-493b-b124-d0852e2b0954 None None] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 11:06:59.448192 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0515e7d-38d5-41bb-8050-a26ee4004a15 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5ac47e3b09f43d19a23a0f457fe9e23', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 11:06:59.448812 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0515e7d-38d5-41bb-8050-a26ee4004a15 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 11:06:59.449097 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0515e7d-38d5-41bb-8050-a26ee4004a15 admin admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 11:06:59.449407 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0515e7d-38d5-41bb-8050-a26ee4004a15 admin admin] PATH_INFO: `/v3/users/a87816bf5e1c46f4b58a6c65fcc3e193` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 11:06:59.450725 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0515e7d-38d5-41bb-8050-a26ee4004a15 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a87816bf5e1c46f4b58a6c65fcc3e193)` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 11:06:59.452671 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0515e7d-38d5-41bb-8050-a26ee4004a15 admin admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 11:06:59.502414 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-d0515e7d-38d5-41bb-8050-a26ee4004a15 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a87816bf5e1c46f4b58a6c65fcc3e193'} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 11:06:59.509808 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-d0515e7d-38d5-41bb-8050-a26ee4004a15 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a87816bf5e1c46f4b58a6c65fcc3e193'} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 11:06:59.521553 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-d0515e7d-38d5-41bb-8050-a26ee4004a15 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a87816bf5e1c46f4b58a6c65fcc3e193'} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 11:06:59.522319 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-d0515e7d-38d5-41bb-8050-a26ee4004a15 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a87816bf5e1c46f4b58a6c65fcc3e193'} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 11:06:59.530286 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-d0515e7d-38d5-41bb-8050-a26ee4004a15 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a87816bf5e1c46f4b58a6c65fcc3e193'} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 11:06:59.537259 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1386/2799] 10.222.0.25 () {58 vars in 1224 bytes} [Sun Aug 7 11:06:59 2022] DELETE /identity/v3/users/a87816bf5e1c46f4b58a6c65fcc3e193 => generated 0 bytes in 158 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 07 11:06:59.541646 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d490667b-3220-443e-8e97-8dc39cb67f11 None None] REQUEST_METHOD: `POST` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 11:06:59.542134 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d490667b-3220-443e-8e97-8dc39cb67f11 None None] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 11:06:59.542234 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d490667b-3220-443e-8e97-8dc39cb67f11 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 11:06:59.562563 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.auth.core [None req-d490667b-3220-443e-8e97-8dc39cb67f11 None None] MFA Rules not processed for user `bac4e6b76fe64217ac8fa44e71cffe6a`. Rule list: `[]` (Enabled: `True`). {{(pid=80144) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 11:06:59.563805 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.fernet_utils [None req-d490667b-3220-443e-8e97-8dc39cb67f11 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80144) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 11:06:59.625509 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 1414/2800] 10.222.0.25 () {58 vars in 958 bytes} [Sun Aug 7 11:06:59 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 07 11:06:59.636012 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0515e7d-38d5-41bb-8050-a26ee4004a15 admin admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 11:06:59.637487 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-d0515e7d-38d5-41bb-8050-a26ee4004a15 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 11:06:59.700022 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0515e7d-38d5-41bb-8050-a26ee4004a15 admin admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 11:06:59.703736 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f70567e-c894-4d29-a692-bcd55caad7b0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be1283c9bf47424c9aa3ad27cab03027', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6de1f3cac02c4bcdb05dd2337176033d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 11:06:59.704361 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f70567e-c894-4d29-a692-bcd55caad7b0 service neutron] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 11:06:59.704636 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f70567e-c894-4d29-a692-bcd55caad7b0 service neutron] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 11:06:59.704905 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f70567e-c894-4d29-a692-bcd55caad7b0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 11:06:59.715140 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f70567e-c894-4d29-a692-bcd55caad7b0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 11:06:59.716578 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f70567e-c894-4d29-a692-bcd55caad7b0 service neutron] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 11:06:59.728600 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1387/2801] 10.222.0.25 () {60 vars in 1365 bytes} [Sun Aug 7 11:06:59 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 07 11:07:00.022739 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d490667b-3220-443e-8e97-8dc39cb67f11 None None] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 11:07:00.024340 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.fernet_utils [None req-d490667b-3220-443e-8e97-8dc39cb67f11 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80144) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 11:07:00.050294 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d490667b-3220-443e-8e97-8dc39cb67f11 None None] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 11:07:00.060055 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e818c9f-7258-47f7-9688-9fc7b9f9c5cb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5ac47e3b09f43d19a23a0f457fe9e23', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 11:07:00.060055 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e818c9f-7258-47f7-9688-9fc7b9f9c5cb admin admin] REQUEST_METHOD: `DELETE` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 11:07:00.060055 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e818c9f-7258-47f7-9688-9fc7b9f9c5cb admin admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 11:07:00.060055 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e818c9f-7258-47f7-9688-9fc7b9f9c5cb admin admin] PATH_INFO: `/v3/projects/c7a01822626b4010aa9a21bed41ea4c3` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 11:07:00.060421 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e818c9f-7258-47f7-9688-9fc7b9f9c5cb admin admin] RBAC: Authorizing `identity:delete_project(project_id=c7a01822626b4010aa9a21bed41ea4c3)` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 11:07:00.060421 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e818c9f-7258-47f7-9688-9fc7b9f9c5cb admin admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 11:07:00.111706 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.notifications [None req-7e818c9f-7258-47f7-9688-9fc7b9f9c5cb admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c7a01822626b4010aa9a21bed41ea4c3', 'request_id': 'req-7e818c9f-7258-47f7-9688-9fc7b9f9c5cb'} {{(pid=80144) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 11:07:00.117223 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.notifications [None req-7e818c9f-7258-47f7-9688-9fc7b9f9c5cb admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c7a01822626b4010aa9a21bed41ea4c3', 'request_id': 'req-7e818c9f-7258-47f7-9688-9fc7b9f9c5cb'} {{(pid=80144) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 11:07:00.124616 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.notifications [None req-7e818c9f-7258-47f7-9688-9fc7b9f9c5cb admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c7a01822626b4010aa9a21bed41ea4c3', 'request_id': 'req-7e818c9f-7258-47f7-9688-9fc7b9f9c5cb'} {{(pid=80144) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 11:07:00.132141 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=90,keystone:INSERT=3,keystone:DELETE=7 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 11:07:00.137855 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.notifications [None req-7e818c9f-7258-47f7-9688-9fc7b9f9c5cb admin admin] The token cache is being invalidate because project c7a01822626b4010aa9a21bed41ea4c3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80144) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 07 11:07:00.138203 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.notifications [None req-7e818c9f-7258-47f7-9688-9fc7b9f9c5cb admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80144) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 11:07:00.138849 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.notifications [None req-7e818c9f-7258-47f7-9688-9fc7b9f9c5cb admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80144) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 11:07:00.140345 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 1415/2802] 10.222.0.25 () {58 vars in 1233 bytes} [Sun Aug 7 11:07:00 2022] DELETE /identity/v3/projects/c7a01822626b4010aa9a21bed41ea4c3 => generated 0 bytes in 118 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 07 11:07:00.144647 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f70567e-c894-4d29-a692-bcd55caad7b0 service neutron] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 11:07:00.155492 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f70567e-c894-4d29-a692-bcd55caad7b0 service neutron] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 11:07:00.175218 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 11:07:00.214383 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-010899d8-ec7d-47d3-8861-466550b61f17 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5ac47e3b09f43d19a23a0f457fe9e23', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 11:07:00.215035 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-010899d8-ec7d-47d3-8861-466550b61f17 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 11:07:00.215325 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-010899d8-ec7d-47d3-8861-466550b61f17 admin admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 11:07:00.215588 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-010899d8-ec7d-47d3-8861-466550b61f17 admin admin] PATH_INFO: `/v3/users/1a14bd06bacc452e988c20befdf534c4` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 11:07:00.216895 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-010899d8-ec7d-47d3-8861-466550b61f17 admin admin] RBAC: Authorizing `identity:delete_user(user_id=1a14bd06bacc452e988c20befdf534c4)` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 11:07:00.218799 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-010899d8-ec7d-47d3-8861-466550b61f17 admin admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 11:07:00.268515 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-010899d8-ec7d-47d3-8861-466550b61f17 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1a14bd06bacc452e988c20befdf534c4'} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 11:07:00.276175 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-010899d8-ec7d-47d3-8861-466550b61f17 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1a14bd06bacc452e988c20befdf534c4'} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 11:07:00.288107 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-010899d8-ec7d-47d3-8861-466550b61f17 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1a14bd06bacc452e988c20befdf534c4'} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 11:07:00.288817 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-010899d8-ec7d-47d3-8861-466550b61f17 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1a14bd06bacc452e988c20befdf534c4'} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 11:07:00.296129 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-010899d8-ec7d-47d3-8861-466550b61f17 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1a14bd06bacc452e988c20befdf534c4'} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 11:07:00.302423 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1388/2803] 10.222.0.25 () {58 vars in 1224 bytes} [Sun Aug 7 11:07:00 2022] DELETE /identity/v3/users/1a14bd06bacc452e988c20befdf534c4 => generated 0 bytes in 158 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 07 11:07:00.579606 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e818c9f-7258-47f7-9688-9fc7b9f9c5cb admin admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 11:07:00.581240 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.fernet_utils [None req-7e818c9f-7258-47f7-9688-9fc7b9f9c5cb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80144) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 11:07:00.597339 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e818c9f-7258-47f7-9688-9fc7b9f9c5cb admin admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 11:07:00.660266 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65cebf9d-9f6b-43a0-955c-b25cc0a0bd59 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5ac47e3b09f43d19a23a0f457fe9e23', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 11:07:00.661439 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65cebf9d-9f6b-43a0-955c-b25cc0a0bd59 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 11:07:00.661725 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65cebf9d-9f6b-43a0-955c-b25cc0a0bd59 admin admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 11:07:00.661995 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65cebf9d-9f6b-43a0-955c-b25cc0a0bd59 admin admin] PATH_INFO: `/v3/projects/747b7cbabb1d4cb7bec32d4f05432c63` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 11:07:00.663153 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65cebf9d-9f6b-43a0-955c-b25cc0a0bd59 admin admin] RBAC: Authorizing `identity:delete_project(project_id=747b7cbabb1d4cb7bec32d4f05432c63)` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 11:07:00.665037 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65cebf9d-9f6b-43a0-955c-b25cc0a0bd59 admin admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 11:07:00.714195 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.notifications [None req-65cebf9d-9f6b-43a0-955c-b25cc0a0bd59 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '747b7cbabb1d4cb7bec32d4f05432c63', 'request_id': 'req-65cebf9d-9f6b-43a0-955c-b25cc0a0bd59'} {{(pid=80144) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 11:07:00.719621 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.notifications [None req-65cebf9d-9f6b-43a0-955c-b25cc0a0bd59 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '747b7cbabb1d4cb7bec32d4f05432c63', 'request_id': 'req-65cebf9d-9f6b-43a0-955c-b25cc0a0bd59'} {{(pid=80144) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 11:07:00.727614 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.notifications [None req-65cebf9d-9f6b-43a0-955c-b25cc0a0bd59 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '747b7cbabb1d4cb7bec32d4f05432c63', 'request_id': 'req-65cebf9d-9f6b-43a0-955c-b25cc0a0bd59'} {{(pid=80144) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 11:07:00.740237 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.notifications [None req-65cebf9d-9f6b-43a0-955c-b25cc0a0bd59 admin admin] The token cache is being invalidate because project 747b7cbabb1d4cb7bec32d4f05432c63 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80144) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 07 11:07:00.740529 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.notifications [None req-65cebf9d-9f6b-43a0-955c-b25cc0a0bd59 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80144) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 11:07:00.741164 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.notifications [None req-65cebf9d-9f6b-43a0-955c-b25cc0a0bd59 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80144) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 11:07:00.742658 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 1416/2804] 10.222.0.25 () {58 vars in 1233 bytes} [Sun Aug 7 11:07:00 2022] DELETE /identity/v3/projects/747b7cbabb1d4cb7bec32d4f05432c63 => generated 0 bytes in 164 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 07 11:07:00.747525 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69e0b40e-34c8-4483-a57c-0ead55c62228 None None] REQUEST_METHOD: `POST` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 11:07:00.748009 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69e0b40e-34c8-4483-a57c-0ead55c62228 None None] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 11:07:00.748123 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69e0b40e-34c8-4483-a57c-0ead55c62228 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 11:07:00.768263 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.auth.core [None req-69e0b40e-34c8-4483-a57c-0ead55c62228 None None] MFA Rules not processed for user `bac4e6b76fe64217ac8fa44e71cffe6a`. Rule list: `[]` (Enabled: `True`). {{(pid=80143) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 11:07:00.769452 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-69e0b40e-34c8-4483-a57c-0ead55c62228 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 11:07:00.827171 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1389/2805] 10.222.0.25 () {58 vars in 958 bytes} [Sun Aug 7 11:07:00 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 07 11:07:00.836530 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-524960bd-6755-45fa-a8e1-cc5d34935164 None None] REQUEST_METHOD: `POST` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 11:07:00.836820 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-524960bd-6755-45fa-a8e1-cc5d34935164 None None] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 11:07:00.837121 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-524960bd-6755-45fa-a8e1-cc5d34935164 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 11:07:00.856398 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.auth.core [None req-524960bd-6755-45fa-a8e1-cc5d34935164 None None] MFA Rules not processed for user `bac4e6b76fe64217ac8fa44e71cffe6a`. Rule list: `[]` (Enabled: `True`). {{(pid=80144) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 11:07:00.857606 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.fernet_utils [None req-524960bd-6755-45fa-a8e1-cc5d34935164 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80144) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 11:07:00.866439 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 1417/2806] 10.222.0.25 () {58 vars in 958 bytes} [Sun Aug 7 11:07:00 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 07 11:07:00.870747 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69e0b40e-34c8-4483-a57c-0ead55c62228 None None] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 11:07:00.881021 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69e0b40e-34c8-4483-a57c-0ead55c62228 None None] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 11:07:00.885621 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5517ab6-d9f6-431e-a6e9-731a2b4eb5c4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5ac47e3b09f43d19a23a0f457fe9e23', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 11:07:00.886214 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5517ab6-d9f6-431e-a6e9-731a2b4eb5c4 admin admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 11:07:00.886488 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5517ab6-d9f6-431e-a6e9-731a2b4eb5c4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 11:07:00.886763 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5517ab6-d9f6-431e-a6e9-731a2b4eb5c4 admin admin] PATH_INFO: `/v3/domains` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 11:07:00.887450 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5517ab6-d9f6-431e-a6e9-731a2b4eb5c4 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80143) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 11:07:00.887755 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5517ab6-d9f6-431e-a6e9-731a2b4eb5c4 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 11:07:00.889350 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5517ab6-d9f6-431e-a6e9-731a2b4eb5c4 admin admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 11:07:00.900843 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1390/2807] 10.222.0.25 () {58 vars in 1153 bytes} [Sun Aug 7 11:07:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 11:07:00.905089 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-524960bd-6755-45fa-a8e1-cc5d34935164 None None] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 11:07:00.914548 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-524960bd-6755-45fa-a8e1-cc5d34935164 None None] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 11:07:00.919172 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-783a14a3-a570-4863-a172-fcdfc030f195 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5ac47e3b09f43d19a23a0f457fe9e23', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 11:07:00.919763 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-783a14a3-a570-4863-a172-fcdfc030f195 admin admin] REQUEST_METHOD: `POST` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 11:07:00.920045 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-783a14a3-a570-4863-a172-fcdfc030f195 admin admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 11:07:00.920321 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-783a14a3-a570-4863-a172-fcdfc030f195 admin admin] PATH_INFO: `/v3/projects` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 11:07:00.920990 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-783a14a3-a570-4863-a172-fcdfc030f195 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 11:07:00.922873 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-783a14a3-a570-4863-a172-fcdfc030f195 admin admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 11:07:00.950664 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 1418/2808] 10.222.0.25 () {60 vars in 1153 bytes} [Sun Aug 7 11:07:00 2022] POST /identity/v3/projects => generated 385 bytes in 46 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 07 11:07:00.954945 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5517ab6-d9f6-431e-a6e9-731a2b4eb5c4 admin admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 11:07:00.964789 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5517ab6-d9f6-431e-a6e9-731a2b4eb5c4 admin admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 11:07:00.991031 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 11:07:01.022451 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bfa9bdf-9a10-4638-a38a-5921001e4696 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5ac47e3b09f43d19a23a0f457fe9e23', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 11:07:01.023093 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bfa9bdf-9a10-4638-a38a-5921001e4696 admin admin] REQUEST_METHOD: `POST` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 11:07:01.023379 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bfa9bdf-9a10-4638-a38a-5921001e4696 admin admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 11:07:01.023655 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bfa9bdf-9a10-4638-a38a-5921001e4696 admin admin] PATH_INFO: `/v3/users` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 11:07:01.024321 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bfa9bdf-9a10-4638-a38a-5921001e4696 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 11:07:01.026223 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bfa9bdf-9a10-4638-a38a-5921001e4696 admin admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 11:07:01.046649 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1391/2809] 10.222.0.25 () {60 vars in 1144 bytes} [Sun Aug 7 11:07:00 2022] POST /identity/v3/users => generated 293 bytes in 92 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 07 11:07:01.050765 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-783a14a3-a570-4863-a172-fcdfc030f195 admin admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 11:07:01.061677 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-783a14a3-a570-4863-a172-fcdfc030f195 admin admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 11:07:01.066510 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e6de290-86ae-45e0-9707-f5be108b4eb7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5ac47e3b09f43d19a23a0f457fe9e23', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 11:07:01.067228 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e6de290-86ae-45e0-9707-f5be108b4eb7 admin admin] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 11:07:01.067508 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e6de290-86ae-45e0-9707-f5be108b4eb7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 11:07:01.067832 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e6de290-86ae-45e0-9707-f5be108b4eb7 admin admin] PATH_INFO: `/v3/roles` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 11:07:01.068463 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e6de290-86ae-45e0-9707-f5be108b4eb7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 11:07:01.070024 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e6de290-86ae-45e0-9707-f5be108b4eb7 admin admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 11:07:01.079855 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 1419/2810] 10.222.0.25 () {58 vars in 1122 bytes} [Sun Aug 7 11:07:01 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 07 11:07:01.084470 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bfa9bdf-9a10-4638-a38a-5921001e4696 admin admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 11:07:01.094191 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bfa9bdf-9a10-4638-a38a-5921001e4696 admin admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 11:07:01.099253 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e3ce1c0-c3b1-41fc-816b-4b5c6632df36 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5ac47e3b09f43d19a23a0f457fe9e23', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 11:07:01.099806 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e3ce1c0-c3b1-41fc-816b-4b5c6632df36 admin admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 11:07:01.100082 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e3ce1c0-c3b1-41fc-816b-4b5c6632df36 admin admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 11:07:01.100355 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e3ce1c0-c3b1-41fc-816b-4b5c6632df36 admin admin] PATH_INFO: `/v3/roles` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 11:07:01.100990 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e3ce1c0-c3b1-41fc-816b-4b5c6632df36 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 11:07:01.102537 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e3ce1c0-c3b1-41fc-816b-4b5c6632df36 admin admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 11:07:01.113181 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1392/2811] 10.222.0.25 () {58 vars in 1122 bytes} [Sun Aug 7 11:07:01 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 07 11:07:01.117913 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e6de290-86ae-45e0-9707-f5be108b4eb7 admin admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 11:07:01.127642 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e6de290-86ae-45e0-9707-f5be108b4eb7 admin admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 11:07:01.133024 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d2f07c3-0258-43f0-b563-89a0ae57b277 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5ac47e3b09f43d19a23a0f457fe9e23', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 11:07:01.133729 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d2f07c3-0258-43f0-b563-89a0ae57b277 admin admin] REQUEST_METHOD: `PUT` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 11:07:01.134012 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d2f07c3-0258-43f0-b563-89a0ae57b277 admin admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 11:07:01.134285 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d2f07c3-0258-43f0-b563-89a0ae57b277 admin admin] PATH_INFO: `/v3/projects/fa2426d53a444ba1802808859ab785b3/users/b96570f7d2e941899f3ea8b75f0195d2/roles/b98dfb3b9fee4644af2887a3ac3fd2e6` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 11:07:01.148773 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d2f07c3-0258-43f0-b563-89a0ae57b277 admin admin] RBAC: Authorizing `identity:create_grant(project_id=fa2426d53a444ba1802808859ab785b3, user_id=b96570f7d2e941899f3ea8b75f0195d2, role_id=b98dfb3b9fee4644af2887a3ac3fd2e6)` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 11:07:01.151477 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d2f07c3-0258-43f0-b563-89a0ae57b277 admin admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 11:07:01.165202 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 1420/2812] 10.222.0.25 () {60 vars in 1483 bytes} [Sun Aug 7 11:07:01 2022] PUT /identity/v3/projects/fa2426d53a444ba1802808859ab785b3/users/b96570f7d2e941899f3ea8b75f0195d2/roles/b98dfb3b9fee4644af2887a3ac3fd2e6 => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 07 11:07:01.175457 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dff98409-4969-476a-afcd-a1f7b12e3fdc None None] REQUEST_METHOD: `POST` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 11:07:01.175738 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dff98409-4969-476a-afcd-a1f7b12e3fdc None None] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 11:07:01.176009 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dff98409-4969-476a-afcd-a1f7b12e3fdc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 11:07:01.203254 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.auth.core [None req-dff98409-4969-476a-afcd-a1f7b12e3fdc None None] MFA Rules not processed for user `b96570f7d2e941899f3ea8b75f0195d2`. Rule list: `[]` (Enabled: `True`). {{(pid=80143) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 11:07:01.204550 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-dff98409-4969-476a-afcd-a1f7b12e3fdc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 11:07:01.272645 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1393/2813] 10.222.0.25 () {58 vars in 958 bytes} [Sun Aug 7 11:07:01 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 07 11:07:01.277301 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d2f07c3-0258-43f0-b563-89a0ae57b277 admin admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 11:07:01.304663 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 11:07:01.341514 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d2f07c3-0258-43f0-b563-89a0ae57b277 admin admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 11:07:01.346167 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30f18847-7db7-4175-bff6-6b382197b9e2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5ac47e3b09f43d19a23a0f457fe9e23', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 11:07:01.346766 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30f18847-7db7-4175-bff6-6b382197b9e2 admin admin] REQUEST_METHOD: `POST` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 11:07:01.347093 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30f18847-7db7-4175-bff6-6b382197b9e2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 11:07:01.347369 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30f18847-7db7-4175-bff6-6b382197b9e2 admin admin] PATH_INFO: `/v3/projects` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 11:07:01.347983 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30f18847-7db7-4175-bff6-6b382197b9e2 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 11:07:01.350894 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30f18847-7db7-4175-bff6-6b382197b9e2 admin admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 11:07:01.377834 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 1421/2814] 10.222.0.25 () {60 vars in 1153 bytes} [Sun Aug 7 11:07:01 2022] POST /identity/v3/projects => generated 385 bytes in 101 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 07 11:07:01.382573 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dff98409-4969-476a-afcd-a1f7b12e3fdc None None] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 11:07:01.393087 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dff98409-4969-476a-afcd-a1f7b12e3fdc None None] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 11:07:01.450869 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db1c3492-6001-4dd0-be83-b634e0c4c149 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5ac47e3b09f43d19a23a0f457fe9e23', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 11:07:01.451529 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db1c3492-6001-4dd0-be83-b634e0c4c149 admin admin] REQUEST_METHOD: `POST` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 11:07:01.451792 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db1c3492-6001-4dd0-be83-b634e0c4c149 admin admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 11:07:01.452068 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db1c3492-6001-4dd0-be83-b634e0c4c149 admin admin] PATH_INFO: `/v3/users` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 11:07:01.452682 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db1c3492-6001-4dd0-be83-b634e0c4c149 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 11:07:01.454608 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db1c3492-6001-4dd0-be83-b634e0c4c149 admin admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 11:07:01.479021 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1394/2815] 10.222.0.25 () {60 vars in 1144 bytes} [Sun Aug 7 11:07:01 2022] POST /identity/v3/users => generated 299 bytes in 97 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 07 11:07:01.483245 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30f18847-7db7-4175-bff6-6b382197b9e2 admin admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 11:07:01.494807 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30f18847-7db7-4175-bff6-6b382197b9e2 admin admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 11:07:01.499504 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e1c8f3d-75e0-4d39-b7c9-c734eea138da admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5ac47e3b09f43d19a23a0f457fe9e23', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 11:07:01.500100 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e1c8f3d-75e0-4d39-b7c9-c734eea138da admin admin] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 11:07:01.500381 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e1c8f3d-75e0-4d39-b7c9-c734eea138da admin admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 11:07:01.500667 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e1c8f3d-75e0-4d39-b7c9-c734eea138da admin admin] PATH_INFO: `/v3/roles` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 11:07:01.501284 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e1c8f3d-75e0-4d39-b7c9-c734eea138da admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 11:07:01.502865 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e1c8f3d-75e0-4d39-b7c9-c734eea138da admin admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 11:07:01.514452 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 1422/2816] 10.222.0.25 () {58 vars in 1122 bytes} [Sun Aug 7 11:07:01 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 07 11:07:01.518866 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db1c3492-6001-4dd0-be83-b634e0c4c149 admin admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 11:07:01.530540 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db1c3492-6001-4dd0-be83-b634e0c4c149 admin admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 11:07:01.535551 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61978a61-f595-401d-9b38-860e0ce2c554 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5ac47e3b09f43d19a23a0f457fe9e23', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 11:07:01.536797 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61978a61-f595-401d-9b38-860e0ce2c554 admin admin] REQUEST_METHOD: `PUT` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 11:07:01.537166 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61978a61-f595-401d-9b38-860e0ce2c554 admin admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 11:07:01.537504 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61978a61-f595-401d-9b38-860e0ce2c554 admin admin] PATH_INFO: `/v3/projects/5a56f9768a36497f82721b298e282288/users/beb84ce4a037469d8ec45609e5c15b9a/roles/81b1f1bcee2a473088fbbd327c1c5b23` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 11:07:01.553809 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61978a61-f595-401d-9b38-860e0ce2c554 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5a56f9768a36497f82721b298e282288, user_id=beb84ce4a037469d8ec45609e5c15b9a, role_id=81b1f1bcee2a473088fbbd327c1c5b23)` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 11:07:01.556559 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61978a61-f595-401d-9b38-860e0ce2c554 admin admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 11:07:01.570453 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1395/2817] 10.222.0.25 () {60 vars in 1483 bytes} [Sun Aug 7 11:07:01 2022] PUT /identity/v3/projects/5a56f9768a36497f82721b298e282288/users/beb84ce4a037469d8ec45609e5c15b9a/roles/81b1f1bcee2a473088fbbd327c1c5b23 => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 07 11:07:01.574502 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e1c8f3d-75e0-4d39-b7c9-c734eea138da admin admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 11:07:01.586797 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e1c8f3d-75e0-4d39-b7c9-c734eea138da admin admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 11:07:01.641042 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-021cb614-53fd-4dc2-a6fd-755b7dcef5a9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5ac47e3b09f43d19a23a0f457fe9e23', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 11:07:01.641650 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-021cb614-53fd-4dc2-a6fd-755b7dcef5a9 admin admin] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 11:07:01.641933 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-021cb614-53fd-4dc2-a6fd-755b7dcef5a9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 11:07:01.642212 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-021cb614-53fd-4dc2-a6fd-755b7dcef5a9 admin admin] PATH_INFO: `/v3/roles` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 11:07:01.642852 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-021cb614-53fd-4dc2-a6fd-755b7dcef5a9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 11:07:01.644639 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-021cb614-53fd-4dc2-a6fd-755b7dcef5a9 admin admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 11:07:01.654837 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 1423/2818] 10.222.0.25 () {58 vars in 1122 bytes} [Sun Aug 7 11:07:01 2022] GET /identity/v3/roles => generated 1420 bytes in 81 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 07 11:07:01.659090 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61978a61-f595-401d-9b38-860e0ce2c554 admin admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 11:07:01.665208 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 11:07:01.671609 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61978a61-f595-401d-9b38-860e0ce2c554 admin admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 11:07:01.676434 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faa56a9b-5863-4335-aa35-e1c56d0346c1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5ac47e3b09f43d19a23a0f457fe9e23', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 11:07:01.677181 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faa56a9b-5863-4335-aa35-e1c56d0346c1 admin admin] REQUEST_METHOD: `PUT` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 11:07:01.677467 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faa56a9b-5863-4335-aa35-e1c56d0346c1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 11:07:01.677745 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faa56a9b-5863-4335-aa35-e1c56d0346c1 admin admin] PATH_INFO: `/v3/projects/5a56f9768a36497f82721b298e282288/users/beb84ce4a037469d8ec45609e5c15b9a/roles/81b1f1bcee2a473088fbbd327c1c5b23` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 11:07:01.679695 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faa56a9b-5863-4335-aa35-e1c56d0346c1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5a56f9768a36497f82721b298e282288, user_id=beb84ce4a037469d8ec45609e5c15b9a, role_id=81b1f1bcee2a473088fbbd327c1c5b23)` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 11:07:01.682336 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faa56a9b-5863-4335-aa35-e1c56d0346c1 admin admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 11:07:01.692592 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1396/2819] 10.222.0.25 () {60 vars in 1483 bytes} [Sun Aug 7 11:07:01 2022] PUT /identity/v3/projects/5a56f9768a36497f82721b298e282288/users/beb84ce4a037469d8ec45609e5c15b9a/roles/81b1f1bcee2a473088fbbd327c1c5b23 => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 07 11:07:01.702770 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e08224bb-1974-4467-b3f6-eb2eefd300d1 None None] REQUEST_METHOD: `POST` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 11:07:01.703089 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e08224bb-1974-4467-b3f6-eb2eefd300d1 None None] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 11:07:01.703371 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e08224bb-1974-4467-b3f6-eb2eefd300d1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 11:07:01.732787 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.auth.core [None req-e08224bb-1974-4467-b3f6-eb2eefd300d1 None None] MFA Rules not processed for user `beb84ce4a037469d8ec45609e5c15b9a`. Rule list: `[]` (Enabled: `True`). {{(pid=80144) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 11:07:01.734268 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.fernet_utils [None req-e08224bb-1974-4467-b3f6-eb2eefd300d1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80144) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 11:07:01.805450 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 1424/2820] 10.222.0.25 () {58 vars in 958 bytes} [Sun Aug 7 11:07:01 2022] POST /identity/v3/auth/tokens => generated 3329 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 07 11:07:01.817899 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faa56a9b-5863-4335-aa35-e1c56d0346c1 admin admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 11:07:01.819851 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-faa56a9b-5863-4335-aa35-e1c56d0346c1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 11:07:01.877888 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faa56a9b-5863-4335-aa35-e1c56d0346c1 admin admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 11:07:01.882106 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f36e03a2-0190-431e-88ae-8a1885141daa service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88dbd6dfd1164994a3e16da553561910', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6de1f3cac02c4bcdb05dd2337176033d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 11:07:01.882712 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f36e03a2-0190-431e-88ae-8a1885141daa service cinder] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 11:07:01.883023 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f36e03a2-0190-431e-88ae-8a1885141daa service cinder] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 11:07:01.883301 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f36e03a2-0190-431e-88ae-8a1885141daa service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 11:07:01.892878 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f36e03a2-0190-431e-88ae-8a1885141daa service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 11:07:01.894289 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f36e03a2-0190-431e-88ae-8a1885141daa service cinder] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 11:07:01.904857 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1397/2821] 10.222.0.25 () {60 vars in 1365 bytes} [Sun Aug 7 11:07:01 2022] GET /identity/v3/auth/tokens => generated 3329 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 07 11:07:01.933817 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e08224bb-1974-4467-b3f6-eb2eefd300d1 None None] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 11:07:01.955011 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 11:07:01.997618 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e08224bb-1974-4467-b3f6-eb2eefd300d1 None None] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 11:07:02.002186 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7030b1f1-1eb6-4d2a-8982-ff6bb23e78ee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5ac47e3b09f43d19a23a0f457fe9e23', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 11:07:02.002808 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7030b1f1-1eb6-4d2a-8982-ff6bb23e78ee admin admin] REQUEST_METHOD: `DELETE` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 11:07:02.003120 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7030b1f1-1eb6-4d2a-8982-ff6bb23e78ee admin admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 11:07:02.003396 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7030b1f1-1eb6-4d2a-8982-ff6bb23e78ee admin admin] PATH_INFO: `/v3/users/b96570f7d2e941899f3ea8b75f0195d2` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 11:07:02.004522 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7030b1f1-1eb6-4d2a-8982-ff6bb23e78ee admin admin] RBAC: Authorizing `identity:delete_user(user_id=b96570f7d2e941899f3ea8b75f0195d2)` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 11:07:02.007339 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7030b1f1-1eb6-4d2a-8982-ff6bb23e78ee admin admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 11:07:02.053755 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.notifications [None req-7030b1f1-1eb6-4d2a-8982-ff6bb23e78ee admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b96570f7d2e941899f3ea8b75f0195d2'} {{(pid=80144) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 11:07:02.060588 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.notifications [None req-7030b1f1-1eb6-4d2a-8982-ff6bb23e78ee admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b96570f7d2e941899f3ea8b75f0195d2'} {{(pid=80144) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 11:07:02.068445 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.notifications [None req-7030b1f1-1eb6-4d2a-8982-ff6bb23e78ee admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b96570f7d2e941899f3ea8b75f0195d2'} {{(pid=80144) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 11:07:02.069218 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.notifications [None req-7030b1f1-1eb6-4d2a-8982-ff6bb23e78ee admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b96570f7d2e941899f3ea8b75f0195d2'} {{(pid=80144) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 11:07:02.069892 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.notifications [None req-7030b1f1-1eb6-4d2a-8982-ff6bb23e78ee admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b96570f7d2e941899f3ea8b75f0195d2'} {{(pid=80144) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 11:07:02.087957 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 1425/2822] 10.222.0.25 () {58 vars in 1224 bytes} [Sun Aug 7 11:07:01 2022] DELETE /identity/v3/users/b96570f7d2e941899f3ea8b75f0195d2 => generated 0 bytes in 155 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 07 11:07:02.092428 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-897daf89-741e-4df7-b40c-c8482c64e036 None None] REQUEST_METHOD: `POST` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 11:07:02.092709 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-897daf89-741e-4df7-b40c-c8482c64e036 None None] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 11:07:02.092980 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-897daf89-741e-4df7-b40c-c8482c64e036 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 11:07:02.111425 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.auth.core [None req-897daf89-741e-4df7-b40c-c8482c64e036 None None] MFA Rules not processed for user `bac4e6b76fe64217ac8fa44e71cffe6a`. Rule list: `[]` (Enabled: `True`). {{(pid=80143) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 11:07:02.112597 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-897daf89-741e-4df7-b40c-c8482c64e036 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 11:07:02.170326 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1398/2823] 10.222.0.25 () {58 vars in 958 bytes} [Sun Aug 7 11:07:02 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 07 11:07:02.180976 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7030b1f1-1eb6-4d2a-8982-ff6bb23e78ee admin admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 11:07:02.182346 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.fernet_utils [None req-7030b1f1-1eb6-4d2a-8982-ff6bb23e78ee admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80144) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 11:07:02.239427 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7030b1f1-1eb6-4d2a-8982-ff6bb23e78ee admin admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 11:07:02.243705 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1808b669-57fb-44a0-93da-646ab24a36c6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be1283c9bf47424c9aa3ad27cab03027', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6de1f3cac02c4bcdb05dd2337176033d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 11:07:02.244309 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1808b669-57fb-44a0-93da-646ab24a36c6 service neutron] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 11:07:02.244586 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1808b669-57fb-44a0-93da-646ab24a36c6 service neutron] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 11:07:02.244852 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1808b669-57fb-44a0-93da-646ab24a36c6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 11:07:02.253630 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1808b669-57fb-44a0-93da-646ab24a36c6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 11:07:02.255086 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1808b669-57fb-44a0-93da-646ab24a36c6 service neutron] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 11:07:02.264826 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 1426/2824] 10.222.0.25 () {60 vars in 1365 bytes} [Sun Aug 7 11:07:02 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 07 11:07:02.528913 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-897daf89-741e-4df7-b40c-c8482c64e036 None None] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 11:07:02.530404 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-897daf89-741e-4df7-b40c-c8482c64e036 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 11:07:02.544428 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-897daf89-741e-4df7-b40c-c8482c64e036 None None] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 11:07:02.549410 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-810e84cf-59f3-41b4-97cc-080edde595bd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5ac47e3b09f43d19a23a0f457fe9e23', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 11:07:02.550193 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-810e84cf-59f3-41b4-97cc-080edde595bd admin admin] REQUEST_METHOD: `DELETE` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 11:07:02.550537 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-810e84cf-59f3-41b4-97cc-080edde595bd admin admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 11:07:02.550893 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-810e84cf-59f3-41b4-97cc-080edde595bd admin admin] PATH_INFO: `/v3/projects/fa2426d53a444ba1802808859ab785b3` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 11:07:02.553335 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-810e84cf-59f3-41b4-97cc-080edde595bd admin admin] RBAC: Authorizing `identity:delete_project(project_id=fa2426d53a444ba1802808859ab785b3)` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 11:07:02.555463 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-810e84cf-59f3-41b4-97cc-080edde595bd admin admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 11:07:02.602152 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-810e84cf-59f3-41b4-97cc-080edde595bd admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'fa2426d53a444ba1802808859ab785b3', 'request_id': 'req-810e84cf-59f3-41b4-97cc-080edde595bd'} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 11:07:02.607810 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-810e84cf-59f3-41b4-97cc-080edde595bd admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'fa2426d53a444ba1802808859ab785b3', 'request_id': 'req-810e84cf-59f3-41b4-97cc-080edde595bd'} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 11:07:02.618567 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-810e84cf-59f3-41b4-97cc-080edde595bd admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'fa2426d53a444ba1802808859ab785b3', 'request_id': 'req-810e84cf-59f3-41b4-97cc-080edde595bd'} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 11:07:02.632334 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-810e84cf-59f3-41b4-97cc-080edde595bd admin admin] The token cache is being invalidate because project fa2426d53a444ba1802808859ab785b3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80143) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 07 11:07:02.632855 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-810e84cf-59f3-41b4-97cc-080edde595bd admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 11:07:02.633488 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-810e84cf-59f3-41b4-97cc-080edde595bd admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 11:07:02.635101 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1399/2825] 10.222.0.25 () {58 vars in 1233 bytes} [Sun Aug 7 11:07:02 2022] DELETE /identity/v3/projects/fa2426d53a444ba1802808859ab785b3 => generated 0 bytes in 107 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 07 11:07:02.639866 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1808b669-57fb-44a0-93da-646ab24a36c6 service neutron] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 11:07:02.650820 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1808b669-57fb-44a0-93da-646ab24a36c6 service neutron] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 11:07:02.695337 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 11:07:02.715547 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca75492a-b48d-4f3f-b067-5903d0e7301a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5ac47e3b09f43d19a23a0f457fe9e23', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 11:07:02.716271 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca75492a-b48d-4f3f-b067-5903d0e7301a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 11:07:02.716557 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca75492a-b48d-4f3f-b067-5903d0e7301a admin admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 11:07:02.716833 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca75492a-b48d-4f3f-b067-5903d0e7301a admin admin] PATH_INFO: `/v3/users/beb84ce4a037469d8ec45609e5c15b9a` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 11:07:02.718130 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca75492a-b48d-4f3f-b067-5903d0e7301a admin admin] RBAC: Authorizing `identity:delete_user(user_id=beb84ce4a037469d8ec45609e5c15b9a)` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 11:07:02.720337 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca75492a-b48d-4f3f-b067-5903d0e7301a admin admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 11:07:02.767653 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.notifications [None req-ca75492a-b48d-4f3f-b067-5903d0e7301a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'beb84ce4a037469d8ec45609e5c15b9a'} {{(pid=80144) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 11:07:02.774735 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.notifications [None req-ca75492a-b48d-4f3f-b067-5903d0e7301a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'beb84ce4a037469d8ec45609e5c15b9a'} {{(pid=80144) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 11:07:02.782041 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.notifications [None req-ca75492a-b48d-4f3f-b067-5903d0e7301a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'beb84ce4a037469d8ec45609e5c15b9a'} {{(pid=80144) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 11:07:02.782644 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.notifications [None req-ca75492a-b48d-4f3f-b067-5903d0e7301a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'beb84ce4a037469d8ec45609e5c15b9a'} {{(pid=80144) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 11:07:02.783294 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.notifications [None req-ca75492a-b48d-4f3f-b067-5903d0e7301a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'beb84ce4a037469d8ec45609e5c15b9a'} {{(pid=80144) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 11:07:02.800899 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 1427/2826] 10.222.0.25 () {58 vars in 1224 bytes} [Sun Aug 7 11:07:02 2022] DELETE /identity/v3/users/beb84ce4a037469d8ec45609e5c15b9a => generated 0 bytes in 162 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 07 11:07:03.073370 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-810e84cf-59f3-41b4-97cc-080edde595bd admin admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 11:07:03.074913 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-810e84cf-59f3-41b4-97cc-080edde595bd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 11:07:03.088549 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-810e84cf-59f3-41b4-97cc-080edde595bd admin admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 11:07:03.101139 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 11:07:03.148045 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6faed3e-a558-411b-ba40-3ecb63c60302 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5ac47e3b09f43d19a23a0f457fe9e23', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 11:07:03.148678 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6faed3e-a558-411b-ba40-3ecb63c60302 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 11:07:03.148960 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6faed3e-a558-411b-ba40-3ecb63c60302 admin admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 11:07:03.149238 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6faed3e-a558-411b-ba40-3ecb63c60302 admin admin] PATH_INFO: `/v3/projects/5a56f9768a36497f82721b298e282288` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 11:07:03.150351 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6faed3e-a558-411b-ba40-3ecb63c60302 admin admin] RBAC: Authorizing `identity:delete_project(project_id=5a56f9768a36497f82721b298e282288)` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 11:07:03.152289 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6faed3e-a558-411b-ba40-3ecb63c60302 admin admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 11:07:03.197768 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-c6faed3e-a558-411b-ba40-3ecb63c60302 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5a56f9768a36497f82721b298e282288', 'request_id': 'req-c6faed3e-a558-411b-ba40-3ecb63c60302'} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 11:07:03.203215 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-c6faed3e-a558-411b-ba40-3ecb63c60302 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5a56f9768a36497f82721b298e282288', 'request_id': 'req-c6faed3e-a558-411b-ba40-3ecb63c60302'} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 11:07:03.209836 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-c6faed3e-a558-411b-ba40-3ecb63c60302 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5a56f9768a36497f82721b298e282288', 'request_id': 'req-c6faed3e-a558-411b-ba40-3ecb63c60302'} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 11:07:03.225048 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-c6faed3e-a558-411b-ba40-3ecb63c60302 admin admin] The token cache is being invalidate because project 5a56f9768a36497f82721b298e282288 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80143) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 07 11:07:03.225341 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-c6faed3e-a558-411b-ba40-3ecb63c60302 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 11:07:03.225967 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-c6faed3e-a558-411b-ba40-3ecb63c60302 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 11:07:03.227491 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1400/2827] 10.222.0.25 () {58 vars in 1233 bytes} [Sun Aug 7 11:07:03 2022] DELETE /identity/v3/projects/5a56f9768a36497f82721b298e282288 => generated 0 bytes in 155 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 07 11:07:03.232521 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c0c14fd-9dbc-47d7-8e01-82060bdba702 None None] REQUEST_METHOD: `POST` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 11:07:03.232807 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c0c14fd-9dbc-47d7-8e01-82060bdba702 None None] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 11:07:03.233080 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c0c14fd-9dbc-47d7-8e01-82060bdba702 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 11:07:03.252476 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.auth.core [None req-3c0c14fd-9dbc-47d7-8e01-82060bdba702 None None] MFA Rules not processed for user `bac4e6b76fe64217ac8fa44e71cffe6a`. Rule list: `[]` (Enabled: `True`). {{(pid=80144) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 11:07:03.253650 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.fernet_utils [None req-3c0c14fd-9dbc-47d7-8e01-82060bdba702 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80144) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 11:07:03.310534 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 1428/2828] 10.222.0.25 () {58 vars in 958 bytes} [Sun Aug 7 11:07:03 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 07 11:07:03.320068 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d02d886-ace5-404b-b49e-459f212d2f7a None None] REQUEST_METHOD: `POST` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 11:07:03.320356 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d02d886-ace5-404b-b49e-459f212d2f7a None None] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 11:07:03.320701 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d02d886-ace5-404b-b49e-459f212d2f7a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 11:07:03.340401 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.auth.core [None req-9d02d886-ace5-404b-b49e-459f212d2f7a None None] MFA Rules not processed for user `bac4e6b76fe64217ac8fa44e71cffe6a`. Rule list: `[]` (Enabled: `True`). {{(pid=80143) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 11:07:03.341590 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-9d02d886-ace5-404b-b49e-459f212d2f7a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 11:07:03.350301 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1401/2829] 10.222.0.25 () {58 vars in 958 bytes} [Sun Aug 7 11:07:03 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 07 11:07:03.354198 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c0c14fd-9dbc-47d7-8e01-82060bdba702 None None] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 11:07:03.364300 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c0c14fd-9dbc-47d7-8e01-82060bdba702 None None] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 11:07:03.369675 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-629dc4b0-d57c-41fc-bb18-492c28f519b9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5ac47e3b09f43d19a23a0f457fe9e23', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 11:07:03.370277 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-629dc4b0-d57c-41fc-bb18-492c28f519b9 admin admin] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 11:07:03.370560 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-629dc4b0-d57c-41fc-bb18-492c28f519b9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 11:07:03.370836 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-629dc4b0-d57c-41fc-bb18-492c28f519b9 admin admin] PATH_INFO: `/v3/domains` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 11:07:03.371517 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-629dc4b0-d57c-41fc-bb18-492c28f519b9 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80144) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 11:07:03.371826 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-629dc4b0-d57c-41fc-bb18-492c28f519b9 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 11:07:03.373404 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-629dc4b0-d57c-41fc-bb18-492c28f519b9 admin admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 11:07:03.384830 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 1429/2830] 10.222.0.25 () {58 vars in 1153 bytes} [Sun Aug 7 11:07:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 11:07:03.388985 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d02d886-ace5-404b-b49e-459f212d2f7a None None] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 11:07:03.398233 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d02d886-ace5-404b-b49e-459f212d2f7a None None] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 11:07:03.402831 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2800c50-4653-46fa-b3cd-77b248c869e6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5ac47e3b09f43d19a23a0f457fe9e23', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 11:07:03.403451 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2800c50-4653-46fa-b3cd-77b248c869e6 admin admin] REQUEST_METHOD: `POST` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 11:07:03.403730 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2800c50-4653-46fa-b3cd-77b248c869e6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 11:07:03.404005 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2800c50-4653-46fa-b3cd-77b248c869e6 admin admin] PATH_INFO: `/v3/projects` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 11:07:03.404621 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2800c50-4653-46fa-b3cd-77b248c869e6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 11:07:03.406527 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2800c50-4653-46fa-b3cd-77b248c869e6 admin admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 11:07:03.434496 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1402/2831] 10.222.0.25 () {60 vars in 1153 bytes} [Sun Aug 7 11:07:03 2022] POST /identity/v3/projects => generated 387 bytes in 46 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 07 11:07:03.438706 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-629dc4b0-d57c-41fc-bb18-492c28f519b9 admin admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 11:07:03.449079 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-629dc4b0-d57c-41fc-bb18-492c28f519b9 admin admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 11:07:03.495069 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 11:07:03.508141 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0961c8a-487f-463f-a0bd-3e93d8b12bf5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5ac47e3b09f43d19a23a0f457fe9e23', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 11:07:03.508749 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0961c8a-487f-463f-a0bd-3e93d8b12bf5 admin admin] REQUEST_METHOD: `POST` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 11:07:03.509032 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0961c8a-487f-463f-a0bd-3e93d8b12bf5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 11:07:03.509308 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0961c8a-487f-463f-a0bd-3e93d8b12bf5 admin admin] PATH_INFO: `/v3/users` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 11:07:03.509940 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0961c8a-487f-463f-a0bd-3e93d8b12bf5 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 11:07:03.511882 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0961c8a-487f-463f-a0bd-3e93d8b12bf5 admin admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 11:07:03.532665 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 1430/2832] 10.222.0.25 () {60 vars in 1144 bytes} [Sun Aug 7 11:07:03 2022] POST /identity/v3/users => generated 294 bytes in 94 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 07 11:07:03.537456 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2800c50-4653-46fa-b3cd-77b248c869e6 admin admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 11:07:03.547873 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2800c50-4653-46fa-b3cd-77b248c869e6 admin admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 11:07:03.553014 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3053f6ae-50bc-46c1-a6a4-a2d189323299 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5ac47e3b09f43d19a23a0f457fe9e23', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 11:07:03.553638 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3053f6ae-50bc-46c1-a6a4-a2d189323299 admin admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 11:07:03.553915 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3053f6ae-50bc-46c1-a6a4-a2d189323299 admin admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 11:07:03.554188 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3053f6ae-50bc-46c1-a6a4-a2d189323299 admin admin] PATH_INFO: `/v3/roles` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 11:07:03.554820 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3053f6ae-50bc-46c1-a6a4-a2d189323299 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 11:07:03.556418 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3053f6ae-50bc-46c1-a6a4-a2d189323299 admin admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 11:07:03.566183 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1403/2833] 10.222.0.25 () {58 vars in 1122 bytes} [Sun Aug 7 11:07:03 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 07 11:07:03.570636 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0961c8a-487f-463f-a0bd-3e93d8b12bf5 admin admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 11:07:03.580518 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0961c8a-487f-463f-a0bd-3e93d8b12bf5 admin admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 11:07:03.586557 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd4715f7-a40e-4988-b4e9-5fd276df4caf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5ac47e3b09f43d19a23a0f457fe9e23', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 11:07:03.587183 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd4715f7-a40e-4988-b4e9-5fd276df4caf admin admin] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 11:07:03.587487 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd4715f7-a40e-4988-b4e9-5fd276df4caf admin admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 11:07:03.587759 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd4715f7-a40e-4988-b4e9-5fd276df4caf admin admin] PATH_INFO: `/v3/roles` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 11:07:03.588392 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd4715f7-a40e-4988-b4e9-5fd276df4caf admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 11:07:03.590003 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd4715f7-a40e-4988-b4e9-5fd276df4caf admin admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 11:07:03.600552 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 1431/2834] 10.222.0.25 () {58 vars in 1122 bytes} [Sun Aug 7 11:07:03 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 07 11:07:03.604468 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3053f6ae-50bc-46c1-a6a4-a2d189323299 admin admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 11:07:03.614338 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3053f6ae-50bc-46c1-a6a4-a2d189323299 admin admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 11:07:03.619607 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81dba40a-bb5a-4753-917a-e4fdc6b66ec0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5ac47e3b09f43d19a23a0f457fe9e23', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 11:07:03.620242 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81dba40a-bb5a-4753-917a-e4fdc6b66ec0 admin admin] REQUEST_METHOD: `PUT` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 11:07:03.620520 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81dba40a-bb5a-4753-917a-e4fdc6b66ec0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 11:07:03.620791 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81dba40a-bb5a-4753-917a-e4fdc6b66ec0 admin admin] PATH_INFO: `/v3/projects/b4470c12104d473cb5b5082baf138315/users/837bdd0a3db34bc98e7b0f5252463eda/roles/b98dfb3b9fee4644af2887a3ac3fd2e6` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 11:07:03.636323 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81dba40a-bb5a-4753-917a-e4fdc6b66ec0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b4470c12104d473cb5b5082baf138315, user_id=837bdd0a3db34bc98e7b0f5252463eda, role_id=b98dfb3b9fee4644af2887a3ac3fd2e6)` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 11:07:03.638905 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81dba40a-bb5a-4753-917a-e4fdc6b66ec0 admin admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 11:07:03.652014 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1404/2835] 10.222.0.25 () {60 vars in 1483 bytes} [Sun Aug 7 11:07:03 2022] PUT /identity/v3/projects/b4470c12104d473cb5b5082baf138315/users/837bdd0a3db34bc98e7b0f5252463eda/roles/b98dfb3b9fee4644af2887a3ac3fd2e6 => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 07 11:07:03.662146 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f0832f0-5a20-4375-a7ba-7df2912743ba None None] REQUEST_METHOD: `POST` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 11:07:03.662652 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f0832f0-5a20-4375-a7ba-7df2912743ba None None] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 11:07:03.662754 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f0832f0-5a20-4375-a7ba-7df2912743ba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 11:07:03.689820 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.auth.core [None req-8f0832f0-5a20-4375-a7ba-7df2912743ba None None] MFA Rules not processed for user `837bdd0a3db34bc98e7b0f5252463eda`. Rule list: `[]` (Enabled: `True`). {{(pid=80144) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 11:07:03.691028 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.fernet_utils [None req-8f0832f0-5a20-4375-a7ba-7df2912743ba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80144) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 11:07:03.751268 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81dba40a-bb5a-4753-917a-e4fdc6b66ec0 admin admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 11:07:03.752940 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-81dba40a-bb5a-4753-917a-e4fdc6b66ec0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 11:07:03.757965 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 1432/2836] 10.222.0.25 () {58 vars in 958 bytes} [Sun Aug 7 11:07:03 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 07 11:07:03.762370 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f0832f0-5a20-4375-a7ba-7df2912743ba None None] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 11:07:03.799717 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 11:07:03.822297 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81dba40a-bb5a-4753-917a-e4fdc6b66ec0 admin admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 11:07:03.823825 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f0832f0-5a20-4375-a7ba-7df2912743ba None None] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 11:07:03.826831 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06247a98-5f4c-4ca5-a904-92b8970eb79c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5ac47e3b09f43d19a23a0f457fe9e23', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 11:07:03.827515 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06247a98-5f4c-4ca5-a904-92b8970eb79c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 11:07:03.827793 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06247a98-5f4c-4ca5-a904-92b8970eb79c admin admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 11:07:03.828066 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06247a98-5f4c-4ca5-a904-92b8970eb79c admin admin] PATH_INFO: `/v3/users/9b12d779775546f392f64db839173141` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 11:07:03.828783 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03724562-8bed-4cd9-92fd-1ee24c4501c4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5ac47e3b09f43d19a23a0f457fe9e23', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 11:07:03.829186 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06247a98-5f4c-4ca5-a904-92b8970eb79c admin admin] RBAC: Authorizing `identity:delete_user(user_id=9b12d779775546f392f64db839173141)` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 11:07:03.829409 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03724562-8bed-4cd9-92fd-1ee24c4501c4 admin admin] REQUEST_METHOD: `POST` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 11:07:03.829819 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03724562-8bed-4cd9-92fd-1ee24c4501c4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 11:07:03.830188 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03724562-8bed-4cd9-92fd-1ee24c4501c4 admin admin] PATH_INFO: `/v3/projects` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 11:07:03.830841 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03724562-8bed-4cd9-92fd-1ee24c4501c4 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 11:07:03.831153 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06247a98-5f4c-4ca5-a904-92b8970eb79c admin admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 11:07:03.832765 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03724562-8bed-4cd9-92fd-1ee24c4501c4 admin admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 11:07:03.860343 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 1433/2837] 10.222.0.25 () {60 vars in 1153 bytes} [Sun Aug 7 11:07:03 2022] POST /identity/v3/projects => generated 387 bytes in 98 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 07 11:07:03.864650 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03724562-8bed-4cd9-92fd-1ee24c4501c4 admin admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 11:07:03.882213 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-06247a98-5f4c-4ca5-a904-92b8970eb79c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9b12d779775546f392f64db839173141'} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 11:07:03.889357 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-06247a98-5f4c-4ca5-a904-92b8970eb79c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9b12d779775546f392f64db839173141'} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 11:07:03.902688 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-06247a98-5f4c-4ca5-a904-92b8970eb79c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9b12d779775546f392f64db839173141'} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 11:07:03.903538 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-06247a98-5f4c-4ca5-a904-92b8970eb79c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9b12d779775546f392f64db839173141'} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 11:07:03.911327 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-06247a98-5f4c-4ca5-a904-92b8970eb79c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9b12d779775546f392f64db839173141'} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 11:07:03.912654 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 11:07:03.919210 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1405/2838] 10.222.0.25 () {58 vars in 1224 bytes} [Sun Aug 7 11:07:03 2022] DELETE /identity/v3/users/9b12d779775546f392f64db839173141 => generated 0 bytes in 168 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 07 11:07:03.923524 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4911a38b-2ef0-4b93-9303-4f861b5d45c1 None None] REQUEST_METHOD: `POST` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 11:07:03.923806 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4911a38b-2ef0-4b93-9303-4f861b5d45c1 None None] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 11:07:03.924084 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4911a38b-2ef0-4b93-9303-4f861b5d45c1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 11:07:03.933007 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03724562-8bed-4cd9-92fd-1ee24c4501c4 admin admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 11:07:03.944397 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.auth.core [None req-4911a38b-2ef0-4b93-9303-4f861b5d45c1 None None] MFA Rules not processed for user `bac4e6b76fe64217ac8fa44e71cffe6a`. Rule list: `[]` (Enabled: `True`). {{(pid=80143) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 11:07:03.945585 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-4911a38b-2ef0-4b93-9303-4f861b5d45c1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 11:07:03.995729 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b1dffb3-8588-4756-964d-d481c0011d5d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5ac47e3b09f43d19a23a0f457fe9e23', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 11:07:03.996317 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b1dffb3-8588-4756-964d-d481c0011d5d admin admin] REQUEST_METHOD: `POST` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 11:07:03.996594 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b1dffb3-8588-4756-964d-d481c0011d5d admin admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 11:07:03.996871 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b1dffb3-8588-4756-964d-d481c0011d5d admin admin] PATH_INFO: `/v3/users` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 11:07:03.997503 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b1dffb3-8588-4756-964d-d481c0011d5d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 11:07:03.999470 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b1dffb3-8588-4756-964d-d481c0011d5d admin admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 11:07:04.009206 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1406/2839] 10.222.0.25 () {58 vars in 958 bytes} [Sun Aug 7 11:07:03 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 07 11:07:04.019384 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4911a38b-2ef0-4b93-9303-4f861b5d45c1 None None] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 11:07:04.020752 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-4911a38b-2ef0-4b93-9303-4f861b5d45c1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 11:07:04.025847 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 1434/2840] 10.222.0.25 () {60 vars in 1144 bytes} [Sun Aug 7 11:07:03 2022] POST /identity/v3/users => generated 300 bytes in 159 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 07 11:07:04.026810 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b1dffb3-8588-4756-964d-d481c0011d5d admin admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 11:07:04.028541 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.fernet_utils [None req-4b1dffb3-8588-4756-964d-d481c0011d5d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80144) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 11:07:04.040677 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b1dffb3-8588-4756-964d-d481c0011d5d admin admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 11:07:04.045479 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fdb0f59-6221-4267-b842-e3c4934e81e5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5ac47e3b09f43d19a23a0f457fe9e23', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 11:07:04.046075 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fdb0f59-6221-4267-b842-e3c4934e81e5 admin admin] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 11:07:04.046380 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fdb0f59-6221-4267-b842-e3c4934e81e5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 11:07:04.046653 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fdb0f59-6221-4267-b842-e3c4934e81e5 admin admin] PATH_INFO: `/v3/roles` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 11:07:04.047320 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fdb0f59-6221-4267-b842-e3c4934e81e5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 11:07:04.048884 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fdb0f59-6221-4267-b842-e3c4934e81e5 admin admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 11:07:04.059215 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 1435/2841] 10.222.0.25 () {58 vars in 1122 bytes} [Sun Aug 7 11:07:04 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 07 11:07:04.063104 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fdb0f59-6221-4267-b842-e3c4934e81e5 admin admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 11:07:04.073286 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fdb0f59-6221-4267-b842-e3c4934e81e5 admin admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 11:07:04.077863 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7898305b-9700-44db-9079-3b9b4d353e67 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5ac47e3b09f43d19a23a0f457fe9e23', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 11:07:04.078208 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4911a38b-2ef0-4b93-9303-4f861b5d45c1 None None] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 11:07:04.078691 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7898305b-9700-44db-9079-3b9b4d353e67 admin admin] REQUEST_METHOD: `PUT` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 11:07:04.078797 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7898305b-9700-44db-9079-3b9b4d353e67 admin admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 11:07:04.079041 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7898305b-9700-44db-9079-3b9b4d353e67 admin admin] PATH_INFO: `/v3/projects/f297e44655a2492fba71d79c44a7e7f9/users/c74b79cc5bc74154963f3e6ca7ec5393/roles/81b1f1bcee2a473088fbbd327c1c5b23` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 11:07:04.082019 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53e2971e-db93-4ba7-b036-7e32e55294ea service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be1283c9bf47424c9aa3ad27cab03027', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6de1f3cac02c4bcdb05dd2337176033d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 11:07:04.082660 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53e2971e-db93-4ba7-b036-7e32e55294ea service neutron] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 11:07:04.082963 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53e2971e-db93-4ba7-b036-7e32e55294ea service neutron] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 11:07:04.083241 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53e2971e-db93-4ba7-b036-7e32e55294ea service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 11:07:04.093382 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53e2971e-db93-4ba7-b036-7e32e55294ea service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 11:07:04.094796 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53e2971e-db93-4ba7-b036-7e32e55294ea service neutron] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 11:07:04.095035 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7898305b-9700-44db-9079-3b9b4d353e67 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f297e44655a2492fba71d79c44a7e7f9, user_id=c74b79cc5bc74154963f3e6ca7ec5393, role_id=81b1f1bcee2a473088fbbd327c1c5b23)` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 11:07:04.097770 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7898305b-9700-44db-9079-3b9b4d353e67 admin admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 11:07:04.104604 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1407/2842] 10.222.0.25 () {60 vars in 1365 bytes} [Sun Aug 7 11:07:04 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 07 11:07:04.112200 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 1436/2843] 10.222.0.25 () {60 vars in 1483 bytes} [Sun Aug 7 11:07:04 2022] PUT /identity/v3/projects/f297e44655a2492fba71d79c44a7e7f9/users/c74b79cc5bc74154963f3e6ca7ec5393/roles/81b1f1bcee2a473088fbbd327c1c5b23 => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 07 11:07:04.116215 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53e2971e-db93-4ba7-b036-7e32e55294ea service neutron] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 11:07:04.122658 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 11:07:04.126686 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53e2971e-db93-4ba7-b036-7e32e55294ea service neutron] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 11:07:04.188652 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-155969c0-5d08-45ab-b5a0-8461ea43bb26 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5ac47e3b09f43d19a23a0f457fe9e23', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 11:07:04.189263 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-155969c0-5d08-45ab-b5a0-8461ea43bb26 admin admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 11:07:04.189655 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-155969c0-5d08-45ab-b5a0-8461ea43bb26 admin admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 11:07:04.189967 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-155969c0-5d08-45ab-b5a0-8461ea43bb26 admin admin] PATH_INFO: `/v3/roles` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 11:07:04.190611 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-155969c0-5d08-45ab-b5a0-8461ea43bb26 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 11:07:04.192243 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-155969c0-5d08-45ab-b5a0-8461ea43bb26 admin admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 11:07:04.203387 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1408/2844] 10.222.0.25 () {58 vars in 1122 bytes} [Sun Aug 7 11:07:04 2022] GET /identity/v3/roles => generated 1420 bytes in 88 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 07 11:07:04.207525 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7898305b-9700-44db-9079-3b9b4d353e67 admin admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 11:07:04.219042 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7898305b-9700-44db-9079-3b9b4d353e67 admin admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 11:07:04.225411 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de88af5b-0ed6-44d5-b22c-1293f8133352 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5ac47e3b09f43d19a23a0f457fe9e23', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 11:07:04.226085 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de88af5b-0ed6-44d5-b22c-1293f8133352 admin admin] REQUEST_METHOD: `PUT` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 11:07:04.226368 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de88af5b-0ed6-44d5-b22c-1293f8133352 admin admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 11:07:04.226649 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de88af5b-0ed6-44d5-b22c-1293f8133352 admin admin] PATH_INFO: `/v3/projects/f297e44655a2492fba71d79c44a7e7f9/users/c74b79cc5bc74154963f3e6ca7ec5393/roles/81b1f1bcee2a473088fbbd327c1c5b23` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 11:07:04.229092 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de88af5b-0ed6-44d5-b22c-1293f8133352 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f297e44655a2492fba71d79c44a7e7f9, user_id=c74b79cc5bc74154963f3e6ca7ec5393, role_id=81b1f1bcee2a473088fbbd327c1c5b23)` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 11:07:04.231770 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de88af5b-0ed6-44d5-b22c-1293f8133352 admin admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 11:07:04.243475 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 1437/2845] 10.222.0.25 () {60 vars in 1483 bytes} [Sun Aug 7 11:07:04 2022] PUT /identity/v3/projects/f297e44655a2492fba71d79c44a7e7f9/users/c74b79cc5bc74154963f3e6ca7ec5393/roles/81b1f1bcee2a473088fbbd327c1c5b23 => generated 0 bytes in 36 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 07 11:07:04.255157 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67de4ac2-4231-4d24-a8b5-f248b872a617 None None] REQUEST_METHOD: `POST` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 11:07:04.255157 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67de4ac2-4231-4d24-a8b5-f248b872a617 None None] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 11:07:04.255157 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67de4ac2-4231-4d24-a8b5-f248b872a617 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 11:07:04.283738 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.auth.core [None req-67de4ac2-4231-4d24-a8b5-f248b872a617 None None] MFA Rules not processed for user `c74b79cc5bc74154963f3e6ca7ec5393`. Rule list: `[]` (Enabled: `True`). {{(pid=80143) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 11:07:04.285023 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-67de4ac2-4231-4d24-a8b5-f248b872a617 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 11:07:04.358027 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1409/2846] 10.222.0.25 () {58 vars in 958 bytes} [Sun Aug 7 11:07:04 2022] POST /identity/v3/auth/tokens => generated 3331 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 07 11:07:04.372126 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de88af5b-0ed6-44d5-b22c-1293f8133352 admin admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 11:07:04.374340 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.fernet_utils [None req-de88af5b-0ed6-44d5-b22c-1293f8133352 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80144) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 11:07:04.399990 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67de4ac2-4231-4d24-a8b5-f248b872a617 None None] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 11:07:04.401490 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-67de4ac2-4231-4d24-a8b5-f248b872a617 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 11:07:04.450627 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de88af5b-0ed6-44d5-b22c-1293f8133352 admin admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 11:07:04.454744 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-543615a5-720d-4564-ad7a-cb4148ded71d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88dbd6dfd1164994a3e16da553561910', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6de1f3cac02c4bcdb05dd2337176033d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 11:07:04.455390 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-543615a5-720d-4564-ad7a-cb4148ded71d service cinder] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 11:07:04.455736 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-543615a5-720d-4564-ad7a-cb4148ded71d service cinder] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 11:07:04.456024 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-543615a5-720d-4564-ad7a-cb4148ded71d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 11:07:04.465924 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-543615a5-720d-4564-ad7a-cb4148ded71d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 11:07:04.467376 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-543615a5-720d-4564-ad7a-cb4148ded71d service cinder] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 11:07:04.475424 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67de4ac2-4231-4d24-a8b5-f248b872a617 None None] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 11:07:04.477980 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 1438/2847] 10.222.0.25 () {60 vars in 1365 bytes} [Sun Aug 7 11:07:04 2022] GET /identity/v3/auth/tokens => generated 3331 bytes in 107 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 07 11:07:04.480258 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8639d461-1524-4bc6-8fdc-4555233ca088 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5ac47e3b09f43d19a23a0f457fe9e23', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 11:07:04.480888 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8639d461-1524-4bc6-8fdc-4555233ca088 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 11:07:04.481169 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8639d461-1524-4bc6-8fdc-4555233ca088 admin admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 11:07:04.481444 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8639d461-1524-4bc6-8fdc-4555233ca088 admin admin] PATH_INFO: `/v3/projects/b2ccc5524e134ceea843b70ac49979b1` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 11:07:04.482576 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8639d461-1524-4bc6-8fdc-4555233ca088 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b2ccc5524e134ceea843b70ac49979b1)` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 11:07:04.484797 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8639d461-1524-4bc6-8fdc-4555233ca088 admin admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 11:07:04.518217 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=98,keystone:DELETE=2 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 11:07:04.535720 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-8639d461-1524-4bc6-8fdc-4555233ca088 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b2ccc5524e134ceea843b70ac49979b1', 'request_id': 'req-8639d461-1524-4bc6-8fdc-4555233ca088'} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 11:07:04.542605 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-8639d461-1524-4bc6-8fdc-4555233ca088 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b2ccc5524e134ceea843b70ac49979b1', 'request_id': 'req-8639d461-1524-4bc6-8fdc-4555233ca088'} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 11:07:04.549377 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-8639d461-1524-4bc6-8fdc-4555233ca088 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b2ccc5524e134ceea843b70ac49979b1', 'request_id': 'req-8639d461-1524-4bc6-8fdc-4555233ca088'} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 11:07:04.566433 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-8639d461-1524-4bc6-8fdc-4555233ca088 admin admin] The token cache is being invalidate because project b2ccc5524e134ceea843b70ac49979b1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80143) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 07 11:07:04.566725 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-8639d461-1524-4bc6-8fdc-4555233ca088 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 11:07:04.567420 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-8639d461-1524-4bc6-8fdc-4555233ca088 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 11:07:04.569111 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1410/2848] 10.222.0.25 () {58 vars in 1233 bytes} [Sun Aug 7 11:07:04 2022] DELETE /identity/v3/projects/b2ccc5524e134ceea843b70ac49979b1 => generated 0 bytes in 170 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 07 11:07:04.621321 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-543615a5-720d-4564-ad7a-cb4148ded71d service cinder] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 11:07:04.632478 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-543615a5-720d-4564-ad7a-cb4148ded71d service cinder] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 11:07:04.666917 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 11:07:04.697905 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb157b73-4d37-4565-9afd-bd9fed74a812 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5ac47e3b09f43d19a23a0f457fe9e23', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 11:07:04.698574 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb157b73-4d37-4565-9afd-bd9fed74a812 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 11:07:04.698854 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb157b73-4d37-4565-9afd-bd9fed74a812 admin admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 11:07:04.699158 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb157b73-4d37-4565-9afd-bd9fed74a812 admin admin] PATH_INFO: `/v3/users/837bdd0a3db34bc98e7b0f5252463eda` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 11:07:04.700311 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb157b73-4d37-4565-9afd-bd9fed74a812 admin admin] RBAC: Authorizing `identity:delete_user(user_id=837bdd0a3db34bc98e7b0f5252463eda)` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 11:07:04.702218 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb157b73-4d37-4565-9afd-bd9fed74a812 admin admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 11:07:04.757006 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.notifications [None req-bb157b73-4d37-4565-9afd-bd9fed74a812 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '837bdd0a3db34bc98e7b0f5252463eda'} {{(pid=80144) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 11:07:04.765091 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.notifications [None req-bb157b73-4d37-4565-9afd-bd9fed74a812 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '837bdd0a3db34bc98e7b0f5252463eda'} {{(pid=80144) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 11:07:04.773564 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.notifications [None req-bb157b73-4d37-4565-9afd-bd9fed74a812 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '837bdd0a3db34bc98e7b0f5252463eda'} {{(pid=80144) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 11:07:04.774340 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.notifications [None req-bb157b73-4d37-4565-9afd-bd9fed74a812 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '837bdd0a3db34bc98e7b0f5252463eda'} {{(pid=80144) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 11:07:04.774987 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.notifications [None req-bb157b73-4d37-4565-9afd-bd9fed74a812 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '837bdd0a3db34bc98e7b0f5252463eda'} {{(pid=80144) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 11:07:04.795201 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 1439/2849] 10.222.0.25 () {58 vars in 1224 bytes} [Sun Aug 7 11:07:04 2022] DELETE /identity/v3/users/837bdd0a3db34bc98e7b0f5252463eda => generated 0 bytes in 174 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 07 11:07:04.799609 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4563ef4-d670-48f8-9582-a1f08e99c494 None None] REQUEST_METHOD: `POST` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 11:07:04.799921 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4563ef4-d670-48f8-9582-a1f08e99c494 None None] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 11:07:04.800177 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4563ef4-d670-48f8-9582-a1f08e99c494 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 11:07:04.821757 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.auth.core [None req-d4563ef4-d670-48f8-9582-a1f08e99c494 None None] MFA Rules not processed for user `bac4e6b76fe64217ac8fa44e71cffe6a`. Rule list: `[]` (Enabled: `True`). {{(pid=80143) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 11:07:04.823013 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-d4563ef4-d670-48f8-9582-a1f08e99c494 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 11:07:04.883056 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1411/2850] 10.222.0.25 () {58 vars in 958 bytes} [Sun Aug 7 11:07:04 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 07 11:07:04.892948 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb157b73-4d37-4565-9afd-bd9fed74a812 admin admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 11:07:04.894372 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.fernet_utils [None req-bb157b73-4d37-4565-9afd-bd9fed74a812 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80144) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 11:07:04.952573 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb157b73-4d37-4565-9afd-bd9fed74a812 admin admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 11:07:04.956229 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75afe219-85a5-4525-8400-e4e57944cea5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be1283c9bf47424c9aa3ad27cab03027', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6de1f3cac02c4bcdb05dd2337176033d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 11:07:04.956866 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75afe219-85a5-4525-8400-e4e57944cea5 service neutron] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 11:07:04.957145 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75afe219-85a5-4525-8400-e4e57944cea5 service neutron] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 11:07:04.957415 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75afe219-85a5-4525-8400-e4e57944cea5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 11:07:04.967322 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75afe219-85a5-4525-8400-e4e57944cea5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 11:07:04.968726 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75afe219-85a5-4525-8400-e4e57944cea5 service neutron] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 11:07:04.978373 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 1440/2851] 10.222.0.25 () {60 vars in 1365 bytes} [Sun Aug 7 11:07:04 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 07 11:07:05.246906 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4563ef4-d670-48f8-9582-a1f08e99c494 None None] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 11:07:05.248418 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-d4563ef4-d670-48f8-9582-a1f08e99c494 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 11:07:05.260430 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4563ef4-d670-48f8-9582-a1f08e99c494 None None] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 11:07:05.264963 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c66d310-b5f3-45c7-a680-6a1c5acc1985 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5ac47e3b09f43d19a23a0f457fe9e23', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 11:07:05.265589 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c66d310-b5f3-45c7-a680-6a1c5acc1985 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 11:07:05.265865 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c66d310-b5f3-45c7-a680-6a1c5acc1985 admin admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 11:07:05.266160 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c66d310-b5f3-45c7-a680-6a1c5acc1985 admin admin] PATH_INFO: `/v3/projects/b4470c12104d473cb5b5082baf138315` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 11:07:05.267297 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c66d310-b5f3-45c7-a680-6a1c5acc1985 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b4470c12104d473cb5b5082baf138315)` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 11:07:05.269208 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c66d310-b5f3-45c7-a680-6a1c5acc1985 admin admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 11:07:05.317648 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-8c66d310-b5f3-45c7-a680-6a1c5acc1985 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b4470c12104d473cb5b5082baf138315', 'request_id': 'req-8c66d310-b5f3-45c7-a680-6a1c5acc1985'} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 11:07:05.323315 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-8c66d310-b5f3-45c7-a680-6a1c5acc1985 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b4470c12104d473cb5b5082baf138315', 'request_id': 'req-8c66d310-b5f3-45c7-a680-6a1c5acc1985'} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 11:07:05.329198 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-8c66d310-b5f3-45c7-a680-6a1c5acc1985 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b4470c12104d473cb5b5082baf138315', 'request_id': 'req-8c66d310-b5f3-45c7-a680-6a1c5acc1985'} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 11:07:05.342625 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-8c66d310-b5f3-45c7-a680-6a1c5acc1985 admin admin] The token cache is being invalidate because project b4470c12104d473cb5b5082baf138315 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80143) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 07 11:07:05.342926 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-8c66d310-b5f3-45c7-a680-6a1c5acc1985 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 11:07:05.343655 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-8c66d310-b5f3-45c7-a680-6a1c5acc1985 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 11:07:05.345124 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1412/2852] 10.222.0.25 () {58 vars in 1233 bytes} [Sun Aug 7 11:07:05 2022] DELETE /identity/v3/projects/b4470c12104d473cb5b5082baf138315 => generated 0 bytes in 99 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 07 11:07:05.350279 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75afe219-85a5-4525-8400-e4e57944cea5 service neutron] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 11:07:05.360411 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75afe219-85a5-4525-8400-e4e57944cea5 service neutron] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 11:07:05.410658 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 11:07:05.419093 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab751cfa-dea9-49f7-b18a-cdfe10e11d8b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5ac47e3b09f43d19a23a0f457fe9e23', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 11:07:05.419872 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab751cfa-dea9-49f7-b18a-cdfe10e11d8b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 11:07:05.420335 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab751cfa-dea9-49f7-b18a-cdfe10e11d8b admin admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 11:07:05.420800 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab751cfa-dea9-49f7-b18a-cdfe10e11d8b admin admin] PATH_INFO: `/v3/users/c74b79cc5bc74154963f3e6ca7ec5393` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 11:07:05.422854 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab751cfa-dea9-49f7-b18a-cdfe10e11d8b admin admin] RBAC: Authorizing `identity:delete_user(user_id=c74b79cc5bc74154963f3e6ca7ec5393)` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 11:07:05.424722 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab751cfa-dea9-49f7-b18a-cdfe10e11d8b admin admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 11:07:05.470944 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.notifications [None req-ab751cfa-dea9-49f7-b18a-cdfe10e11d8b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c74b79cc5bc74154963f3e6ca7ec5393'} {{(pid=80144) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 11:07:05.477967 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.notifications [None req-ab751cfa-dea9-49f7-b18a-cdfe10e11d8b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c74b79cc5bc74154963f3e6ca7ec5393'} {{(pid=80144) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 11:07:05.485682 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.notifications [None req-ab751cfa-dea9-49f7-b18a-cdfe10e11d8b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c74b79cc5bc74154963f3e6ca7ec5393'} {{(pid=80144) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 11:07:05.486578 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.notifications [None req-ab751cfa-dea9-49f7-b18a-cdfe10e11d8b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c74b79cc5bc74154963f3e6ca7ec5393'} {{(pid=80144) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 11:07:05.487387 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.notifications [None req-ab751cfa-dea9-49f7-b18a-cdfe10e11d8b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c74b79cc5bc74154963f3e6ca7ec5393'} {{(pid=80144) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 11:07:05.504652 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 1441/2853] 10.222.0.25 () {58 vars in 1224 bytes} [Sun Aug 7 11:07:05 2022] DELETE /identity/v3/users/c74b79cc5bc74154963f3e6ca7ec5393 => generated 0 bytes in 156 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 07 11:07:05.792632 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c66d310-b5f3-45c7-a680-6a1c5acc1985 admin admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 11:07:05.794230 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-8c66d310-b5f3-45c7-a680-6a1c5acc1985 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 11:07:05.810068 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c66d310-b5f3-45c7-a680-6a1c5acc1985 admin admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 11:07:05.850898 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=86,keystone:DELETE=12,keystone:INSERT=2 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 11:07:05.871844 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fa86387-8f32-4a5d-b40d-c9b165aedfd3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5ac47e3b09f43d19a23a0f457fe9e23', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 11:07:05.872415 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fa86387-8f32-4a5d-b40d-c9b165aedfd3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 11:07:05.872699 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fa86387-8f32-4a5d-b40d-c9b165aedfd3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 11:07:05.872975 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fa86387-8f32-4a5d-b40d-c9b165aedfd3 admin admin] PATH_INFO: `/v3/projects/f297e44655a2492fba71d79c44a7e7f9` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 11:07:05.874109 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fa86387-8f32-4a5d-b40d-c9b165aedfd3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f297e44655a2492fba71d79c44a7e7f9)` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 11:07:05.876049 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fa86387-8f32-4a5d-b40d-c9b165aedfd3 admin admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 11:07:05.922060 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-0fa86387-8f32-4a5d-b40d-c9b165aedfd3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f297e44655a2492fba71d79c44a7e7f9', 'request_id': 'req-0fa86387-8f32-4a5d-b40d-c9b165aedfd3'} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 11:07:05.928666 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-0fa86387-8f32-4a5d-b40d-c9b165aedfd3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f297e44655a2492fba71d79c44a7e7f9', 'request_id': 'req-0fa86387-8f32-4a5d-b40d-c9b165aedfd3'} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 11:07:05.934851 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-0fa86387-8f32-4a5d-b40d-c9b165aedfd3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f297e44655a2492fba71d79c44a7e7f9', 'request_id': 'req-0fa86387-8f32-4a5d-b40d-c9b165aedfd3'} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 11:07:05.948223 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-0fa86387-8f32-4a5d-b40d-c9b165aedfd3 admin admin] The token cache is being invalidate because project f297e44655a2492fba71d79c44a7e7f9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80143) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 07 11:07:05.948509 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-0fa86387-8f32-4a5d-b40d-c9b165aedfd3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 11:07:05.949128 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-0fa86387-8f32-4a5d-b40d-c9b165aedfd3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 11:07:05.950686 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1413/2854] 10.222.0.25 () {58 vars in 1233 bytes} [Sun Aug 7 11:07:05 2022] DELETE /identity/v3/projects/f297e44655a2492fba71d79c44a7e7f9 => generated 0 bytes in 159 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 07 11:07:05.955603 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dd32e3f-988d-490f-8637-14f17bbdf0da None None] REQUEST_METHOD: `POST` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 11:07:05.956083 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dd32e3f-988d-490f-8637-14f17bbdf0da None None] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 11:07:05.956184 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dd32e3f-988d-490f-8637-14f17bbdf0da None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 11:07:05.975322 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.auth.core [None req-7dd32e3f-988d-490f-8637-14f17bbdf0da None None] MFA Rules not processed for user `bac4e6b76fe64217ac8fa44e71cffe6a`. Rule list: `[]` (Enabled: `True`). {{(pid=80144) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 11:07:05.976528 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.fernet_utils [None req-7dd32e3f-988d-490f-8637-14f17bbdf0da None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80144) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 11:07:06.033929 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 1442/2855] 10.222.0.25 () {58 vars in 958 bytes} [Sun Aug 7 11:07:05 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 07 11:07:06.041819 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7839a4cf-c1b1-4835-98ce-74375ddd4244 None None] REQUEST_METHOD: `POST` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 11:07:06.042103 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7839a4cf-c1b1-4835-98ce-74375ddd4244 None None] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 11:07:06.042371 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7839a4cf-c1b1-4835-98ce-74375ddd4244 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 11:07:06.061846 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.auth.core [None req-7839a4cf-c1b1-4835-98ce-74375ddd4244 None None] MFA Rules not processed for user `bac4e6b76fe64217ac8fa44e71cffe6a`. Rule list: `[]` (Enabled: `True`). {{(pid=80143) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 11:07:06.063055 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-7839a4cf-c1b1-4835-98ce-74375ddd4244 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 11:07:06.071372 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1414/2856] 10.222.0.25 () {58 vars in 958 bytes} [Sun Aug 7 11:07:06 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 07 11:07:06.075513 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dd32e3f-988d-490f-8637-14f17bbdf0da None None] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 11:07:06.086045 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dd32e3f-988d-490f-8637-14f17bbdf0da None None] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 11:07:06.090587 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a4d9a52-3dba-42f2-8900-2e6332241e95 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5ac47e3b09f43d19a23a0f457fe9e23', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 11:07:06.091228 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a4d9a52-3dba-42f2-8900-2e6332241e95 admin admin] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 11:07:06.091510 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a4d9a52-3dba-42f2-8900-2e6332241e95 admin admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 11:07:06.091782 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a4d9a52-3dba-42f2-8900-2e6332241e95 admin admin] PATH_INFO: `/v3/domains` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 11:07:06.092422 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a4d9a52-3dba-42f2-8900-2e6332241e95 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80144) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 11:07:06.092726 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a4d9a52-3dba-42f2-8900-2e6332241e95 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 11:07:06.094287 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a4d9a52-3dba-42f2-8900-2e6332241e95 admin admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 11:07:06.106481 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 1443/2857] 10.222.0.25 () {58 vars in 1153 bytes} [Sun Aug 7 11:07:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 11:07:06.111012 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7839a4cf-c1b1-4835-98ce-74375ddd4244 None None] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 11:07:06.120289 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7839a4cf-c1b1-4835-98ce-74375ddd4244 None None] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 11:07:06.124956 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7f7b949-92fd-42c1-95d4-8cbd286d7425 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5ac47e3b09f43d19a23a0f457fe9e23', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 11:07:06.125483 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7f7b949-92fd-42c1-95d4-8cbd286d7425 admin admin] REQUEST_METHOD: `POST` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 11:07:06.125761 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7f7b949-92fd-42c1-95d4-8cbd286d7425 admin admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 11:07:06.126034 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7f7b949-92fd-42c1-95d4-8cbd286d7425 admin admin] PATH_INFO: `/v3/projects` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 11:07:06.126655 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7f7b949-92fd-42c1-95d4-8cbd286d7425 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 11:07:06.128553 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7f7b949-92fd-42c1-95d4-8cbd286d7425 admin admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 11:07:06.155095 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1415/2858] 10.222.0.25 () {60 vars in 1153 bytes} [Sun Aug 7 11:07:06 2022] POST /identity/v3/projects => generated 373 bytes in 44 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 07 11:07:06.159240 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a4d9a52-3dba-42f2-8900-2e6332241e95 admin admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 11:07:06.170050 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a4d9a52-3dba-42f2-8900-2e6332241e95 admin admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 11:07:06.227620 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-203644d0-e48c-4c81-963b-dcf870d4d4ee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5ac47e3b09f43d19a23a0f457fe9e23', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 11:07:06.228208 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-203644d0-e48c-4c81-963b-dcf870d4d4ee admin admin] REQUEST_METHOD: `POST` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 11:07:06.228516 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-203644d0-e48c-4c81-963b-dcf870d4d4ee admin admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 11:07:06.228790 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-203644d0-e48c-4c81-963b-dcf870d4d4ee admin admin] PATH_INFO: `/v3/users` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 11:07:06.229419 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-203644d0-e48c-4c81-963b-dcf870d4d4ee admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 11:07:06.231350 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-203644d0-e48c-4c81-963b-dcf870d4d4ee admin admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 11:07:06.241024 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=88,keystone:DELETE=10,keystone:INSERT=2 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 11:07:06.255803 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 1444/2859] 10.222.0.25 () {60 vars in 1144 bytes} [Sun Aug 7 11:07:06 2022] POST /identity/v3/users => generated 287 bytes in 95 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 07 11:07:06.258411 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7f7b949-92fd-42c1-95d4-8cbd286d7425 admin admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 11:07:06.269023 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7f7b949-92fd-42c1-95d4-8cbd286d7425 admin admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 11:07:06.274357 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b7ac496-d04f-4cad-82e0-7839a68128f0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5ac47e3b09f43d19a23a0f457fe9e23', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 11:07:06.274969 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b7ac496-d04f-4cad-82e0-7839a68128f0 admin admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 11:07:06.275247 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b7ac496-d04f-4cad-82e0-7839a68128f0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 11:07:06.275515 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b7ac496-d04f-4cad-82e0-7839a68128f0 admin admin] PATH_INFO: `/v3/roles` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 11:07:06.276144 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b7ac496-d04f-4cad-82e0-7839a68128f0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 11:07:06.277700 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b7ac496-d04f-4cad-82e0-7839a68128f0 admin admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 11:07:06.287689 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1416/2860] 10.222.0.25 () {58 vars in 1122 bytes} [Sun Aug 7 11:07:06 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 07 11:07:06.291767 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-203644d0-e48c-4c81-963b-dcf870d4d4ee admin admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 11:07:06.301445 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-203644d0-e48c-4c81-963b-dcf870d4d4ee admin admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 11:07:06.306532 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dc59fbf-fabc-448e-977c-099da321e123 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5ac47e3b09f43d19a23a0f457fe9e23', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 11:07:06.307158 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dc59fbf-fabc-448e-977c-099da321e123 admin admin] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 11:07:06.307438 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dc59fbf-fabc-448e-977c-099da321e123 admin admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 11:07:06.307715 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dc59fbf-fabc-448e-977c-099da321e123 admin admin] PATH_INFO: `/v3/roles` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 11:07:06.308349 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dc59fbf-fabc-448e-977c-099da321e123 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 11:07:06.309962 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dc59fbf-fabc-448e-977c-099da321e123 admin admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 11:07:06.321264 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 1445/2861] 10.222.0.25 () {58 vars in 1122 bytes} [Sun Aug 7 11:07:06 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 07 11:07:06.325383 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b7ac496-d04f-4cad-82e0-7839a68128f0 admin admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 11:07:06.335797 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b7ac496-d04f-4cad-82e0-7839a68128f0 admin admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 11:07:06.340349 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92c1d6da-b455-4a49-8200-0345070a8ca2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5ac47e3b09f43d19a23a0f457fe9e23', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 11:07:06.340982 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92c1d6da-b455-4a49-8200-0345070a8ca2 admin admin] REQUEST_METHOD: `PUT` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 11:07:06.341258 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92c1d6da-b455-4a49-8200-0345070a8ca2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 11:07:06.341530 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92c1d6da-b455-4a49-8200-0345070a8ca2 admin admin] PATH_INFO: `/v3/projects/9214e95972d2407aad2d54d0ab813bf0/users/261e7684de1443b2a9e67c2ae7b3f512/roles/b98dfb3b9fee4644af2887a3ac3fd2e6` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 11:07:06.355592 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92c1d6da-b455-4a49-8200-0345070a8ca2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9214e95972d2407aad2d54d0ab813bf0, user_id=261e7684de1443b2a9e67c2ae7b3f512, role_id=b98dfb3b9fee4644af2887a3ac3fd2e6)` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 11:07:06.358189 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92c1d6da-b455-4a49-8200-0345070a8ca2 admin admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 11:07:06.371968 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1417/2862] 10.222.0.25 () {60 vars in 1483 bytes} [Sun Aug 7 11:07:06 2022] PUT /identity/v3/projects/9214e95972d2407aad2d54d0ab813bf0/users/261e7684de1443b2a9e67c2ae7b3f512/roles/b98dfb3b9fee4644af2887a3ac3fd2e6 => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 07 11:07:06.376969 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06e25c4b-2b4a-403f-ab7c-6037f0bf1525 None None] REQUEST_METHOD: `POST` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 11:07:06.377194 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06e25c4b-2b4a-403f-ab7c-6037f0bf1525 None None] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 11:07:06.377468 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06e25c4b-2b4a-403f-ab7c-6037f0bf1525 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 11:07:06.397747 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.auth.core [None req-06e25c4b-2b4a-403f-ab7c-6037f0bf1525 None None] MFA Rules not processed for user `bac4e6b76fe64217ac8fa44e71cffe6a`. Rule list: `[]` (Enabled: `True`). {{(pid=80144) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 11:07:06.399014 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.fernet_utils [None req-06e25c4b-2b4a-403f-ab7c-6037f0bf1525 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80144) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 11:07:06.457271 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 1446/2863] 10.222.0.25 () {58 vars in 958 bytes} [Sun Aug 7 11:07:06 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 07 11:07:06.467414 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92c1d6da-b455-4a49-8200-0345070a8ca2 admin admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 11:07:06.469134 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-92c1d6da-b455-4a49-8200-0345070a8ca2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 11:07:06.526530 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92c1d6da-b455-4a49-8200-0345070a8ca2 admin admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 11:07:06.530176 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b36f045-86c3-4438-ba7d-73dba3e101b2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be1283c9bf47424c9aa3ad27cab03027', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6de1f3cac02c4bcdb05dd2337176033d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 11:07:06.530778 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b36f045-86c3-4438-ba7d-73dba3e101b2 service neutron] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 11:07:06.531293 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b36f045-86c3-4438-ba7d-73dba3e101b2 service neutron] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 11:07:06.531581 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b36f045-86c3-4438-ba7d-73dba3e101b2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 11:07:06.541107 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b36f045-86c3-4438-ba7d-73dba3e101b2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 11:07:06.542553 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b36f045-86c3-4438-ba7d-73dba3e101b2 service neutron] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 11:07:06.551834 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1418/2864] 10.222.0.25 () {60 vars in 1365 bytes} [Sun Aug 7 11:07:06 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 07 11:07:07.160054 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-257a7dbf-b80e-4121-a70b-7c609ae19200 None None] REQUEST_METHOD: `POST` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 11:07:07.160595 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-257a7dbf-b80e-4121-a70b-7c609ae19200 None None] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 11:07:07.160699 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-257a7dbf-b80e-4121-a70b-7c609ae19200 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 11:07:07.189009 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.auth.core [None req-257a7dbf-b80e-4121-a70b-7c609ae19200 None None] MFA Rules not processed for user `261e7684de1443b2a9e67c2ae7b3f512`. Rule list: `[]` (Enabled: `True`). {{(pid=80144) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 11:07:07.190354 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.fernet_utils [None req-257a7dbf-b80e-4121-a70b-7c609ae19200 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80144) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 11:07:07.260172 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 1447/2865] 10.222.0.25 () {58 vars in 958 bytes} [Sun Aug 7 11:07:07 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 07 11:07:07.264783 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b36f045-86c3-4438-ba7d-73dba3e101b2 service neutron] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 11:07:07.275479 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b36f045-86c3-4438-ba7d-73dba3e101b2 service neutron] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 11:07:07.279951 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd8735ea-c69d-4538-9024-ac582191c008 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5ac47e3b09f43d19a23a0f457fe9e23', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 11:07:07.280540 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd8735ea-c69d-4538-9024-ac582191c008 admin admin] REQUEST_METHOD: `POST` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 11:07:07.280818 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd8735ea-c69d-4538-9024-ac582191c008 admin admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 11:07:07.281115 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd8735ea-c69d-4538-9024-ac582191c008 admin admin] PATH_INFO: `/v3/projects` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 11:07:07.281730 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd8735ea-c69d-4538-9024-ac582191c008 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 11:07:07.283643 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd8735ea-c69d-4538-9024-ac582191c008 admin admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 11:07:07.289357 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 11:07:07.313991 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1419/2866] 10.222.0.25 () {60 vars in 1153 bytes} [Sun Aug 7 11:07:07 2022] POST /identity/v3/projects => generated 373 bytes in 50 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 07 11:07:07.318383 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-257a7dbf-b80e-4121-a70b-7c609ae19200 None None] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 11:07:07.329586 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-257a7dbf-b80e-4121-a70b-7c609ae19200 None None] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 11:07:07.387956 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d50b57d8-7867-43d9-bb51-e7d5a59f486c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5ac47e3b09f43d19a23a0f457fe9e23', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 11:07:07.388614 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d50b57d8-7867-43d9-bb51-e7d5a59f486c admin admin] REQUEST_METHOD: `POST` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 11:07:07.388896 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d50b57d8-7867-43d9-bb51-e7d5a59f486c admin admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 11:07:07.389169 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d50b57d8-7867-43d9-bb51-e7d5a59f486c admin admin] PATH_INFO: `/v3/users` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 11:07:07.389794 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d50b57d8-7867-43d9-bb51-e7d5a59f486c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 11:07:07.391701 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d50b57d8-7867-43d9-bb51-e7d5a59f486c admin admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 11:07:07.412341 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:INSERT=5,keystone:SELECT=95 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 11:07:07.416689 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 1448/2867] 10.222.0.25 () {60 vars in 1144 bytes} [Sun Aug 7 11:07:07 2022] POST /identity/v3/users => generated 293 bytes in 99 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 07 11:07:07.417749 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd8735ea-c69d-4538-9024-ac582191c008 admin admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 11:07:07.428065 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd8735ea-c69d-4538-9024-ac582191c008 admin admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 11:07:07.433243 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edbc4bf1-6c18-473e-8042-09ecc664b342 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5ac47e3b09f43d19a23a0f457fe9e23', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 11:07:07.433839 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edbc4bf1-6c18-473e-8042-09ecc664b342 admin admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 11:07:07.434118 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edbc4bf1-6c18-473e-8042-09ecc664b342 admin admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 11:07:07.434393 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edbc4bf1-6c18-473e-8042-09ecc664b342 admin admin] PATH_INFO: `/v3/roles` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 11:07:07.435061 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edbc4bf1-6c18-473e-8042-09ecc664b342 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 11:07:07.436633 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edbc4bf1-6c18-473e-8042-09ecc664b342 admin admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 11:07:07.447528 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1420/2868] 10.222.0.25 () {58 vars in 1122 bytes} [Sun Aug 7 11:07:07 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 07 11:07:07.451617 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d50b57d8-7867-43d9-bb51-e7d5a59f486c admin admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 11:07:07.461527 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d50b57d8-7867-43d9-bb51-e7d5a59f486c admin admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 11:07:07.466037 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20a6d38a-8172-4097-9327-9b92fa46c2b9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5ac47e3b09f43d19a23a0f457fe9e23', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 11:07:07.466680 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20a6d38a-8172-4097-9327-9b92fa46c2b9 admin admin] REQUEST_METHOD: `PUT` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 11:07:07.466988 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20a6d38a-8172-4097-9327-9b92fa46c2b9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 11:07:07.467266 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20a6d38a-8172-4097-9327-9b92fa46c2b9 admin admin] PATH_INFO: `/v3/projects/a648afd106d74548aaa58a80f644c62a/users/1157877a15254a208cbac86a3a47f64b/roles/81b1f1bcee2a473088fbbd327c1c5b23` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 11:07:07.481989 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20a6d38a-8172-4097-9327-9b92fa46c2b9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a648afd106d74548aaa58a80f644c62a, user_id=1157877a15254a208cbac86a3a47f64b, role_id=81b1f1bcee2a473088fbbd327c1c5b23)` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 11:07:07.485321 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20a6d38a-8172-4097-9327-9b92fa46c2b9 admin admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 11:07:07.498492 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 1449/2869] 10.222.0.25 () {60 vars in 1483 bytes} [Sun Aug 7 11:07:07 2022] PUT /identity/v3/projects/a648afd106d74548aaa58a80f644c62a/users/1157877a15254a208cbac86a3a47f64b/roles/81b1f1bcee2a473088fbbd327c1c5b23 => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 07 11:07:07.502525 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edbc4bf1-6c18-473e-8042-09ecc664b342 admin admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 11:07:07.512879 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edbc4bf1-6c18-473e-8042-09ecc664b342 admin admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 11:07:07.567351 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd1062fd-88d0-40a6-bf22-07f8f696cbff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5ac47e3b09f43d19a23a0f457fe9e23', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 11:07:07.567877 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd1062fd-88d0-40a6-bf22-07f8f696cbff admin admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 11:07:07.568155 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd1062fd-88d0-40a6-bf22-07f8f696cbff admin admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 11:07:07.568424 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd1062fd-88d0-40a6-bf22-07f8f696cbff admin admin] PATH_INFO: `/v3/roles` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 11:07:07.569053 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd1062fd-88d0-40a6-bf22-07f8f696cbff admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 11:07:07.571273 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd1062fd-88d0-40a6-bf22-07f8f696cbff admin admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 11:07:07.580595 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1421/2870] 10.222.0.25 () {58 vars in 1122 bytes} [Sun Aug 7 11:07:07 2022] GET /identity/v3/roles => generated 1420 bytes in 79 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 07 11:07:07.584718 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20a6d38a-8172-4097-9327-9b92fa46c2b9 admin admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 11:07:07.595695 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20a6d38a-8172-4097-9327-9b92fa46c2b9 admin admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 11:07:07.600486 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b6091b6-695f-490d-9763-b7e9ea2c0660 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5ac47e3b09f43d19a23a0f457fe9e23', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 11:07:07.601224 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b6091b6-695f-490d-9763-b7e9ea2c0660 admin admin] REQUEST_METHOD: `PUT` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 11:07:07.601507 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b6091b6-695f-490d-9763-b7e9ea2c0660 admin admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 11:07:07.601784 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b6091b6-695f-490d-9763-b7e9ea2c0660 admin admin] PATH_INFO: `/v3/projects/a648afd106d74548aaa58a80f644c62a/users/1157877a15254a208cbac86a3a47f64b/roles/81b1f1bcee2a473088fbbd327c1c5b23` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 11:07:07.603758 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b6091b6-695f-490d-9763-b7e9ea2c0660 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a648afd106d74548aaa58a80f644c62a, user_id=1157877a15254a208cbac86a3a47f64b, role_id=81b1f1bcee2a473088fbbd327c1c5b23)` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 11:07:07.606402 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b6091b6-695f-490d-9763-b7e9ea2c0660 admin admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 11:07:07.617360 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 1450/2871] 10.222.0.25 () {60 vars in 1483 bytes} [Sun Aug 7 11:07:07 2022] PUT /identity/v3/projects/a648afd106d74548aaa58a80f644c62a/users/1157877a15254a208cbac86a3a47f64b/roles/81b1f1bcee2a473088fbbd327c1c5b23 => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 07 11:07:08.315434 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71d219d8-623a-4e79-af6e-fbe02140cd75 None None] REQUEST_METHOD: `POST` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 11:07:08.316035 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71d219d8-623a-4e79-af6e-fbe02140cd75 None None] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 11:07:08.316035 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71d219d8-623a-4e79-af6e-fbe02140cd75 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 11:07:08.345018 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.auth.core [None req-71d219d8-623a-4e79-af6e-fbe02140cd75 None None] MFA Rules not processed for user `1157877a15254a208cbac86a3a47f64b`. Rule list: `[]` (Enabled: `True`). {{(pid=80143) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 11:07:08.346245 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-71d219d8-623a-4e79-af6e-fbe02140cd75 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 11:07:08.423173 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1422/2872] 10.222.0.25 () {58 vars in 958 bytes} [Sun Aug 7 11:07:08 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 07 11:07:08.436796 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b6091b6-695f-490d-9763-b7e9ea2c0660 admin admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 11:07:08.438598 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.fernet_utils [None req-5b6091b6-695f-490d-9763-b7e9ea2c0660 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80144) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 11:07:08.496428 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b6091b6-695f-490d-9763-b7e9ea2c0660 admin admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 11:07:08.499973 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c180e10-38f1-4f49-bc1b-08518227f82c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88dbd6dfd1164994a3e16da553561910', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6de1f3cac02c4bcdb05dd2337176033d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 11:07:08.500569 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c180e10-38f1-4f49-bc1b-08518227f82c service cinder] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 11:07:08.500845 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c180e10-38f1-4f49-bc1b-08518227f82c service cinder] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 11:07:08.501117 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c180e10-38f1-4f49-bc1b-08518227f82c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 11:07:08.555995 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c180e10-38f1-4f49-bc1b-08518227f82c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 11:07:08.557403 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c180e10-38f1-4f49-bc1b-08518227f82c service cinder] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 11:07:08.566071 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 1451/2873] 10.222.0.25 () {60 vars in 1365 bytes} [Sun Aug 7 11:07:08 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 131 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 07 11:07:09.920292 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71d219d8-623a-4e79-af6e-fbe02140cd75 None None] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 11:07:09.921698 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-71d219d8-623a-4e79-af6e-fbe02140cd75 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 11:07:09.934431 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71d219d8-623a-4e79-af6e-fbe02140cd75 None None] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 11:07:09.937981 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32b85339-395e-446b-a5fd-715ddd894501 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88dbd6dfd1164994a3e16da553561910', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6de1f3cac02c4bcdb05dd2337176033d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 11:07:09.938582 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32b85339-395e-446b-a5fd-715ddd894501 service cinder] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 11:07:09.938859 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32b85339-395e-446b-a5fd-715ddd894501 service cinder] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 11:07:09.939157 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32b85339-395e-446b-a5fd-715ddd894501 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 11:07:09.948552 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32b85339-395e-446b-a5fd-715ddd894501 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 11:07:09.949973 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32b85339-395e-446b-a5fd-715ddd894501 service cinder] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 11:07:09.959286 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1423/2874] 10.222.0.25 () {60 vars in 1365 bytes} [Sun Aug 7 11:07:09 2022] GET /identity/v3/auth/tokens => generated 3317 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 07 11:07:18.562123 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=63,keystone:INSERT=2 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 11:07:19.955155 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=79,keystone:INSERT=1 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 11:07:22.351512 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c180e10-38f1-4f49-bc1b-08518227f82c service cinder] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 11:07:22.353090 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.fernet_utils [None req-1c180e10-38f1-4f49-bc1b-08518227f82c service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80144) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 11:07:22.449823 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c180e10-38f1-4f49-bc1b-08518227f82c service cinder] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 11:07:22.454951 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59e768e5-c362-49fa-b03f-45f30114d229 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6ee2e4c08af04c219ed395d7ca7085a6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6de1f3cac02c4bcdb05dd2337176033d', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 11:07:22.455594 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59e768e5-c362-49fa-b03f-45f30114d229 service placement] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 11:07:22.455862 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59e768e5-c362-49fa-b03f-45f30114d229 service placement] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 11:07:22.456136 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59e768e5-c362-49fa-b03f-45f30114d229 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 11:07:22.457866 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.fernet_utils [None req-59e768e5-c362-49fa-b03f-45f30114d229 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80144) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 11:07:22.470397 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59e768e5-c362-49fa-b03f-45f30114d229 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 11:07:22.472107 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59e768e5-c362-49fa-b03f-45f30114d229 service placement] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 11:07:22.482911 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 1452/2875] 10.222.0.25 () {60 vars in 1365 bytes} [Sun Aug 7 11:07:22 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 132 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 07 11:07:22.925231 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32b85339-395e-446b-a5fd-715ddd894501 service cinder] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 11:07:22.991904 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32b85339-395e-446b-a5fd-715ddd894501 service cinder] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 11:07:22.997073 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad7ba708-0396-470d-adde-f64db01c1b6f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5ac47e3b09f43d19a23a0f457fe9e23', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 11:07:22.997717 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad7ba708-0396-470d-adde-f64db01c1b6f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 11:07:22.998001 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad7ba708-0396-470d-adde-f64db01c1b6f admin admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 11:07:22.998267 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad7ba708-0396-470d-adde-f64db01c1b6f admin admin] PATH_INFO: `/v3/users/261e7684de1443b2a9e67c2ae7b3f512` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 11:07:22.999391 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad7ba708-0396-470d-adde-f64db01c1b6f admin admin] RBAC: Authorizing `identity:delete_user(user_id=261e7684de1443b2a9e67c2ae7b3f512)` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 11:07:23.001279 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad7ba708-0396-470d-adde-f64db01c1b6f admin admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 11:07:23.046255 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-ad7ba708-0396-470d-adde-f64db01c1b6f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '261e7684de1443b2a9e67c2ae7b3f512'} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 11:07:23.053020 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-ad7ba708-0396-470d-adde-f64db01c1b6f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '261e7684de1443b2a9e67c2ae7b3f512'} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 11:07:23.066007 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-ad7ba708-0396-470d-adde-f64db01c1b6f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '261e7684de1443b2a9e67c2ae7b3f512'} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 11:07:23.066678 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-ad7ba708-0396-470d-adde-f64db01c1b6f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '261e7684de1443b2a9e67c2ae7b3f512'} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 11:07:23.073363 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-ad7ba708-0396-470d-adde-f64db01c1b6f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '261e7684de1443b2a9e67c2ae7b3f512'} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 11:07:23.079156 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1424/2876] 10.222.0.25 () {58 vars in 1224 bytes} [Sun Aug 7 11:07:22 2022] DELETE /identity/v3/users/261e7684de1443b2a9e67c2ae7b3f512 => generated 0 bytes in 155 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 07 11:07:23.248844 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59e768e5-c362-49fa-b03f-45f30114d229 service placement] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 11:07:23.259158 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59e768e5-c362-49fa-b03f-45f30114d229 service placement] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 11:07:23.319642 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d2035ef-1cae-48f8-83f3-2a387e9f1730 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5ac47e3b09f43d19a23a0f457fe9e23', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 11:07:23.320272 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d2035ef-1cae-48f8-83f3-2a387e9f1730 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 11:07:23.320550 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d2035ef-1cae-48f8-83f3-2a387e9f1730 admin admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 11:07:23.320919 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d2035ef-1cae-48f8-83f3-2a387e9f1730 admin admin] PATH_INFO: `/v3/projects/9214e95972d2407aad2d54d0ab813bf0` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 11:07:23.322205 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d2035ef-1cae-48f8-83f3-2a387e9f1730 admin admin] RBAC: Authorizing `identity:delete_project(project_id=9214e95972d2407aad2d54d0ab813bf0)` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 11:07:23.324414 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d2035ef-1cae-48f8-83f3-2a387e9f1730 admin admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 11:07:23.370705 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.notifications [None req-8d2035ef-1cae-48f8-83f3-2a387e9f1730 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9214e95972d2407aad2d54d0ab813bf0', 'request_id': 'req-8d2035ef-1cae-48f8-83f3-2a387e9f1730'} {{(pid=80144) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 11:07:23.376081 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.notifications [None req-8d2035ef-1cae-48f8-83f3-2a387e9f1730 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9214e95972d2407aad2d54d0ab813bf0', 'request_id': 'req-8d2035ef-1cae-48f8-83f3-2a387e9f1730'} {{(pid=80144) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 11:07:23.383856 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.notifications [None req-8d2035ef-1cae-48f8-83f3-2a387e9f1730 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9214e95972d2407aad2d54d0ab813bf0', 'request_id': 'req-8d2035ef-1cae-48f8-83f3-2a387e9f1730'} {{(pid=80144) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 11:07:23.394447 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.notifications [None req-8d2035ef-1cae-48f8-83f3-2a387e9f1730 admin admin] The token cache is being invalidate because project 9214e95972d2407aad2d54d0ab813bf0 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80144) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 07 11:07:23.394732 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.notifications [None req-8d2035ef-1cae-48f8-83f3-2a387e9f1730 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80144) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 11:07:23.395408 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.notifications [None req-8d2035ef-1cae-48f8-83f3-2a387e9f1730 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80144) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 11:07:23.396858 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 1453/2877] 10.222.0.25 () {58 vars in 1233 bytes} [Sun Aug 7 11:07:23 2022] DELETE /identity/v3/projects/9214e95972d2407aad2d54d0ab813bf0 => generated 0 bytes in 149 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 07 11:07:23.402225 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad7ba708-0396-470d-adde-f64db01c1b6f admin admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 11:07:23.403637 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-ad7ba708-0396-470d-adde-f64db01c1b6f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 11:07:23.416309 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad7ba708-0396-470d-adde-f64db01c1b6f admin admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 11:07:23.474031 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7bd4a3b-17a4-46f3-8984-9f7678037597 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5ac47e3b09f43d19a23a0f457fe9e23', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 11:07:23.474659 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7bd4a3b-17a4-46f3-8984-9f7678037597 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 11:07:23.474974 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7bd4a3b-17a4-46f3-8984-9f7678037597 admin admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 11:07:23.475294 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7bd4a3b-17a4-46f3-8984-9f7678037597 admin admin] PATH_INFO: `/v3/users/1157877a15254a208cbac86a3a47f64b` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 11:07:23.476379 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7bd4a3b-17a4-46f3-8984-9f7678037597 admin admin] RBAC: Authorizing `identity:delete_user(user_id=1157877a15254a208cbac86a3a47f64b)` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 11:07:23.478807 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7bd4a3b-17a4-46f3-8984-9f7678037597 admin admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 11:07:23.500472 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=86,keystone:DELETE=13,keystone:INSERT=1 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 11:07:23.525153 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-b7bd4a3b-17a4-46f3-8984-9f7678037597 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1157877a15254a208cbac86a3a47f64b'} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 11:07:23.531970 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-b7bd4a3b-17a4-46f3-8984-9f7678037597 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1157877a15254a208cbac86a3a47f64b'} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 11:07:23.543565 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-b7bd4a3b-17a4-46f3-8984-9f7678037597 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1157877a15254a208cbac86a3a47f64b'} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 11:07:23.544221 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-b7bd4a3b-17a4-46f3-8984-9f7678037597 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1157877a15254a208cbac86a3a47f64b'} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 11:07:23.551337 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-b7bd4a3b-17a4-46f3-8984-9f7678037597 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1157877a15254a208cbac86a3a47f64b'} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 11:07:23.557233 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1425/2878] 10.222.0.25 () {58 vars in 1224 bytes} [Sun Aug 7 11:07:23 2022] DELETE /identity/v3/users/1157877a15254a208cbac86a3a47f64b => generated 0 bytes in 155 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 07 11:07:23.734382 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d2035ef-1cae-48f8-83f3-2a387e9f1730 admin admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 11:07:23.735829 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.fernet_utils [None req-8d2035ef-1cae-48f8-83f3-2a387e9f1730 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80144) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 11:07:23.749264 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d2035ef-1cae-48f8-83f3-2a387e9f1730 admin admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 11:07:23.766691 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 11:07:23.807944 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae98a92b-dba4-4a75-862a-b604c37b9f9f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5ac47e3b09f43d19a23a0f457fe9e23', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 11:07:23.808577 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae98a92b-dba4-4a75-862a-b604c37b9f9f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 11:07:23.808853 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae98a92b-dba4-4a75-862a-b604c37b9f9f admin admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 11:07:23.809125 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae98a92b-dba4-4a75-862a-b604c37b9f9f admin admin] PATH_INFO: `/v3/projects/a648afd106d74548aaa58a80f644c62a` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 11:07:23.810230 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae98a92b-dba4-4a75-862a-b604c37b9f9f admin admin] RBAC: Authorizing `identity:delete_project(project_id=a648afd106d74548aaa58a80f644c62a)` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 11:07:23.812141 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae98a92b-dba4-4a75-862a-b604c37b9f9f admin admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 11:07:23.857849 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.notifications [None req-ae98a92b-dba4-4a75-862a-b604c37b9f9f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a648afd106d74548aaa58a80f644c62a', 'request_id': 'req-ae98a92b-dba4-4a75-862a-b604c37b9f9f'} {{(pid=80144) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 11:07:23.863963 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.notifications [None req-ae98a92b-dba4-4a75-862a-b604c37b9f9f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a648afd106d74548aaa58a80f644c62a', 'request_id': 'req-ae98a92b-dba4-4a75-862a-b604c37b9f9f'} {{(pid=80144) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 11:07:23.871380 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.notifications [None req-ae98a92b-dba4-4a75-862a-b604c37b9f9f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a648afd106d74548aaa58a80f644c62a', 'request_id': 'req-ae98a92b-dba4-4a75-862a-b604c37b9f9f'} {{(pid=80144) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 11:07:23.882217 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.notifications [None req-ae98a92b-dba4-4a75-862a-b604c37b9f9f admin admin] The token cache is being invalidate because project a648afd106d74548aaa58a80f644c62a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80144) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 07 11:07:23.882510 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.notifications [None req-ae98a92b-dba4-4a75-862a-b604c37b9f9f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80144) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 11:07:23.883237 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.notifications [None req-ae98a92b-dba4-4a75-862a-b604c37b9f9f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80144) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 11:07:23.885179 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 1454/2879] 10.222.0.25 () {58 vars in 1233 bytes} [Sun Aug 7 11:07:23 2022] DELETE /identity/v3/projects/a648afd106d74548aaa58a80f644c62a => generated 0 bytes in 152 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 07 11:07:23.891790 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31643496-5fdd-49eb-ad85-429112810b4e None None] REQUEST_METHOD: `POST` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 11:07:23.892074 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31643496-5fdd-49eb-ad85-429112810b4e None None] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 11:07:23.892346 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31643496-5fdd-49eb-ad85-429112810b4e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 11:07:23.912380 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.auth.core [None req-31643496-5fdd-49eb-ad85-429112810b4e None None] MFA Rules not processed for user `bac4e6b76fe64217ac8fa44e71cffe6a`. Rule list: `[]` (Enabled: `True`). {{(pid=80143) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 11:07:23.913488 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-31643496-5fdd-49eb-ad85-429112810b4e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 11:07:23.971769 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1426/2880] 10.222.0.25 () {58 vars in 958 bytes} [Sun Aug 7 11:07:23 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 07 11:07:23.980627 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c4831f7-1add-4a93-9f7e-f72d3329c959 None None] REQUEST_METHOD: `POST` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 11:07:23.980913 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c4831f7-1add-4a93-9f7e-f72d3329c959 None None] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 11:07:23.981187 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c4831f7-1add-4a93-9f7e-f72d3329c959 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 11:07:24.002326 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.auth.core [None req-7c4831f7-1add-4a93-9f7e-f72d3329c959 None None] MFA Rules not processed for user `bac4e6b76fe64217ac8fa44e71cffe6a`. Rule list: `[]` (Enabled: `True`). {{(pid=80144) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 11:07:24.003820 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.fernet_utils [None req-7c4831f7-1add-4a93-9f7e-f72d3329c959 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80144) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 11:07:24.011959 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 1455/2881] 10.222.0.25 () {58 vars in 958 bytes} [Sun Aug 7 11:07:23 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 07 11:07:24.016237 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31643496-5fdd-49eb-ad85-429112810b4e None None] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 11:07:24.026707 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31643496-5fdd-49eb-ad85-429112810b4e None None] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 11:07:24.031281 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3e416e5-06a7-4a6a-8a49-7f0269c81260 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5ac47e3b09f43d19a23a0f457fe9e23', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 11:07:24.031876 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3e416e5-06a7-4a6a-8a49-7f0269c81260 admin admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 11:07:24.032156 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3e416e5-06a7-4a6a-8a49-7f0269c81260 admin admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 11:07:24.032428 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3e416e5-06a7-4a6a-8a49-7f0269c81260 admin admin] PATH_INFO: `/v3/domains` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 11:07:24.033081 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3e416e5-06a7-4a6a-8a49-7f0269c81260 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80143) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 07 11:07:24.033382 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3e416e5-06a7-4a6a-8a49-7f0269c81260 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 11:07:24.034965 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3e416e5-06a7-4a6a-8a49-7f0269c81260 admin admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 11:07:24.047164 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1427/2882] 10.222.0.25 () {58 vars in 1153 bytes} [Sun Aug 7 11:07:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 07 11:07:24.051646 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c4831f7-1add-4a93-9f7e-f72d3329c959 None None] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 11:07:24.060960 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c4831f7-1add-4a93-9f7e-f72d3329c959 None None] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 11:07:24.065617 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec969df8-3fe5-4342-98e3-86be638699a1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5ac47e3b09f43d19a23a0f457fe9e23', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 11:07:24.066221 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec969df8-3fe5-4342-98e3-86be638699a1 admin admin] REQUEST_METHOD: `POST` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 11:07:24.066499 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec969df8-3fe5-4342-98e3-86be638699a1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 11:07:24.066772 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec969df8-3fe5-4342-98e3-86be638699a1 admin admin] PATH_INFO: `/v3/projects` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 11:07:24.067436 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec969df8-3fe5-4342-98e3-86be638699a1 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 11:07:24.069309 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec969df8-3fe5-4342-98e3-86be638699a1 admin admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 11:07:24.097698 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 1456/2883] 10.222.0.25 () {60 vars in 1153 bytes} [Sun Aug 7 11:07:24 2022] POST /identity/v3/projects => generated 379 bytes in 47 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 07 11:07:24.102727 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3e416e5-06a7-4a6a-8a49-7f0269c81260 admin admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 11:07:24.113060 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3e416e5-06a7-4a6a-8a49-7f0269c81260 admin admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 11:07:24.169693 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-006fb0ea-9d97-42c1-b3b3-f1913de5757a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5ac47e3b09f43d19a23a0f457fe9e23', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 11:07:24.170226 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-006fb0ea-9d97-42c1-b3b3-f1913de5757a admin admin] REQUEST_METHOD: `POST` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 11:07:24.170502 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-006fb0ea-9d97-42c1-b3b3-f1913de5757a admin admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 11:07:24.170775 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-006fb0ea-9d97-42c1-b3b3-f1913de5757a admin admin] PATH_INFO: `/v3/users` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 11:07:24.171436 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-006fb0ea-9d97-42c1-b3b3-f1913de5757a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 11:07:24.173325 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-006fb0ea-9d97-42c1-b3b3-f1913de5757a admin admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 11:07:24.193719 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1428/2884] 10.222.0.25 () {60 vars in 1144 bytes} [Sun Aug 7 11:07:24 2022] POST /identity/v3/users => generated 290 bytes in 92 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 07 11:07:24.198801 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec969df8-3fe5-4342-98e3-86be638699a1 admin admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 11:07:24.208799 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec969df8-3fe5-4342-98e3-86be638699a1 admin admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 11:07:24.214124 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-653cd594-04bd-4877-a3bd-c1d2cbeac0b5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5ac47e3b09f43d19a23a0f457fe9e23', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 11:07:24.214719 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-653cd594-04bd-4877-a3bd-c1d2cbeac0b5 admin admin] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 11:07:24.215023 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-653cd594-04bd-4877-a3bd-c1d2cbeac0b5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 11:07:24.215295 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-653cd594-04bd-4877-a3bd-c1d2cbeac0b5 admin admin] PATH_INFO: `/v3/roles` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 11:07:24.215993 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-653cd594-04bd-4877-a3bd-c1d2cbeac0b5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 11:07:24.217548 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-653cd594-04bd-4877-a3bd-c1d2cbeac0b5 admin admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 11:07:24.228382 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 1457/2885] 10.222.0.25 () {58 vars in 1122 bytes} [Sun Aug 7 11:07:24 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 07 11:07:24.232323 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-006fb0ea-9d97-42c1-b3b3-f1913de5757a admin admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 11:07:24.242903 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-006fb0ea-9d97-42c1-b3b3-f1913de5757a admin admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 11:07:24.247427 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-195f3ea2-4c6a-48b5-a903-7a04da07e8bf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5ac47e3b09f43d19a23a0f457fe9e23', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 11:07:24.248164 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-195f3ea2-4c6a-48b5-a903-7a04da07e8bf admin admin] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 11:07:24.248450 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-195f3ea2-4c6a-48b5-a903-7a04da07e8bf admin admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 11:07:24.248724 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-195f3ea2-4c6a-48b5-a903-7a04da07e8bf admin admin] PATH_INFO: `/v3/roles` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 11:07:24.249382 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-195f3ea2-4c6a-48b5-a903-7a04da07e8bf admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 11:07:24.250956 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-195f3ea2-4c6a-48b5-a903-7a04da07e8bf admin admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 11:07:24.261713 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1429/2886] 10.222.0.25 () {58 vars in 1122 bytes} [Sun Aug 7 11:07:24 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 07 11:07:24.265961 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-653cd594-04bd-4877-a3bd-c1d2cbeac0b5 admin admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 11:07:24.276787 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-653cd594-04bd-4877-a3bd-c1d2cbeac0b5 admin admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 11:07:24.281442 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5e25d94-5376-42f9-8b3e-a88f7162adf0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5ac47e3b09f43d19a23a0f457fe9e23', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 11:07:24.282096 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5e25d94-5376-42f9-8b3e-a88f7162adf0 admin admin] REQUEST_METHOD: `PUT` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 11:07:24.282381 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5e25d94-5376-42f9-8b3e-a88f7162adf0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 11:07:24.282658 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5e25d94-5376-42f9-8b3e-a88f7162adf0 admin admin] PATH_INFO: `/v3/projects/a100e5012adf44d793aac8838ec5b120/users/0957d7b6060648bca681160dcbc731e8/roles/b98dfb3b9fee4644af2887a3ac3fd2e6` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 11:07:24.297065 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5e25d94-5376-42f9-8b3e-a88f7162adf0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a100e5012adf44d793aac8838ec5b120, user_id=0957d7b6060648bca681160dcbc731e8, role_id=b98dfb3b9fee4644af2887a3ac3fd2e6)` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 11:07:24.299724 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5e25d94-5376-42f9-8b3e-a88f7162adf0 admin admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 11:07:24.312042 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 1458/2887] 10.222.0.25 () {60 vars in 1483 bytes} [Sun Aug 7 11:07:24 2022] PUT /identity/v3/projects/a100e5012adf44d793aac8838ec5b120/users/0957d7b6060648bca681160dcbc731e8/roles/b98dfb3b9fee4644af2887a3ac3fd2e6 => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 07 11:07:24.317162 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbc48c58-bea8-4a3e-a382-987cef914df5 None None] REQUEST_METHOD: `POST` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 11:07:24.317448 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbc48c58-bea8-4a3e-a382-987cef914df5 None None] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 11:07:24.317750 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbc48c58-bea8-4a3e-a382-987cef914df5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 11:07:24.326609 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:DELETE=7,keystone:SELECT=89,keystone:INSERT=4 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 11:07:24.339857 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.auth.core [None req-fbc48c58-bea8-4a3e-a382-987cef914df5 None None] MFA Rules not processed for user `bac4e6b76fe64217ac8fa44e71cffe6a`. Rule list: `[]` (Enabled: `True`). {{(pid=80143) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 11:07:24.341071 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-fbc48c58-bea8-4a3e-a382-987cef914df5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 11:07:24.398404 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1430/2888] 10.222.0.25 () {58 vars in 958 bytes} [Sun Aug 7 11:07:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 07 11:07:24.408632 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5e25d94-5376-42f9-8b3e-a88f7162adf0 admin admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 11:07:24.410401 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.fernet_utils [None req-a5e25d94-5376-42f9-8b3e-a88f7162adf0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80144) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 11:07:24.463278 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 11:07:24.468904 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5e25d94-5376-42f9-8b3e-a88f7162adf0 admin admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 11:07:24.473660 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb62cab7-0fd4-46d8-8d98-01b8218e9f03 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be1283c9bf47424c9aa3ad27cab03027', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6de1f3cac02c4bcdb05dd2337176033d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 11:07:24.474275 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb62cab7-0fd4-46d8-8d98-01b8218e9f03 service neutron] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 11:07:24.474559 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb62cab7-0fd4-46d8-8d98-01b8218e9f03 service neutron] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 11:07:24.474836 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb62cab7-0fd4-46d8-8d98-01b8218e9f03 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 11:07:24.484301 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb62cab7-0fd4-46d8-8d98-01b8218e9f03 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 11:07:24.485712 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb62cab7-0fd4-46d8-8d98-01b8218e9f03 service neutron] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 11:07:24.495692 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 1459/2889] 10.222.0.25 () {60 vars in 1365 bytes} [Sun Aug 7 11:07:24 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 07 11:07:28.692851 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c3416f9-2b5e-41ca-8b6e-c042d2f4d7a3 None None] REQUEST_METHOD: `POST` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 11:07:28.693384 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c3416f9-2b5e-41ca-8b6e-c042d2f4d7a3 None None] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 11:07:28.693384 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c3416f9-2b5e-41ca-8b6e-c042d2f4d7a3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 11:07:28.723023 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.auth.core [None req-6c3416f9-2b5e-41ca-8b6e-c042d2f4d7a3 None None] MFA Rules not processed for user `0957d7b6060648bca681160dcbc731e8`. Rule list: `[]` (Enabled: `True`). {{(pid=80143) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 11:07:28.724260 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-6c3416f9-2b5e-41ca-8b6e-c042d2f4d7a3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 11:07:28.802873 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1431/2890] 10.222.0.25 () {58 vars in 958 bytes} [Sun Aug 7 11:07:28 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 07 11:07:28.807643 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb62cab7-0fd4-46d8-8d98-01b8218e9f03 service neutron] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 11:07:28.817652 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb62cab7-0fd4-46d8-8d98-01b8218e9f03 service neutron] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 11:07:28.824670 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cad1d239-cd22-4831-aa1f-a1b4ecc8cc64 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5ac47e3b09f43d19a23a0f457fe9e23', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 11:07:28.825346 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cad1d239-cd22-4831-aa1f-a1b4ecc8cc64 admin admin] REQUEST_METHOD: `POST` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 11:07:28.825630 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cad1d239-cd22-4831-aa1f-a1b4ecc8cc64 admin admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 11:07:28.825906 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cad1d239-cd22-4831-aa1f-a1b4ecc8cc64 admin admin] PATH_INFO: `/v3/projects` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 11:07:28.826642 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cad1d239-cd22-4831-aa1f-a1b4ecc8cc64 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 11:07:28.828613 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cad1d239-cd22-4831-aa1f-a1b4ecc8cc64 admin admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 11:07:28.860084 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 1460/2891] 10.222.0.25 () {60 vars in 1153 bytes} [Sun Aug 7 11:07:28 2022] POST /identity/v3/projects => generated 379 bytes in 50 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 07 11:07:28.862414 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c3416f9-2b5e-41ca-8b6e-c042d2f4d7a3 None None] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 11:07:28.874748 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c3416f9-2b5e-41ca-8b6e-c042d2f4d7a3 None None] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 11:07:28.938385 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6f18520-5741-4457-8cab-1aaf1a1a4ff4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5ac47e3b09f43d19a23a0f457fe9e23', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 11:07:28.939026 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6f18520-5741-4457-8cab-1aaf1a1a4ff4 admin admin] REQUEST_METHOD: `POST` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 11:07:28.939305 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6f18520-5741-4457-8cab-1aaf1a1a4ff4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 11:07:28.939572 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6f18520-5741-4457-8cab-1aaf1a1a4ff4 admin admin] PATH_INFO: `/v3/users` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 11:07:28.940191 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6f18520-5741-4457-8cab-1aaf1a1a4ff4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 11:07:28.942081 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6f18520-5741-4457-8cab-1aaf1a1a4ff4 admin admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 11:07:28.964086 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1432/2892] 10.222.0.25 () {60 vars in 1144 bytes} [Sun Aug 7 11:07:28 2022] POST /identity/v3/users => generated 296 bytes in 102 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 07 11:07:28.968367 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cad1d239-cd22-4831-aa1f-a1b4ecc8cc64 admin admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 11:07:28.978884 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cad1d239-cd22-4831-aa1f-a1b4ecc8cc64 admin admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 11:07:28.983352 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4324d0b1-1c1a-47b7-96b7-5d006145f60c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5ac47e3b09f43d19a23a0f457fe9e23', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 11:07:28.983950 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4324d0b1-1c1a-47b7-96b7-5d006145f60c admin admin] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 11:07:28.984492 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4324d0b1-1c1a-47b7-96b7-5d006145f60c admin admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 11:07:28.984767 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4324d0b1-1c1a-47b7-96b7-5d006145f60c admin admin] PATH_INFO: `/v3/roles` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 11:07:28.985404 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4324d0b1-1c1a-47b7-96b7-5d006145f60c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 11:07:28.987011 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4324d0b1-1c1a-47b7-96b7-5d006145f60c admin admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 11:07:28.998555 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 1461/2893] 10.222.0.25 () {58 vars in 1122 bytes} [Sun Aug 7 11:07:28 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 07 11:07:29.002942 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6f18520-5741-4457-8cab-1aaf1a1a4ff4 admin admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 11:07:29.013438 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6f18520-5741-4457-8cab-1aaf1a1a4ff4 admin admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 11:07:29.017882 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ded982d-f205-486c-b031-2c0a0fa779e4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5ac47e3b09f43d19a23a0f457fe9e23', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 11:07:29.018522 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ded982d-f205-486c-b031-2c0a0fa779e4 admin admin] REQUEST_METHOD: `PUT` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 11:07:29.018826 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ded982d-f205-486c-b031-2c0a0fa779e4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 11:07:29.019107 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ded982d-f205-486c-b031-2c0a0fa779e4 admin admin] PATH_INFO: `/v3/projects/41f48911f8e140dfb5eaf539c4238146/users/e25b7a7602124d47a9ca883a6e7cc163/roles/81b1f1bcee2a473088fbbd327c1c5b23` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 11:07:29.033385 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ded982d-f205-486c-b031-2c0a0fa779e4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=41f48911f8e140dfb5eaf539c4238146, user_id=e25b7a7602124d47a9ca883a6e7cc163, role_id=81b1f1bcee2a473088fbbd327c1c5b23)` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 11:07:29.036025 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ded982d-f205-486c-b031-2c0a0fa779e4 admin admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 11:07:29.044508 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 11:07:29.049671 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1433/2894] 10.222.0.25 () {60 vars in 1483 bytes} [Sun Aug 7 11:07:29 2022] PUT /identity/v3/projects/41f48911f8e140dfb5eaf539c4238146/users/e25b7a7602124d47a9ca883a6e7cc163/roles/81b1f1bcee2a473088fbbd327c1c5b23 => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 07 11:07:29.053715 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4324d0b1-1c1a-47b7-96b7-5d006145f60c admin admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 11:07:29.064463 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4324d0b1-1c1a-47b7-96b7-5d006145f60c admin admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 11:07:29.121702 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab658a19-8781-4cff-a1dc-f83e432eac59 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5ac47e3b09f43d19a23a0f457fe9e23', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 11:07:29.122310 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab658a19-8781-4cff-a1dc-f83e432eac59 admin admin] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 11:07:29.122593 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab658a19-8781-4cff-a1dc-f83e432eac59 admin admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 11:07:29.122870 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab658a19-8781-4cff-a1dc-f83e432eac59 admin admin] PATH_INFO: `/v3/roles` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 11:07:29.123543 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab658a19-8781-4cff-a1dc-f83e432eac59 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 11:07:29.125127 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab658a19-8781-4cff-a1dc-f83e432eac59 admin admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 11:07:29.135339 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 1462/2895] 10.222.0.25 () {58 vars in 1122 bytes} [Sun Aug 7 11:07:29 2022] GET /identity/v3/roles => generated 1420 bytes in 82 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 07 11:07:29.139679 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ded982d-f205-486c-b031-2c0a0fa779e4 admin admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 11:07:29.150398 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ded982d-f205-486c-b031-2c0a0fa779e4 admin admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 11:07:29.154952 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aeb1833-5ec3-4709-89a5-3c86a7520c09 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5ac47e3b09f43d19a23a0f457fe9e23', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 11:07:29.155606 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aeb1833-5ec3-4709-89a5-3c86a7520c09 admin admin] REQUEST_METHOD: `PUT` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 11:07:29.155884 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aeb1833-5ec3-4709-89a5-3c86a7520c09 admin admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 11:07:29.156158 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aeb1833-5ec3-4709-89a5-3c86a7520c09 admin admin] PATH_INFO: `/v3/projects/41f48911f8e140dfb5eaf539c4238146/users/e25b7a7602124d47a9ca883a6e7cc163/roles/81b1f1bcee2a473088fbbd327c1c5b23` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 11:07:29.158061 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4aeb1833-5ec3-4709-89a5-3c86a7520c09 admin admin] RBAC: Authorizing `identity:create_grant(project_id=41f48911f8e140dfb5eaf539c4238146, user_id=e25b7a7602124d47a9ca883a6e7cc163, role_id=81b1f1bcee2a473088fbbd327c1c5b23)` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 11:07:29.160738 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4aeb1833-5ec3-4709-89a5-3c86a7520c09 admin admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 11:07:29.171097 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1434/2896] 10.222.0.25 () {60 vars in 1483 bytes} [Sun Aug 7 11:07:29 2022] PUT /identity/v3/projects/41f48911f8e140dfb5eaf539c4238146/users/e25b7a7602124d47a9ca883a6e7cc163/roles/81b1f1bcee2a473088fbbd327c1c5b23 => generated 0 bytes in 32 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 07 11:07:34.014403 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4185109-edbb-4ce1-a483-3c8a8d1afcfb None None] REQUEST_METHOD: `POST` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 11:07:34.015095 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4185109-edbb-4ce1-a483-3c8a8d1afcfb None None] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 11:07:34.016500 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4185109-edbb-4ce1-a483-3c8a8d1afcfb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 11:07:34.052131 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.auth.core [None req-c4185109-edbb-4ce1-a483-3c8a8d1afcfb None None] MFA Rules not processed for user `e25b7a7602124d47a9ca883a6e7cc163`. Rule list: `[]` (Enabled: `True`). {{(pid=80144) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 11:07:34.053637 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.fernet_utils [None req-c4185109-edbb-4ce1-a483-3c8a8d1afcfb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80144) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 11:07:34.142036 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 1463/2897] 10.222.0.25 () {58 vars in 958 bytes} [Sun Aug 7 11:07:34 2022] POST /identity/v3/auth/tokens => generated 3323 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 07 11:07:34.159181 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aeb1833-5ec3-4709-89a5-3c86a7520c09 admin admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 11:07:34.161188 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-4aeb1833-5ec3-4709-89a5-3c86a7520c09 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 11:07:34.248138 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aeb1833-5ec3-4709-89a5-3c86a7520c09 admin admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 11:07:34.255062 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e03e533-ff24-4651-8e3a-439eb5cbfdab service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31d16d1b96104b4eb690398432b9829e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6de1f3cac02c4bcdb05dd2337176033d', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'reader', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 11:07:34.255697 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e03e533-ff24-4651-8e3a-439eb5cbfdab service nova] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 11:07:34.255980 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e03e533-ff24-4651-8e3a-439eb5cbfdab service nova] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 11:07:34.256254 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e03e533-ff24-4651-8e3a-439eb5cbfdab service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 11:07:34.321786 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e03e533-ff24-4651-8e3a-439eb5cbfdab service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 11:07:34.323278 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e03e533-ff24-4651-8e3a-439eb5cbfdab service nova] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 11:07:34.336801 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1435/2898] 10.222.0.25 () {60 vars in 1365 bytes} [Sun Aug 7 11:07:34 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 177 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 07 11:07:44.138105 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=82,keystone:INSERT=1 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 11:07:44.330235 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=61,keystone:INSERT=1 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 11:07:56.897453 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f83fc4e0-c588-4a5f-a720-2d4c9594374d None None] REQUEST_METHOD: `POST` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 11:07:56.897942 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f83fc4e0-c588-4a5f-a720-2d4c9594374d None None] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 11:07:56.898070 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f83fc4e0-c588-4a5f-a720-2d4c9594374d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 11:07:56.917695 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.auth.core [None req-f83fc4e0-c588-4a5f-a720-2d4c9594374d None None] MFA Rules not processed for user `0957d7b6060648bca681160dcbc731e8`. Rule list: `[]` (Enabled: `True`). {{(pid=80144) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 11:07:56.918872 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.fernet_utils [None req-f83fc4e0-c588-4a5f-a720-2d4c9594374d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80144) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 11:07:56.926322 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 1464/2899] 10.222.0.25 () {58 vars in 958 bytes} [Sun Aug 7 11:07:56 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 07 11:08:06.913134 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=3 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 11:08:14.354093 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e03e533-ff24-4651-8e3a-439eb5cbfdab service nova] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 11:08:14.355536 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-0e03e533-ff24-4651-8e3a-439eb5cbfdab service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 11:08:14.411935 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e03e533-ff24-4651-8e3a-439eb5cbfdab service nova] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 11:08:14.415654 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbe83187-e015-440c-8dca-4b423101e9fe service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be1283c9bf47424c9aa3ad27cab03027', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6de1f3cac02c4bcdb05dd2337176033d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 11:08:14.416273 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbe83187-e015-440c-8dca-4b423101e9fe service neutron] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 11:08:14.416556 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbe83187-e015-440c-8dca-4b423101e9fe service neutron] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 11:08:14.416831 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbe83187-e015-440c-8dca-4b423101e9fe service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 11:08:14.426351 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbe83187-e015-440c-8dca-4b423101e9fe service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 11:08:14.427815 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbe83187-e015-440c-8dca-4b423101e9fe service neutron] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 11:08:14.437197 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1436/2900] 10.222.0.25 () {60 vars in 1365 bytes} [Sun Aug 7 11:08:14 2022] GET /identity/v3/auth/tokens => generated 3323 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 07 11:08:24.433023 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=26 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 11:08:33.309476 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f83fc4e0-c588-4a5f-a720-2d4c9594374d None None] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 11:08:33.310863 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.fernet_utils [None req-f83fc4e0-c588-4a5f-a720-2d4c9594374d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80144) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 11:08:33.379177 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f83fc4e0-c588-4a5f-a720-2d4c9594374d None None] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 11:08:33.384151 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3c415fe-0d35-45a8-89d4-529eaa2865a0 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6ee2e4c08af04c219ed395d7ca7085a6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6de1f3cac02c4bcdb05dd2337176033d', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 11:08:33.384753 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3c415fe-0d35-45a8-89d4-529eaa2865a0 service placement] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 11:08:33.385030 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3c415fe-0d35-45a8-89d4-529eaa2865a0 service placement] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 11:08:33.385299 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3c415fe-0d35-45a8-89d4-529eaa2865a0 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 11:08:33.387013 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.fernet_utils [None req-b3c415fe-0d35-45a8-89d4-529eaa2865a0 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80144) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 11:08:33.398191 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3c415fe-0d35-45a8-89d4-529eaa2865a0 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 11:08:33.399785 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3c415fe-0d35-45a8-89d4-529eaa2865a0 service placement] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 11:08:33.409637 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 1465/2901] 10.222.0.25 () {60 vars in 1365 bytes} [Sun Aug 7 11:08:33 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 101 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 07 11:08:43.405224 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=32 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 11:09:34.163991 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbe83187-e015-440c-8dca-4b423101e9fe service neutron] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 11:09:34.172603 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbe83187-e015-440c-8dca-4b423101e9fe service neutron] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 11:09:34.176258 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2aca0ea-0875-4cd0-864c-60ed139a8fc7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be1283c9bf47424c9aa3ad27cab03027', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6de1f3cac02c4bcdb05dd2337176033d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 11:09:34.176843 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2aca0ea-0875-4cd0-864c-60ed139a8fc7 service neutron] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 11:09:34.177118 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2aca0ea-0875-4cd0-864c-60ed139a8fc7 service neutron] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 11:09:34.177386 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2aca0ea-0875-4cd0-864c-60ed139a8fc7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 11:09:34.179152 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-d2aca0ea-0875-4cd0-864c-60ed139a8fc7 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 11:09:34.188602 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2aca0ea-0875-4cd0-864c-60ed139a8fc7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 11:09:34.190083 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2aca0ea-0875-4cd0-864c-60ed139a8fc7 service neutron] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 11:09:34.198163 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1437/2902] 10.222.0.25 () {60 vars in 1365 bytes} [Sun Aug 7 11:09:34 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 35 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 07 11:09:44.194401 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=6 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 11:10:30.521936 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3c415fe-0d35-45a8-89d4-529eaa2865a0 service placement] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 11:10:30.532165 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3c415fe-0d35-45a8-89d4-529eaa2865a0 service placement] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 11:10:30.535814 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01780333-8938-4498-9788-9ec242a937de service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be1283c9bf47424c9aa3ad27cab03027', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6de1f3cac02c4bcdb05dd2337176033d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 11:10:30.536415 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01780333-8938-4498-9788-9ec242a937de service neutron] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 11:10:30.536698 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01780333-8938-4498-9788-9ec242a937de service neutron] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 11:10:30.536970 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01780333-8938-4498-9788-9ec242a937de service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 11:10:30.538749 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.fernet_utils [None req-01780333-8938-4498-9788-9ec242a937de service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80144) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 11:10:30.552461 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01780333-8938-4498-9788-9ec242a937de service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 11:10:30.553868 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01780333-8938-4498-9788-9ec242a937de service neutron] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 11:10:30.565131 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 1466/2903] 10.222.0.25 () {60 vars in 1365 bytes} [Sun Aug 7 11:10:30 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 07 11:10:31.755595 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2aca0ea-0875-4cd0-864c-60ed139a8fc7 service neutron] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 11:10:31.764840 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2aca0ea-0875-4cd0-864c-60ed139a8fc7 service neutron] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 11:10:31.768455 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1aa20e1-c662-4c25-be1b-9c2549e93850 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be1283c9bf47424c9aa3ad27cab03027', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6de1f3cac02c4bcdb05dd2337176033d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 11:10:31.769079 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1aa20e1-c662-4c25-be1b-9c2549e93850 service neutron] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 11:10:31.769362 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1aa20e1-c662-4c25-be1b-9c2549e93850 service neutron] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 11:10:31.769632 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1aa20e1-c662-4c25-be1b-9c2549e93850 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 11:10:31.771404 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-c1aa20e1-c662-4c25-be1b-9c2549e93850 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 11:10:31.785772 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1aa20e1-c662-4c25-be1b-9c2549e93850 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 11:10:31.787255 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1aa20e1-c662-4c25-be1b-9c2549e93850 service neutron] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 11:10:31.798842 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1438/2904] 10.222.0.25 () {60 vars in 1365 bytes} [Sun Aug 7 11:10:31 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 07 11:10:33.251784 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01780333-8938-4498-9788-9ec242a937de service neutron] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 11:10:33.253813 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.fernet_utils [None req-01780333-8938-4498-9788-9ec242a937de service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80144) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 11:10:33.273447 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01780333-8938-4498-9788-9ec242a937de service neutron] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 11:10:33.280554 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f52783b5-d442-4d68-896f-bb25d5bce6dc service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31d16d1b96104b4eb690398432b9829e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6de1f3cac02c4bcdb05dd2337176033d', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'reader', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 11:10:33.281180 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f52783b5-d442-4d68-896f-bb25d5bce6dc service nova] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 11:10:33.281646 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f52783b5-d442-4d68-896f-bb25d5bce6dc service nova] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 11:10:33.281920 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f52783b5-d442-4d68-896f-bb25d5bce6dc service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 11:10:33.283898 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.fernet_utils [None req-f52783b5-d442-4d68-896f-bb25d5bce6dc service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80144) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 11:10:33.301488 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f52783b5-d442-4d68-896f-bb25d5bce6dc service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 11:10:33.303233 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f52783b5-d442-4d68-896f-bb25d5bce6dc service nova] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 11:10:33.318372 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 1467/2905] 10.222.0.25 () {60 vars in 1365 bytes} [Sun Aug 7 11:10:33 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 67 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 07 11:10:35.249874 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6affbd2a-fc13-491f-acb3-cc5b7f369553 None None] REQUEST_METHOD: `POST` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 11:10:35.250351 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6affbd2a-fc13-491f-acb3-cc5b7f369553 None None] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 11:10:35.250461 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6affbd2a-fc13-491f-acb3-cc5b7f369553 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 11:10:35.269485 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.auth.core [None req-6affbd2a-fc13-491f-acb3-cc5b7f369553 None None] MFA Rules not processed for user `0957d7b6060648bca681160dcbc731e8`. Rule list: `[]` (Enabled: `True`). {{(pid=80143) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 11:10:35.270667 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-6affbd2a-fc13-491f-acb3-cc5b7f369553 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 11:10:35.277676 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1439/2906] 10.222.0.25 () {58 vars in 958 bytes} [Sun Aug 7 11:10:35 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 07 11:10:36.016509 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f52783b5-d442-4d68-896f-bb25d5bce6dc service nova] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 11:10:36.018104 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.fernet_utils [None req-f52783b5-d442-4d68-896f-bb25d5bce6dc service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80144) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 11:10:36.032774 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f52783b5-d442-4d68-896f-bb25d5bce6dc service nova] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 11:10:36.037897 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08a2516e-78de-4005-b4e8-8fb914029a84 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6ee2e4c08af04c219ed395d7ca7085a6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6de1f3cac02c4bcdb05dd2337176033d', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 11:10:36.038500 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08a2516e-78de-4005-b4e8-8fb914029a84 service placement] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 11:10:36.038776 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08a2516e-78de-4005-b4e8-8fb914029a84 service placement] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 11:10:36.039431 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08a2516e-78de-4005-b4e8-8fb914029a84 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 11:10:36.041141 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.fernet_utils [None req-08a2516e-78de-4005-b4e8-8fb914029a84 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80144) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 11:10:36.056574 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08a2516e-78de-4005-b4e8-8fb914029a84 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 11:10:36.058721 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08a2516e-78de-4005-b4e8-8fb914029a84 service placement] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 11:10:36.070944 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 1468/2907] 10.222.0.25 () {60 vars in 1365 bytes} [Sun Aug 7 11:10:36 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 55 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 07 11:10:45.264252 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=9 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 11:10:46.065107 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=18 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 11:10:50.290527 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6affbd2a-fc13-491f-acb3-cc5b7f369553 None None] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 11:10:50.306726 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6affbd2a-fc13-491f-acb3-cc5b7f369553 None None] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 11:10:50.313267 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cab24ae9-a11b-41ee-ae27-654b98ea7ec4 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31d16d1b96104b4eb690398432b9829e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6de1f3cac02c4bcdb05dd2337176033d', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'reader', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 11:10:50.313922 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cab24ae9-a11b-41ee-ae27-654b98ea7ec4 service nova] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 11:10:50.314188 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cab24ae9-a11b-41ee-ae27-654b98ea7ec4 service nova] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 11:10:50.314446 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cab24ae9-a11b-41ee-ae27-654b98ea7ec4 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 11:10:50.316219 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-cab24ae9-a11b-41ee-ae27-654b98ea7ec4 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 11:10:50.330435 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cab24ae9-a11b-41ee-ae27-654b98ea7ec4 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 11:10:50.332579 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cab24ae9-a11b-41ee-ae27-654b98ea7ec4 service nova] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 11:10:50.344220 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1440/2908] 10.222.0.25 () {60 vars in 1365 bytes} [Sun Aug 7 11:10:50 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 54 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 07 11:11:00.340181 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=6 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 11:12:40.827232 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08a2516e-78de-4005-b4e8-8fb914029a84 service placement] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 11:12:40.828767 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.fernet_utils [None req-08a2516e-78de-4005-b4e8-8fb914029a84 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80144) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 11:12:40.912708 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08a2516e-78de-4005-b4e8-8fb914029a84 service placement] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 11:12:40.917170 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a43b1f8a-f9f6-4da3-81fd-b390d271621a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88dbd6dfd1164994a3e16da553561910', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6de1f3cac02c4bcdb05dd2337176033d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 11:12:40.917770 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a43b1f8a-f9f6-4da3-81fd-b390d271621a service cinder] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 11:12:40.918052 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a43b1f8a-f9f6-4da3-81fd-b390d271621a service cinder] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 11:12:40.918329 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a43b1f8a-f9f6-4da3-81fd-b390d271621a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 11:12:40.939960 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a43b1f8a-f9f6-4da3-81fd-b390d271621a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 11:12:40.941385 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a43b1f8a-f9f6-4da3-81fd-b390d271621a service cinder] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 11:12:40.950231 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 1469/2909] 10.222.0.25 () {60 vars in 1365 bytes} [Sun Aug 7 11:12:40 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 124 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 07 11:12:50.946121 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=45 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 11:12:51.957213 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6708d199-713f-4494-b55f-f4d13b3d4088 None None] REQUEST_METHOD: `POST` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 11:12:51.957721 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6708d199-713f-4494-b55f-f4d13b3d4088 None None] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 11:12:51.957837 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6708d199-713f-4494-b55f-f4d13b3d4088 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 11:12:51.976629 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.auth.core [None req-6708d199-713f-4494-b55f-f4d13b3d4088 None None] MFA Rules not processed for user `0957d7b6060648bca681160dcbc731e8`. Rule list: `[]` (Enabled: `True`). {{(pid=80143) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 07 11:12:51.977807 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-6708d199-713f-4494-b55f-f4d13b3d4088 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 11:12:51.985373 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1441/2910] 10.222.0.25 () {58 vars in 958 bytes} [Sun Aug 7 11:12:51 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 07 11:12:52.307950 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a43b1f8a-f9f6-4da3-81fd-b390d271621a service cinder] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 11:12:52.317124 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a43b1f8a-f9f6-4da3-81fd-b390d271621a service cinder] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 11:12:52.320708 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-575ffc89-1b95-449d-88fb-16f422440e95 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be1283c9bf47424c9aa3ad27cab03027', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6de1f3cac02c4bcdb05dd2337176033d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 11:12:52.321299 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-575ffc89-1b95-449d-88fb-16f422440e95 service neutron] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 11:12:52.321578 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-575ffc89-1b95-449d-88fb-16f422440e95 service neutron] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 11:12:52.321850 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-575ffc89-1b95-449d-88fb-16f422440e95 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 11:12:52.324143 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.fernet_utils [None req-575ffc89-1b95-449d-88fb-16f422440e95 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80144) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 11:12:52.333439 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-575ffc89-1b95-449d-88fb-16f422440e95 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 11:12:52.334835 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-575ffc89-1b95-449d-88fb-16f422440e95 service neutron] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 11:12:52.343450 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 1470/2911] 10.222.0.25 () {60 vars in 1365 bytes} [Sun Aug 7 11:12:52 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 07 11:13:01.971752 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=3 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 11:13:02.339747 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=6 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 11:13:39.299498 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6708d199-713f-4494-b55f-f4d13b3d4088 None None] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 11:13:39.327918 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6708d199-713f-4494-b55f-f4d13b3d4088 None None] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 11:13:39.332878 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a15d5c22-b2e4-412f-a559-f3f7e0c40a4b service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6ee2e4c08af04c219ed395d7ca7085a6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6de1f3cac02c4bcdb05dd2337176033d', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 11:13:39.333469 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a15d5c22-b2e4-412f-a559-f3f7e0c40a4b service placement] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 11:13:39.333768 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a15d5c22-b2e4-412f-a559-f3f7e0c40a4b service placement] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 11:13:39.334046 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a15d5c22-b2e4-412f-a559-f3f7e0c40a4b service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 11:13:39.343215 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a15d5c22-b2e4-412f-a559-f3f7e0c40a4b service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 11:13:39.344773 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a15d5c22-b2e4-412f-a559-f3f7e0c40a4b service placement] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 11:13:39.354566 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1442/2912] 10.222.0.25 () {60 vars in 1365 bytes} [Sun Aug 7 11:13:39 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 56 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 07 11:13:49.350312 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=13 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 11:14:38.017302 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-575ffc89-1b95-449d-88fb-16f422440e95 service neutron] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 11:14:38.036615 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-575ffc89-1b95-449d-88fb-16f422440e95 service neutron] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 11:14:38.040271 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2594524-3239-45f6-bba8-a75a3f9f5a11 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be1283c9bf47424c9aa3ad27cab03027', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6de1f3cac02c4bcdb05dd2337176033d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 11:14:38.040870 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2594524-3239-45f6-bba8-a75a3f9f5a11 service neutron] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 11:14:38.041145 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2594524-3239-45f6-bba8-a75a3f9f5a11 service neutron] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 11:14:38.041414 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2594524-3239-45f6-bba8-a75a3f9f5a11 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 11:14:38.048699 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2594524-3239-45f6-bba8-a75a3f9f5a11 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 11:14:38.050111 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2594524-3239-45f6-bba8-a75a3f9f5a11 service neutron] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 11:14:38.058274 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 1471/2913] 10.222.0.25 () {60 vars in 1365 bytes} [Sun Aug 7 11:14:38 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 07 11:14:48.054364 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=10 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 11:15:14.611705 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a15d5c22-b2e4-412f-a559-f3f7e0c40a4b service placement] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 11:15:14.644597 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a15d5c22-b2e4-412f-a559-f3f7e0c40a4b service placement] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 11:15:14.649694 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1fca8ad-b55f-4e15-a739-513e85cb9721 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '31d16d1b96104b4eb690398432b9829e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6de1f3cac02c4bcdb05dd2337176033d', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'reader', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 11:15:14.650364 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1fca8ad-b55f-4e15-a739-513e85cb9721 service nova] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 11:15:14.650715 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1fca8ad-b55f-4e15-a739-513e85cb9721 service nova] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 11:15:14.651115 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1fca8ad-b55f-4e15-a739-513e85cb9721 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 11:15:14.653458 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-a1fca8ad-b55f-4e15-a739-513e85cb9721 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 11:15:14.680053 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1fca8ad-b55f-4e15-a739-513e85cb9721 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 11:15:14.682046 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1fca8ad-b55f-4e15-a739-513e85cb9721 service nova] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 11:15:14.694945 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1443/2914] 10.222.0.25 () {60 vars in 1365 bytes} [Sun Aug 7 11:15:14 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 07 11:15:24.689743 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=18 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 11:15:27.058718 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2594524-3239-45f6-bba8-a75a3f9f5a11 service neutron] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 11:15:27.067608 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2594524-3239-45f6-bba8-a75a3f9f5a11 service neutron] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 11:15:27.071535 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0dcee56-ded6-4fcc-ad52-8392ea3caa64 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'be1283c9bf47424c9aa3ad27cab03027', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6de1f3cac02c4bcdb05dd2337176033d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 11:15:27.072161 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0dcee56-ded6-4fcc-ad52-8392ea3caa64 service neutron] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 11:15:27.072445 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0dcee56-ded6-4fcc-ad52-8392ea3caa64 service neutron] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 11:15:27.072725 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0dcee56-ded6-4fcc-ad52-8392ea3caa64 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 11:15:27.142350 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0dcee56-ded6-4fcc-ad52-8392ea3caa64 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 11:15:27.143733 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0dcee56-ded6-4fcc-ad52-8392ea3caa64 service neutron] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 11:15:27.162718 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 1472/2915] 10.222.0.25 () {60 vars in 1365 bytes} [Sun Aug 7 11:15:27 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 104 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 07 11:15:30.009874 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1fca8ad-b55f-4e15-a739-513e85cb9721 service nova] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 11:15:30.021479 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1fca8ad-b55f-4e15-a739-513e85cb9721 service nova] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 11:15:30.026448 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-135cb561-c004-41c4-b293-07e2e25063ef service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6ee2e4c08af04c219ed395d7ca7085a6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6de1f3cac02c4bcdb05dd2337176033d', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 11:15:30.027095 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-135cb561-c004-41c4-b293-07e2e25063ef service placement] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 11:15:30.027373 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-135cb561-c004-41c4-b293-07e2e25063ef service placement] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 11:15:30.028174 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-135cb561-c004-41c4-b293-07e2e25063ef service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 11:15:30.030694 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-135cb561-c004-41c4-b293-07e2e25063ef service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 11:15:30.063940 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-135cb561-c004-41c4-b293-07e2e25063ef service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 11:15:30.066041 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-135cb561-c004-41c4-b293-07e2e25063ef service placement] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 11:15:30.085889 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1444/2916] 10.222.0.25 () {60 vars in 1365 bytes} [Sun Aug 7 11:15:30 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 76 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 07 11:15:33.194689 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0dcee56-ded6-4fcc-ad52-8392ea3caa64 service neutron] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 11:15:33.208468 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0dcee56-ded6-4fcc-ad52-8392ea3caa64 service neutron] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 11:15:33.214589 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b59b404d-6c2a-40f3-96ef-0593ce64e6ff service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6ee2e4c08af04c219ed395d7ca7085a6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6de1f3cac02c4bcdb05dd2337176033d', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 11:15:33.215891 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b59b404d-6c2a-40f3-96ef-0593ce64e6ff service placement] REQUEST_METHOD: `GET` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 11:15:33.216386 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b59b404d-6c2a-40f3-96ef-0593ce64e6ff service placement] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 11:15:33.217039 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b59b404d-6c2a-40f3-96ef-0593ce64e6ff service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 11:15:33.218870 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.fernet_utils [None req-b59b404d-6c2a-40f3-96ef-0593ce64e6ff service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80144) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 11:15:33.234593 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b59b404d-6c2a-40f3-96ef-0593ce64e6ff service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 11:15:33.236867 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b59b404d-6c2a-40f3-96ef-0593ce64e6ff service placement] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 11:15:33.253413 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-135cb561-c004-41c4-b293-07e2e25063ef service placement] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 11:15:33.260580 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 1473/2917] 10.222.0.25 () {60 vars in 1365 bytes} [Sun Aug 7 11:15:33 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 66 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 07 11:15:33.267321 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-135cb561-c004-41c4-b293-07e2e25063ef service placement] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 11:15:33.272159 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ea1171b-cd39-40aa-adfa-e0bf1873bcdd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5ac47e3b09f43d19a23a0f457fe9e23', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 11:15:33.272795 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ea1171b-cd39-40aa-adfa-e0bf1873bcdd admin admin] REQUEST_METHOD: `DELETE` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 11:15:33.273080 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ea1171b-cd39-40aa-adfa-e0bf1873bcdd admin admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 11:15:33.273357 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ea1171b-cd39-40aa-adfa-e0bf1873bcdd admin admin] PATH_INFO: `/v3/users/0957d7b6060648bca681160dcbc731e8` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 11:15:33.274735 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ea1171b-cd39-40aa-adfa-e0bf1873bcdd admin admin] RBAC: Authorizing `identity:delete_user(user_id=0957d7b6060648bca681160dcbc731e8)` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 11:15:33.276621 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ea1171b-cd39-40aa-adfa-e0bf1873bcdd admin admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 11:15:33.325938 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-7ea1171b-cd39-40aa-adfa-e0bf1873bcdd admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0957d7b6060648bca681160dcbc731e8'} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 11:15:33.332913 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-7ea1171b-cd39-40aa-adfa-e0bf1873bcdd admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0957d7b6060648bca681160dcbc731e8'} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 11:15:33.345207 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-7ea1171b-cd39-40aa-adfa-e0bf1873bcdd admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0957d7b6060648bca681160dcbc731e8'} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 11:15:33.345811 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-7ea1171b-cd39-40aa-adfa-e0bf1873bcdd admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0957d7b6060648bca681160dcbc731e8'} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 11:15:33.353235 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-7ea1171b-cd39-40aa-adfa-e0bf1873bcdd admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0957d7b6060648bca681160dcbc731e8'} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 11:15:33.359194 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1445/2918] 10.222.0.25 () {58 vars in 1224 bytes} [Sun Aug 7 11:15:33 2022] DELETE /identity/v3/users/0957d7b6060648bca681160dcbc731e8 => generated 0 bytes in 106 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 07 11:15:33.547052 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b59b404d-6c2a-40f3-96ef-0593ce64e6ff service placement] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 11:15:33.559650 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b59b404d-6c2a-40f3-96ef-0593ce64e6ff service placement] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 11:15:33.617210 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0280d2c-66b5-43d9-a90a-d783270b4b82 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5ac47e3b09f43d19a23a0f457fe9e23', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 11:15:33.617982 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0280d2c-66b5-43d9-a90a-d783270b4b82 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 11:15:33.618299 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0280d2c-66b5-43d9-a90a-d783270b4b82 admin admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 11:15:33.618599 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0280d2c-66b5-43d9-a90a-d783270b4b82 admin admin] PATH_INFO: `/v3/projects/a100e5012adf44d793aac8838ec5b120` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 11:15:33.619737 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0280d2c-66b5-43d9-a90a-d783270b4b82 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a100e5012adf44d793aac8838ec5b120)` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 11:15:33.621737 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0280d2c-66b5-43d9-a90a-d783270b4b82 admin admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 11:15:33.666466 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.notifications [None req-c0280d2c-66b5-43d9-a90a-d783270b4b82 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a100e5012adf44d793aac8838ec5b120', 'request_id': 'req-c0280d2c-66b5-43d9-a90a-d783270b4b82'} {{(pid=80144) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 11:15:33.672742 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.notifications [None req-c0280d2c-66b5-43d9-a90a-d783270b4b82 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a100e5012adf44d793aac8838ec5b120', 'request_id': 'req-c0280d2c-66b5-43d9-a90a-d783270b4b82'} {{(pid=80144) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 11:15:33.679851 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.notifications [None req-c0280d2c-66b5-43d9-a90a-d783270b4b82 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a100e5012adf44d793aac8838ec5b120', 'request_id': 'req-c0280d2c-66b5-43d9-a90a-d783270b4b82'} {{(pid=80144) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 11:15:33.680942 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 11:15:33.705151 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.notifications [None req-c0280d2c-66b5-43d9-a90a-d783270b4b82 admin admin] The token cache is being invalidate because project a100e5012adf44d793aac8838ec5b120 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80144) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 07 11:15:33.705441 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.notifications [None req-c0280d2c-66b5-43d9-a90a-d783270b4b82 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80144) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 11:15:33.706069 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.notifications [None req-c0280d2c-66b5-43d9-a90a-d783270b4b82 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80144) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 11:15:33.707918 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 1474/2919] 10.222.0.25 () {58 vars in 1233 bytes} [Sun Aug 7 11:15:33 2022] DELETE /identity/v3/projects/a100e5012adf44d793aac8838ec5b120 => generated 0 bytes in 161 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 07 11:15:33.711643 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ea1171b-cd39-40aa-adfa-e0bf1873bcdd admin admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 11:15:33.713086 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-7ea1171b-cd39-40aa-adfa-e0bf1873bcdd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 11:15:33.726405 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ea1171b-cd39-40aa-adfa-e0bf1873bcdd admin admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 11:15:33.783670 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-674e3be2-d999-44ed-91bb-36a5ccd3ae38 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5ac47e3b09f43d19a23a0f457fe9e23', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 11:15:33.784310 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-674e3be2-d999-44ed-91bb-36a5ccd3ae38 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 11:15:33.784597 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-674e3be2-d999-44ed-91bb-36a5ccd3ae38 admin admin] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 11:15:33.784876 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-674e3be2-d999-44ed-91bb-36a5ccd3ae38 admin admin] PATH_INFO: `/v3/users/e25b7a7602124d47a9ca883a6e7cc163` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 11:15:33.786015 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-674e3be2-d999-44ed-91bb-36a5ccd3ae38 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e25b7a7602124d47a9ca883a6e7cc163)` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 11:15:33.787965 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-674e3be2-d999-44ed-91bb-36a5ccd3ae38 admin admin] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 11:15:33.839368 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-674e3be2-d999-44ed-91bb-36a5ccd3ae38 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e25b7a7602124d47a9ca883a6e7cc163'} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 11:15:33.842882 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=82,keystone:DELETE=17,keystone:INSERT=1 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 11:15:33.848590 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-674e3be2-d999-44ed-91bb-36a5ccd3ae38 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e25b7a7602124d47a9ca883a6e7cc163'} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 11:15:33.863171 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-674e3be2-d999-44ed-91bb-36a5ccd3ae38 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e25b7a7602124d47a9ca883a6e7cc163'} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 11:15:33.863894 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-674e3be2-d999-44ed-91bb-36a5ccd3ae38 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e25b7a7602124d47a9ca883a6e7cc163'} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 11:15:33.872497 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.notifications [None req-674e3be2-d999-44ed-91bb-36a5ccd3ae38 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e25b7a7602124d47a9ca883a6e7cc163'} {{(pid=80143) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 11:15:33.878536 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1446/2920] 10.222.0.25 () {58 vars in 1224 bytes} [Sun Aug 7 11:15:33 2022] DELETE /identity/v3/users/e25b7a7602124d47a9ca883a6e7cc163 => generated 0 bytes in 167 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 07 11:15:34.051411 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0280d2c-66b5-43d9-a90a-d783270b4b82 admin admin] Authenticating user token {{(pid=80144) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 11:15:34.052957 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.fernet_utils [None req-c0280d2c-66b5-43d9-a90a-d783270b4b82 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80144) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 11:15:34.066972 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0280d2c-66b5-43d9-a90a-d783270b4b82 admin admin] Validating token access rules against request {{(pid=80144) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 11:15:34.136120 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00a782ff-8ec9-463d-b2ce-0cebd4e1dc8b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bac4e6b76fe64217ac8fa44e71cffe6a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd5ac47e3b09f43d19a23a0f457fe9e23', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80144) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 11:15:34.136743 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00a782ff-8ec9-463d-b2ce-0cebd4e1dc8b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 11:15:34.137616 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00a782ff-8ec9-463d-b2ce-0cebd4e1dc8b admin admin] SCRIPT_NAME: `/identity` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 11:15:34.137952 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00a782ff-8ec9-463d-b2ce-0cebd4e1dc8b admin admin] PATH_INFO: `/v3/projects/41f48911f8e140dfb5eaf539c4238146` {{(pid=80144) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 11:15:34.139191 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00a782ff-8ec9-463d-b2ce-0cebd4e1dc8b admin admin] RBAC: Authorizing `identity:delete_project(project_id=41f48911f8e140dfb5eaf539c4238146)` {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 11:15:34.141090 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00a782ff-8ec9-463d-b2ce-0cebd4e1dc8b admin admin] RBAC: Authorization granted {{(pid=80144) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 11:15:34.188535 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.notifications [None req-00a782ff-8ec9-463d-b2ce-0cebd4e1dc8b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '41f48911f8e140dfb5eaf539c4238146', 'request_id': 'req-00a782ff-8ec9-463d-b2ce-0cebd4e1dc8b'} {{(pid=80144) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 11:15:34.194053 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.notifications [None req-00a782ff-8ec9-463d-b2ce-0cebd4e1dc8b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '41f48911f8e140dfb5eaf539c4238146', 'request_id': 'req-00a782ff-8ec9-463d-b2ce-0cebd4e1dc8b'} {{(pid=80144) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 11:15:34.202757 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.notifications [None req-00a782ff-8ec9-463d-b2ce-0cebd4e1dc8b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '41f48911f8e140dfb5eaf539c4238146', 'request_id': 'req-00a782ff-8ec9-463d-b2ce-0cebd4e1dc8b'} {{(pid=80144) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 11:15:34.214316 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.notifications [None req-00a782ff-8ec9-463d-b2ce-0cebd4e1dc8b admin admin] The token cache is being invalidate because project 41f48911f8e140dfb5eaf539c4238146 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80144) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 07 11:15:34.214610 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.notifications [None req-00a782ff-8ec9-463d-b2ce-0cebd4e1dc8b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80144) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 11:15:34.215824 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG keystone.notifications [None req-00a782ff-8ec9-463d-b2ce-0cebd4e1dc8b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80144) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 07 11:15:34.217318 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: [pid: 80144|app: 0|req: 1475/2921] 10.222.0.25 () {58 vars in 1233 bytes} [Sun Aug 7 11:15:34 2022] DELETE /identity/v3/projects/41f48911f8e140dfb5eaf539c4238146 => generated 0 bytes in 167 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 07 11:15:43.874970 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=11,keystone:DELETE=3,keystone:INSERT=1 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 11:15:44.211301 dv-cinder-850320-3-iscsi devstack@keystone.service[80144]: DEBUG dbcounter [-] [80144] Writing DB stats keystone:SELECT=59,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80144) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 07 11:18:46.294295 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-674e3be2-d999-44ed-91bb-36a5ccd3ae38 admin admin] Authenticating user token {{(pid=80143) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 07 11:18:46.295740 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-674e3be2-d999-44ed-91bb-36a5ccd3ae38 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 11:18:46.364211 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-674e3be2-d999-44ed-91bb-36a5ccd3ae38 admin admin] Validating token access rules against request {{(pid=80143) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 07 11:18:46.422534 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94492958-7312-4094-bde3-1d7b2f511c64 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6ee2e4c08af04c219ed395d7ca7085a6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6de1f3cac02c4bcdb05dd2337176033d', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80143) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 07 11:18:46.423173 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94492958-7312-4094-bde3-1d7b2f511c64 service placement] REQUEST_METHOD: `GET` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 07 11:18:46.423452 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94492958-7312-4094-bde3-1d7b2f511c64 service placement] SCRIPT_NAME: `/identity` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 07 11:18:46.423722 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94492958-7312-4094-bde3-1d7b2f511c64 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80143) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 07 11:18:46.425448 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.fernet_utils [None req-94492958-7312-4094-bde3-1d7b2f511c64 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80143) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 07 11:18:46.436587 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94492958-7312-4094-bde3-1d7b2f511c64 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 07 11:18:46.438770 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94492958-7312-4094-bde3-1d7b2f511c64 service placement] RBAC: Authorization granted {{(pid=80143) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 07 11:18:46.448457 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: [pid: 80143|app: 0|req: 1447/2922] 10.222.0.25 () {60 vars in 1365 bytes} [Sun Aug 7 11:18:46 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 155 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 07 11:18:56.444279 dv-cinder-850320-3-iscsi devstack@keystone.service[80143]: DEBUG dbcounter [-] [80143] Writing DB stats keystone:SELECT=58 {{(pid=80143) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}}